[jira] [Work started] (IMPALA-7381) Prevent build failure after switching to a new CDH_BUILD_NUMBER
[ https://issues.apache.org/jira/browse/IMPALA-7381?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on IMPALA-7381 started by Fredy Wijaya. > Prevent build failure after switching to a new CDH_BUILD_NUMBER > --- > > Key: IMPALA-7381 > URL: https://issues.apache.org/jira/browse/IMPALA-7381 > Project: IMPALA > Issue Type: Improvement > Components: Infrastructure >Reporter: Fredy Wijaya >Assignee: Fredy Wijaya >Priority: Major > Fix For: Impala 3.0 > > > Switching to a new CDH_BUILD_NUMBER requires cleaning up the toolchain > directory as well as forcing Maven to update its local repository. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Commented] (IMPALA-7378) test_strict_mode failed on an ASAN build: expected "Error converting column: 5 to DOUBLE"
[
https://issues.apache.org/jira/browse/IMPALA-7378?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16564645#comment-16564645
]
Sailesh Mukil commented on IMPALA-7378:
---
[~poojanilangekar] When you say the coordinator completed the query, do you
mean that it completed it successfully? If it did, then what I mention below
_might_ be happening.
Certain queries can complete before all the fragment instances have finished
running. This is because the coordinator might have received enough row(s) to
send back to the client. Meanwhile, some fragment instances may continue
running since they're unaware that the query has completed. So, the coordinator
will send a Cancel() RPC which will stop those fragment instances eventually.
Example queries where this happens all the time are LIMIT queries.
It may be the case that the coordinator finished the query successfully from
its point of view, but then a fragment instance unaware of the query completion
hit IMPALA-7335 (or some error).
So, to summarize, even though the query may have completed successfully, you
may have hit an error, and the completion of the query may have nothing to do
with that.
> test_strict_mode failed on an ASAN build: expected "Error converting column:
> 5 to DOUBLE"
> -
>
> Key: IMPALA-7378
> URL: https://issues.apache.org/jira/browse/IMPALA-7378
> Project: IMPALA
> Issue Type: Bug
> Components: Backend
>Affects Versions: Impala 3.1.0
>Reporter: David Knupp
>Assignee: Tim Armstrong
>Priority: Critical
> Labels: broken-build
>
> *Stacktrace*
> {noformat}
> query_test/test_queries.py:159: in test_strict_mode
> self.run_test_case('QueryTest/strict-mode-abort', vector)
> common/impala_test_suite.py:420: in run_test_case
> assert False, "Expected exception: %s" % expected_str
> E AssertionError: Expected exception: Error converting column: 5 to DOUBLE
> {noformat}
> *Standard Error*
> {noformat}
> -- executing against localhost:21000
> use functional;
> SET strict_mode=1;
> SET batch_size=0;
> SET num_nodes=0;
> SET disable_codegen_rows_threshold=0;
> SET disable_codegen=False;
> SET abort_on_error=0;
> SET exec_single_node_rows_threshold=0;
> -- executing against localhost:21000
> select * from overflow;
> -- executing against localhost:21000
> use functional;
> SET strict_mode=1;
> SET batch_size=0;
> SET num_nodes=0;
> SET disable_codegen_rows_threshold=0;
> SET disable_codegen=False;
> SET abort_on_error=1;
> SET exec_single_node_rows_threshold=0;
> -- executing against localhost:21000
> select tinyint_col from overflow;
> -- executing against localhost:21000
> select smallint_col from overflow;
> -- executing against localhost:21000
> select int_col from overflow;
> -- executing against localhost:21000
> select bigint_col from overflow;
> -- executing against localhost:21000
> select float_col from overflow;
> -- executing against localhost:21000
> select double_col from overflow;
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Commented] (IMPALA-6671) Metadata operations that modify a table blocks topic updates for other unrelated operations
[
https://issues.apache.org/jira/browse/IMPALA-6671?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16564616#comment-16564616
]
bharath v commented on IMPALA-6671:
---
[~tlipcon] The idea of pushing the offending tables to the next update makes
sense to me. Couple of questions,
1. Instead of {{table.pendingWriteVersion = max(table.pendingWriteVersion,
toVersion + 1)}}, should we do something like {{table.pendingWriteVersion =
getNewVersionNumber()}} ? The model as such relies on each object having a
unique version number.
2. This is not related to your idea, but I feel the default for
MAX_NUM_SKIPPED_TOPIC_UPDATES (2) seems a bit low. A reasonable DDL load on a
single table can still easily get us into this state. Should it be increased?
Ofcourse the side effect is that the table propagation might be delayed.
> Metadata operations that modify a table blocks topic updates for other
> unrelated operations
> ---
>
> Key: IMPALA-6671
> URL: https://issues.apache.org/jira/browse/IMPALA-6671
> Project: IMPALA
> Issue Type: Bug
> Components: Catalog
>Affects Versions: Impala 2.10.0, Impala 2.11.0, Impala 2.12.0
>Reporter: Mostafa Mokhtar
>Priority: Critical
> Labels: catalog-server, perfomance
>
> Metadata operations that mutate the state of a table like "compute stats foo"
> or "alter recover partitions" block topic updates for read only operations
> against unrelated tables as "describe bar".
> Thread for blocked operation
> {code}
> "Thread-7" prio=10 tid=0x11613000 nid=0x21b3b waiting on condition
> [0x7f5f2ef52000]
>java.lang.Thread.State: WAITING (parking)
> at sun.misc.Unsafe.park(Native Method)
> - parking to wait for <0x7f6f57ff0240> (a
> java.util.concurrent.locks.ReentrantLock$NonfairSync)
> at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:834)
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireQueued(AbstractQueuedSynchronizer.java:867)
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer.acquire(AbstractQueuedSynchronizer.java:1197)
> at
> java.util.concurrent.locks.ReentrantLock$NonfairSync.lock(ReentrantLock.java:214)
> at
> java.util.concurrent.locks.ReentrantLock.lock(ReentrantLock.java:290)
> at
> org.apache.impala.catalog.CatalogServiceCatalog.addTableToCatalogDeltaHelper(CatalogServiceCatalog.java:639)
> at
> org.apache.impala.catalog.CatalogServiceCatalog.addTableToCatalogDelta(CatalogServiceCatalog.java:611)
> at
> org.apache.impala.catalog.CatalogServiceCatalog.addDatabaseToCatalogDelta(CatalogServiceCatalog.java:567)
> at
> org.apache.impala.catalog.CatalogServiceCatalog.getCatalogDelta(CatalogServiceCatalog.java:449)
> at
> org.apache.impala.service.JniCatalog.getCatalogDelta(JniCatalog.java:126)
> {code}
> Thread for blocking operation
> {code}
> "Thread-130" prio=10 tid=0x113d5800 nid=0x2499d runnable
> [0x7f5ef80d]
>java.lang.Thread.State: RUNNABLE
> at java.net.SocketInputStream.socketRead0(Native Method)
> at java.net.SocketInputStream.read(SocketInputStream.java:152)
> at java.net.SocketInputStream.read(SocketInputStream.java:122)
> at java.io.BufferedInputStream.fill(BufferedInputStream.java:235)
> at java.io.BufferedInputStream.read1(BufferedInputStream.java:275)
> at java.io.BufferedInputStream.read(BufferedInputStream.java:334)
> - locked <0x7f5fffcd9f18> (a java.io.BufferedInputStream)
> at
> org.apache.thrift.transport.TIOStreamTransport.read(TIOStreamTransport.java:127)
> at org.apache.thrift.transport.TTransport.readAll(TTransport.java:84)
> at
> org.apache.thrift.transport.TSaslTransport.readLength(TSaslTransport.java:346)
> at
> org.apache.thrift.transport.TSaslTransport.readFrame(TSaslTransport.java:423)
> at
> org.apache.thrift.transport.TSaslTransport.read(TSaslTransport.java:405)
> at
> org.apache.thrift.transport.TSaslClientTransport.read(TSaslClientTransport.java:37)
> at org.apache.thrift.transport.TTransport.readAll(TTransport.java:84)
> at
> org.apache.hadoop.hive.thrift.TFilterTransport.readAll(TFilterTransport.java:62)
> at
> org.apache.thrift.protocol.TBinaryProtocol.readAll(TBinaryProtocol.java:378)
> at
> org.apache.thrift.protocol.TBinaryProtocol.readI32(TBinaryProtocol.java:297)
> at
> org.apache.thrift.protocol.TBinaryProtocol.readMessageBegin(TBinaryProtocol.java:204)
> at
>
[jira] [Commented] (IMPALA-7335) Assertion Failure - test_corrupt_files
[
https://issues.apache.org/jira/browse/IMPALA-7335?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16564560#comment-16564560
]
Pooja Nilangekar commented on IMPALA-7335:
--
I think it is closer to IMPALA-7378. In this case, the error was logged and the
query status was set appropriately. Here is the line in the log:
{code:java}
ee_tests/impalad.ip-172-31-28-177.ubuntu.log.INFO.20180730-233654.20124:I0731
00:52:07.904188 105583 query-state.cc:403] Instance completed.
instance_id=b04c3db85fca19f5:2927aa830001 #in-flight=13
status=PARQUET_BAD_VERSION_NUMBER: File
'hdfs://localhost:20500/test-warehouse/bad_magic_number_parquet/bad_magic_number.parquet'
has an invalid version number:
{code}
However, I can't find all the logs for this query. (Specifically the
coordinator logs.)
I will look at possible race conditions where an error status might be set
without propagating it back to the coordinator.
> Assertion Failure - test_corrupt_files
> --
>
> Key: IMPALA-7335
> URL: https://issues.apache.org/jira/browse/IMPALA-7335
> Project: IMPALA
> Issue Type: Bug
>Affects Versions: Impala 3.1.0
>Reporter: nithya
>Assignee: Pooja Nilangekar
>Priority: Critical
> Labels: broken-build
>
> test_corrupt_files fails
>
> query_test.test_scanners.TestParquet.test_corrupt_files[exec_option:
> \\{'batch_size': 0, 'num_nodes': 0, 'disable_codegen_rows_threshold': 0,
> 'disable_codegen': False, 'abort_on_error': 1, 'debug_action': None,
> 'exec_single_node_rows_threshold': 0} | table_format: parquet/none] (from
> pytest)
>
> {code:java}
> Error Message
> query_test/test_scanners.py:300: in test_corrupt_files
> self.run_test_case('QueryTest/parquet-abort-on-error', vector)
> common/impala_test_suite.py:420: in run_test_case assert False, "Expected
> exception: %s" % expected_str E AssertionError: Expected exception: Column
> metadata states there are 11 values, but read 10 values from column id.
> STACKTRACE
> query_test/test_scanners.py:300: in test_corrupt_files
> self.run_test_case('QueryTest/parquet-abort-on-error', vector)
> common/impala_test_suite.py:420: in run_test_case
> assert False, "Expected exception: %s" % expected_str
> E AssertionError: Expected exception: Column metadata states there are 11
> values, but read 10 values from column id.
> Standard Error
> -- executing against localhost:21000
> use functional_parquet;
> SET batch_size=0;
> SET num_nodes=0;
> SET disable_codegen_rows_threshold=0;
> SET disable_codegen=False;
> SET abort_on_error=0;
> SET exec_single_node_rows_threshold=0;
> -- executing against localhost:21000
> set num_nodes=1;
> -- executing against localhost:21000
> set num_scanner_threads=1;
> -- executing against localhost:21000
> select id, cnt from bad_column_metadata t, (select count(*) cnt from
> t.int_array) v;
> -- executing against localhost:21000
> SET NUM_NODES="0";
> -- executing against localhost:21000
> SET NUM_SCANNER_THREADS="0";
> -- executing against localhost:21000
> set num_nodes=1;
> -- executing against localhost:21000
> set num_scanner_threads=1;
> -- executing against localhost:21000
> select id from bad_column_metadata;
> -- executing against localhost:21000
> SET NUM_NODES="0";
> -- executing against localhost:21000
> SET NUM_SCANNER_THREADS="0";
> -- executing against localhost:21000
> SELECT * from bad_parquet_strings_negative_len;
> -- executing against localhost:21000
> SELECT * from bad_parquet_strings_out_of_bounds;
> -- executing against localhost:21000
> use functional_parquet;
> SET batch_size=0;
> SET num_nodes=0;
> SET disable_codegen_rows_threshold=0;
> SET disable_codegen=False;
> SET abort_on_error=1;
> SET exec_single_node_rows_threshold=0;
> -- executing against localhost:21000
> set num_nodes=1;
> -- executing against localhost:21000
> set num_scanner_threads=1;
> -- executing against localhost:21000
> select id, cnt from bad_column_metadata t, (select count(*) cnt from
> t.int_array) v;
> -- executing against localhost:21000
> SET NUM_NODES="0";
> -- executing against localhost:21000
> SET NUM_SCANNER_THREADS="0";
> -- executing against localhost:21000
> set num_nodes=1;
> -- executing against localhost:21000
> set num_scanner_threads=1;
> -- executing against localhost:21000
> select id from bad_column_metadata;
> -- executing against localhost:21000
> SET NUM_NODES="0";
> -- executing against localhost:21000
> SET NUM_SCANNER_THREADS="0";
> {code}
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Commented] (IMPALA-7364) Upgrade RapidJson to the latest version
[ https://issues.apache.org/jira/browse/IMPALA-7364?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16564557#comment-16564557 ] Quanlong Huang commented on IMPALA-7364: Sure. Thank you! > Upgrade RapidJson to the latest version > --- > > Key: IMPALA-7364 > URL: https://issues.apache.org/jira/browse/IMPALA-7364 > Project: IMPALA > Issue Type: Improvement >Reporter: Quanlong Huang >Priority: Major > > Currently, we're using rapidjson-0.11. It's old enough and can't show > detailed parse errors. In the implementation of built-in JSON function in > IMPALA-376, we can't show warnings on how we fail to parse a json string and > where is the error location like this: > http://rapidjson.org/group___r_a_p_i_d_j_s_o_n___e_r_r_o_r_s.html#structrapidjson_1_1_parse_result > This JIRA aim to upgrade the version of RapidJson to v1.1. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Updated] (IMPALA-7377) Update Sentry for object ownership feature
[ https://issues.apache.org/jira/browse/IMPALA-7377?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Fredy Wijaya updated IMPALA-7377: - Description: The new Sentry update has a way to automatically grant owner privilege: [https://github.com/apache/sentry/commit/4d9bc9cea94f13b7994817d33c016c776fa7dcda.] In order to take advantage of that feature, Impala needs to update its Sentry dependency. > Update Sentry for object ownership feature > -- > > Key: IMPALA-7377 > URL: https://issues.apache.org/jira/browse/IMPALA-7377 > Project: IMPALA > Issue Type: Sub-task > Components: Infrastructure >Reporter: Fredy Wijaya >Assignee: Fredy Wijaya >Priority: Major > Labels: security > Fix For: Impala 3.0 > > > The new Sentry update has a way to automatically grant owner privilege: > [https://github.com/apache/sentry/commit/4d9bc9cea94f13b7994817d33c016c776fa7dcda.] > In order to take advantage of that feature, Impala needs to update its > Sentry dependency. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Updated] (IMPALA-7075) Add support for object ownership for Impala
[ https://issues.apache.org/jira/browse/IMPALA-7075?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Fredy Wijaya updated IMPALA-7075: - Description: Currently > Add support for object ownership for Impala > --- > > Key: IMPALA-7075 > URL: https://issues.apache.org/jira/browse/IMPALA-7075 > Project: IMPALA > Issue Type: Epic > Components: Frontend >Reporter: Fredy Wijaya >Assignee: Adam Holley >Priority: Major > Labels: security > > Currently -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Commented] (IMPALA-7378) test_strict_mode failed on an ASAN build: expected "Error converting column: 5 to DOUBLE"
[
https://issues.apache.org/jira/browse/IMPALA-7378?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16564495#comment-16564495
]
Pooja Nilangekar commented on IMPALA-7378:
--
[~tarmstrong] This seems weirder. In case of IMPALA-7335, the error was never
logged. However, in this case the error was generated and logged at
00:29:39.658874 and yet the coordinator completed the query at 00:29:39.741559.
[~sailesh] Could this possibly be an RPC issue?
> test_strict_mode failed on an ASAN build: expected "Error converting column:
> 5 to DOUBLE"
> -
>
> Key: IMPALA-7378
> URL: https://issues.apache.org/jira/browse/IMPALA-7378
> Project: IMPALA
> Issue Type: Bug
> Components: Backend
>Affects Versions: Impala 3.1.0
>Reporter: David Knupp
>Assignee: Tim Armstrong
>Priority: Critical
> Labels: broken-build
>
> *Stacktrace*
> {noformat}
> query_test/test_queries.py:159: in test_strict_mode
> self.run_test_case('QueryTest/strict-mode-abort', vector)
> common/impala_test_suite.py:420: in run_test_case
> assert False, "Expected exception: %s" % expected_str
> E AssertionError: Expected exception: Error converting column: 5 to DOUBLE
> {noformat}
> *Standard Error*
> {noformat}
> -- executing against localhost:21000
> use functional;
> SET strict_mode=1;
> SET batch_size=0;
> SET num_nodes=0;
> SET disable_codegen_rows_threshold=0;
> SET disable_codegen=False;
> SET abort_on_error=0;
> SET exec_single_node_rows_threshold=0;
> -- executing against localhost:21000
> select * from overflow;
> -- executing against localhost:21000
> use functional;
> SET strict_mode=1;
> SET batch_size=0;
> SET num_nodes=0;
> SET disable_codegen_rows_threshold=0;
> SET disable_codegen=False;
> SET abort_on_error=1;
> SET exec_single_node_rows_threshold=0;
> -- executing against localhost:21000
> select tinyint_col from overflow;
> -- executing against localhost:21000
> select smallint_col from overflow;
> -- executing against localhost:21000
> select int_col from overflow;
> -- executing against localhost:21000
> select bigint_col from overflow;
> -- executing against localhost:21000
> select float_col from overflow;
> -- executing against localhost:21000
> select double_col from overflow;
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Commented] (IMPALA-7379) test_random_rpc_timeout failed on exhaustive build: Debug webpage did not become available in expected time
[
https://issues.apache.org/jira/browse/IMPALA-7379?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16564492#comment-16564492
]
Michael Ho commented on IMPALA-7379:
Looks to me it's one of those failure where Impala failed to startup in time
similar to IMPALA-6642
> test_random_rpc_timeout failed on exhaustive build: Debug webpage did not
> become available in expected time
> ---
>
> Key: IMPALA-7379
> URL: https://issues.apache.org/jira/browse/IMPALA-7379
> Project: IMPALA
> Issue Type: Bug
> Components: Backend
>Affects Versions: Impala 3.1.0
>Reporter: David Knupp
>Assignee: Michael Ho
>Priority: Major
>
> *Stacktrace*
> {noformat}
> custom_cluster/test_rpc_timeout.py:131: in test_random_rpc_timeout
> self.execute_query_verify_metrics(self.TEST_QUERY, 10)
> custom_cluster/test_rpc_timeout.py:51: in execute_query_verify_metrics
> v.wait_for_metric("impala-server.num-fragments-in-flight", 0)
> verifiers/metric_verifier.py:62: in wait_for_metric
> self.impalad_service.wait_for_metric_value(metric_name, expected_value,
> timeout)
> common/impala_service.py:132: in wait_for_metric_value
> json.dumps(self.read_debug_webpage('memz?json')),
> common/impala_service.py:63: in read_debug_webpage
> return self.open_debug_webpage(page_name, timeout=timeout,
> interval=interval).read()
> common/impala_service.py:60: in open_debug_webpage
> assert 0, 'Debug webpage did not become available in expected time.'
> E AssertionError: Debug webpage did not become available in expected time.
> {noformat}
> *Standard Error*
> {noformat}
> 10:27:58 MainThread: Starting State Store logging to
> /data/jenkins/workspace/impala-asf-master-exhaustive/repos/Impala/logs/custom_cluster_tests/statestored.INFO
> 10:27:58 MainThread: Starting Catalog Service logging to
> /data/jenkins/workspace/impala-asf-master-exhaustive/repos/Impala/logs/custom_cluster_tests/catalogd.INFO
> 10:27:59 MainThread: Starting Impala Daemon logging to
> /data/jenkins/workspace/impala-asf-master-exhaustive/repos/Impala/logs/custom_cluster_tests/impalad.INFO
> 10:28:00 MainThread: Starting Impala Daemon logging to
> /data/jenkins/workspace/impala-asf-master-exhaustive/repos/Impala/logs/custom_cluster_tests/impalad_node1.INFO
> 10:28:01 MainThread: Starting Impala Daemon logging to
> /data/jenkins/workspace/impala-asf-master-exhaustive/repos/Impala/logs/custom_cluster_tests/impalad_node2.INFO
> 10:28:04 MainThread: Found 3 impalad/1 statestored/1 catalogd process(es)
> 10:28:04 MainThread: Waiting for num_known_live_backends=3. Current value: 1
> 10:28:05 MainThread: Waiting for num_known_live_backends=3. Current value: 2
> 10:28:06 MainThread: num_known_live_backends has reached value: 3
> 10:28:06 MainThread: num_known_live_backends has reached value: 3
> 10:28:06 MainThread: num_known_live_backends has reached value: 3
> 10:28:06 MainThread: Impala Cluster Running with 3 nodes (3 coordinators, 3
> executors).
> MainThread: Found 3 impalad/1 statestored/1 catalogd process(es)
> MainThread: Metric 'statestore.live-backends' has reached desired value: 4
> MainThread: num_known_live_backends has reached value: 3
> MainThread: num_known_live_backends has reached value: 3
> MainThread: num_known_live_backends has reached value: 3
> -- connecting to: localhost:21000
> -- executing against localhost:21000
> select count(c2.string_col) from functional.alltypestiny join
> functional.alltypessmall c2;
> -- executing against localhost:21000
> select count(c2.string_col) from functional.alltypestiny join
> functional.alltypessmall c2;
> -- executing against localhost:21000
> select count(c2.string_col) from functional.alltypestiny join
> functional.alltypessmall c2;
> -- executing against localhost:21000
> select count(c2.string_col) from functional.alltypestiny join
> functional.alltypessmall c2;
> -- executing against localhost:21000
> select count(c2.string_col) from functional.alltypestiny join
> functional.alltypessmall c2;
> -- executing against localhost:21000
> select count(c2.string_col) from functional.alltypestiny join
> functional.alltypessmall c2;
> -- executing against localhost:21000
> select count(c2.string_col) from functional.alltypestiny join
> functional.alltypessmall c2;
> -- executing against localhost:21000
> select count(c2.string_col) from functional.alltypestiny join
> functional.alltypessmall c2;
> -- executing against localhost:21000
> select count(c2.string_col) from functional.alltypestiny join
> functional.alltypessmall c2;
> -- executing against localhost:21000
> select count(c2.string_col) from functional.alltypestiny join
> functional.alltypessmall c2;
> MainThread: no
[jira] [Updated] (IMPALA-7379) test_random_rpc_timeout failed on exhaustive build: Debug webpage did not become available in expected time
[
https://issues.apache.org/jira/browse/IMPALA-7379?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
David Knupp updated IMPALA-7379:
Description:
*Stacktrace*
{noformat}
custom_cluster/test_rpc_timeout.py:131: in test_random_rpc_timeout
self.execute_query_verify_metrics(self.TEST_QUERY, 10)
custom_cluster/test_rpc_timeout.py:51: in execute_query_verify_metrics
v.wait_for_metric("impala-server.num-fragments-in-flight", 0)
verifiers/metric_verifier.py:62: in wait_for_metric
self.impalad_service.wait_for_metric_value(metric_name, expected_value,
timeout)
common/impala_service.py:132: in wait_for_metric_value
json.dumps(self.read_debug_webpage('memz?json')),
common/impala_service.py:63: in read_debug_webpage
return self.open_debug_webpage(page_name, timeout=timeout,
interval=interval).read()
common/impala_service.py:60: in open_debug_webpage
assert 0, 'Debug webpage did not become available in expected time.'
E AssertionError: Debug webpage did not become available in expected time.
{noformat}
*Standard Error*
{noformat}
10:27:58 MainThread: Starting State Store logging to
/data/jenkins/workspace/impala-asf-master-exhaustive/repos/Impala/logs/custom_cluster_tests/statestored.INFO
10:27:58 MainThread: Starting Catalog Service logging to
/data/jenkins/workspace/impala-asf-master-exhaustive/repos/Impala/logs/custom_cluster_tests/catalogd.INFO
10:27:59 MainThread: Starting Impala Daemon logging to
/data/jenkins/workspace/impala-asf-master-exhaustive/repos/Impala/logs/custom_cluster_tests/impalad.INFO
10:28:00 MainThread: Starting Impala Daemon logging to
/data/jenkins/workspace/impala-asf-master-exhaustive/repos/Impala/logs/custom_cluster_tests/impalad_node1.INFO
10:28:01 MainThread: Starting Impala Daemon logging to
/data/jenkins/workspace/impala-asf-master-exhaustive/repos/Impala/logs/custom_cluster_tests/impalad_node2.INFO
10:28:04 MainThread: Found 3 impalad/1 statestored/1 catalogd process(es)
10:28:04 MainThread: Waiting for num_known_live_backends=3. Current value: 1
10:28:05 MainThread: Waiting for num_known_live_backends=3. Current value: 2
10:28:06 MainThread: num_known_live_backends has reached value: 3
10:28:06 MainThread: num_known_live_backends has reached value: 3
10:28:06 MainThread: num_known_live_backends has reached value: 3
10:28:06 MainThread: Impala Cluster Running with 3 nodes (3 coordinators, 3
executors).
MainThread: Found 3 impalad/1 statestored/1 catalogd process(es)
MainThread: Metric 'statestore.live-backends' has reached desired value: 4
MainThread: num_known_live_backends has reached value: 3
MainThread: num_known_live_backends has reached value: 3
MainThread: num_known_live_backends has reached value: 3
-- connecting to: localhost:21000
-- executing against localhost:21000
select count(c2.string_col) from functional.alltypestiny join
functional.alltypessmall c2;
-- executing against localhost:21000
select count(c2.string_col) from functional.alltypestiny join
functional.alltypessmall c2;
-- executing against localhost:21000
select count(c2.string_col) from functional.alltypestiny join
functional.alltypessmall c2;
-- executing against localhost:21000
select count(c2.string_col) from functional.alltypestiny join
functional.alltypessmall c2;
-- executing against localhost:21000
select count(c2.string_col) from functional.alltypestiny join
functional.alltypessmall c2;
-- executing against localhost:21000
select count(c2.string_col) from functional.alltypestiny join
functional.alltypessmall c2;
-- executing against localhost:21000
select count(c2.string_col) from functional.alltypestiny join
functional.alltypessmall c2;
-- executing against localhost:21000
select count(c2.string_col) from functional.alltypestiny join
functional.alltypessmall c2;
-- executing against localhost:21000
select count(c2.string_col) from functional.alltypestiny join
functional.alltypessmall c2;
-- executing against localhost:21000
select count(c2.string_col) from functional.alltypestiny join
functional.alltypessmall c2;
MainThread: no process found with pid 19511
MainThread: Found 3 impalad/1 statestored/1 catalogd process(es)
MainThread: Getting metric: impala-server.num-fragments-in-flight from
impala-ec2-centos74-m5-4xlarge-ondemand-0458.vpc.cloudera.com:25000
MainThread: Debug webpage not yet available.
MainThread: Debug webpage not yet available.
MainThread: Debug webpage not yet available.
MainThread: Debug webpage not yet available.
MainThread: Debug webpage not yet available.
MainThread: Debug webpage not yet available.
MainThread: Debug webpage not yet available.
MainThread: Debug webpage not yet available.
MainThread: Debug webpage not yet available.
MainThread: Debug webpage not yet available.
MainThread: Debug webpage did not become available in expected time.
MainThread: Waiting for metric value
[jira] [Commented] (IMPALA-7379) test_random_rpc_timeout failed on exhaustive build: Debug webpage did not become available in expected time
[
https://issues.apache.org/jira/browse/IMPALA-7379?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16564486#comment-16564486
]
David Knupp commented on IMPALA-7379:
-
Assigning to [~kwho] because it's an RPC timeout issue. Please reassign if this
is not correct.
Thanks.
> test_random_rpc_timeout failed on exhaustive build: Debug webpage did not
> become available in expected time
> ---
>
> Key: IMPALA-7379
> URL: https://issues.apache.org/jira/browse/IMPALA-7379
> Project: IMPALA
> Issue Type: Bug
> Components: Backend
>Affects Versions: Impala 3.1.0
>Reporter: David Knupp
>Assignee: Michael Ho
>Priority: Major
>
> *Stacktrace*
> {noformat}
> custom_cluster/test_rpc_timeout.py:131: in test_random_rpc_timeout
> self.execute_query_verify_metrics(self.TEST_QUERY, 10)
> custom_cluster/test_rpc_timeout.py:51: in execute_query_verify_metrics
> v.wait_for_metric("impala-server.num-fragments-in-flight", 0)
> verifiers/metric_verifier.py:62: in wait_for_metric
> self.impalad_service.wait_for_metric_value(metric_name, expected_value,
> timeout)
> common/impala_service.py:132: in wait_for_metric_value
> json.dumps(self.read_debug_webpage('memz?json')),
> common/impala_service.py:63: in read_debug_webpage
> return self.open_debug_webpage(page_name, timeout=timeout,
> interval=interval).read()
> common/impala_service.py:60: in open_debug_webpage
> assert 0, 'Debug webpage did not become available in expected time.'
> E AssertionError: Debug webpage did not become available in expected time.
> {noformat}
> *Standard Error*
> {noformat}
> 10:27:58 MainThread: Starting State Store logging to
> /data/jenkins/workspace/impala-asf-master-exhaustive/repos/Impala/logs/custom_cluster_tests/statestored.INFO
> 10:27:58 MainThread: Starting Catalog Service logging to
> /data/jenkins/workspace/impala-asf-master-exhaustive/repos/Impala/logs/custom_cluster_tests/catalogd.INFO
> 10:27:59 MainThread: Starting Impala Daemon logging to
> /data/jenkins/workspace/impala-asf-master-exhaustive/repos/Impala/logs/custom_cluster_tests/impalad.INFO
> 10:28:00 MainThread: Starting Impala Daemon logging to
> /data/jenkins/workspace/impala-asf-master-exhaustive/repos/Impala/logs/custom_cluster_tests/impalad_node1.INFO
> 10:28:01 MainThread: Starting Impala Daemon logging to
> /data/jenkins/workspace/impala-asf-master-exhaustive/repos/Impala/logs/custom_cluster_tests/impalad_node2.INFO
> 10:28:04 MainThread: Found 3 impalad/1 statestored/1 catalogd process(es)
> 10:28:04 MainThread: Getting num_known_live_backends from
> impala-ec2-centos74-m5-4xlarge-ondemand-0458.vpc.cloudera.com:25000
> 10:28:04 MainThread: Waiting for num_known_live_backends=3. Current value: 1
> 10:28:05 MainThread: Getting num_known_live_backends from
> impala-ec2-centos74-m5-4xlarge-ondemand-0458.vpc.cloudera.com:25000
> 10:28:05 MainThread: Waiting for num_known_live_backends=3. Current value: 2
> 10:28:06 MainThread: Getting num_known_live_backends from
> impala-ec2-centos74-m5-4xlarge-ondemand-0458.vpc.cloudera.com:25000
> 10:28:06 MainThread: num_known_live_backends has reached value: 3
> 10:28:06 MainThread: Getting num_known_live_backends from
> impala-ec2-centos74-m5-4xlarge-ondemand-0458.vpc.cloudera.com:25001
> 10:28:06 MainThread: num_known_live_backends has reached value: 3
> 10:28:06 MainThread: Getting num_known_live_backends from
> impala-ec2-centos74-m5-4xlarge-ondemand-0458.vpc.cloudera.com:25002
> 10:28:06 MainThread: num_known_live_backends has reached value: 3
> 10:28:06 MainThread: Impala Cluster Running with 3 nodes (3 coordinators, 3
> executors).
> MainThread: Found 3 impalad/1 statestored/1 catalogd process(es)
> MainThread: Getting metric: statestore.live-backends from
> impala-ec2-centos74-m5-4xlarge-ondemand-0458.vpc.cloudera.com:25010
> MainThread: Metric 'statestore.live-backends' has reached desired value: 4
> MainThread: Getting num_known_live_backends from
> impala-ec2-centos74-m5-4xlarge-ondemand-0458.vpc.cloudera.com:25000
> MainThread: num_known_live_backends has reached value: 3
> MainThread: Getting num_known_live_backends from
> impala-ec2-centos74-m5-4xlarge-ondemand-0458.vpc.cloudera.com:25001
> MainThread: num_known_live_backends has reached value: 3
> MainThread: Getting num_known_live_backends from
> impala-ec2-centos74-m5-4xlarge-ondemand-0458.vpc.cloudera.com:25002
> MainThread: num_known_live_backends has reached value: 3
> -- connecting to: localhost:21000
> -- executing against localhost:21000
> select count(c2.string_col) from functional.alltypestiny join
> functional.alltypessmall c2;
> -- executing against localhost:21000
> select count(c2.string_col) from functional.alltypestiny join
>
[jira] [Created] (IMPALA-7379) test_random_rpc_timeout failed on exhaustive build: Debug webpage did not become available in expected time
David Knupp created IMPALA-7379:
---
Summary: test_random_rpc_timeout failed on exhaustive build: Debug
webpage did not become available in expected time
Key: IMPALA-7379
URL: https://issues.apache.org/jira/browse/IMPALA-7379
Project: IMPALA
Issue Type: Bug
Components: Backend
Affects Versions: Impala 3.1.0
Reporter: David Knupp
Assignee: Michael Ho
*Stacktrace*
{noformat}
custom_cluster/test_rpc_timeout.py:131: in test_random_rpc_timeout
self.execute_query_verify_metrics(self.TEST_QUERY, 10)
custom_cluster/test_rpc_timeout.py:51: in execute_query_verify_metrics
v.wait_for_metric("impala-server.num-fragments-in-flight", 0)
verifiers/metric_verifier.py:62: in wait_for_metric
self.impalad_service.wait_for_metric_value(metric_name, expected_value,
timeout)
common/impala_service.py:132: in wait_for_metric_value
json.dumps(self.read_debug_webpage('memz?json')),
common/impala_service.py:63: in read_debug_webpage
return self.open_debug_webpage(page_name, timeout=timeout,
interval=interval).read()
common/impala_service.py:60: in open_debug_webpage
assert 0, 'Debug webpage did not become available in expected time.'
E AssertionError: Debug webpage did not become available in expected time.
{noformat}
*Standard Error*
{noformat}
10:27:58 MainThread: Starting State Store logging to
/data/jenkins/workspace/impala-asf-master-exhaustive/repos/Impala/logs/custom_cluster_tests/statestored.INFO
10:27:58 MainThread: Starting Catalog Service logging to
/data/jenkins/workspace/impala-asf-master-exhaustive/repos/Impala/logs/custom_cluster_tests/catalogd.INFO
10:27:59 MainThread: Starting Impala Daemon logging to
/data/jenkins/workspace/impala-asf-master-exhaustive/repos/Impala/logs/custom_cluster_tests/impalad.INFO
10:28:00 MainThread: Starting Impala Daemon logging to
/data/jenkins/workspace/impala-asf-master-exhaustive/repos/Impala/logs/custom_cluster_tests/impalad_node1.INFO
10:28:01 MainThread: Starting Impala Daemon logging to
/data/jenkins/workspace/impala-asf-master-exhaustive/repos/Impala/logs/custom_cluster_tests/impalad_node2.INFO
10:28:04 MainThread: Found 3 impalad/1 statestored/1 catalogd process(es)
10:28:04 MainThread: Getting num_known_live_backends from
impala-ec2-centos74-m5-4xlarge-ondemand-0458.vpc.cloudera.com:25000
10:28:04 MainThread: Waiting for num_known_live_backends=3. Current value: 1
10:28:05 MainThread: Getting num_known_live_backends from
impala-ec2-centos74-m5-4xlarge-ondemand-0458.vpc.cloudera.com:25000
10:28:05 MainThread: Waiting for num_known_live_backends=3. Current value: 2
10:28:06 MainThread: Getting num_known_live_backends from
impala-ec2-centos74-m5-4xlarge-ondemand-0458.vpc.cloudera.com:25000
10:28:06 MainThread: num_known_live_backends has reached value: 3
10:28:06 MainThread: Getting num_known_live_backends from
impala-ec2-centos74-m5-4xlarge-ondemand-0458.vpc.cloudera.com:25001
10:28:06 MainThread: num_known_live_backends has reached value: 3
10:28:06 MainThread: Getting num_known_live_backends from
impala-ec2-centos74-m5-4xlarge-ondemand-0458.vpc.cloudera.com:25002
10:28:06 MainThread: num_known_live_backends has reached value: 3
10:28:06 MainThread: Impala Cluster Running with 3 nodes (3 coordinators, 3
executors).
MainThread: Found 3 impalad/1 statestored/1 catalogd process(es)
MainThread: Getting metric: statestore.live-backends from
impala-ec2-centos74-m5-4xlarge-ondemand-0458.vpc.cloudera.com:25010
MainThread: Metric 'statestore.live-backends' has reached desired value: 4
MainThread: Getting num_known_live_backends from
impala-ec2-centos74-m5-4xlarge-ondemand-0458.vpc.cloudera.com:25000
MainThread: num_known_live_backends has reached value: 3
MainThread: Getting num_known_live_backends from
impala-ec2-centos74-m5-4xlarge-ondemand-0458.vpc.cloudera.com:25001
MainThread: num_known_live_backends has reached value: 3
MainThread: Getting num_known_live_backends from
impala-ec2-centos74-m5-4xlarge-ondemand-0458.vpc.cloudera.com:25002
MainThread: num_known_live_backends has reached value: 3
-- connecting to: localhost:21000
-- executing against localhost:21000
select count(c2.string_col) from functional.alltypestiny join
functional.alltypessmall c2;
-- executing against localhost:21000
select count(c2.string_col) from functional.alltypestiny join
functional.alltypessmall c2;
-- executing against localhost:21000
select count(c2.string_col) from functional.alltypestiny join
functional.alltypessmall c2;
-- executing against localhost:21000
select count(c2.string_col) from functional.alltypestiny join
functional.alltypessmall c2;
-- executing against localhost:21000
select count(c2.string_col) from functional.alltypestiny join
functional.alltypessmall c2;
-- executing against localhost:21000
select count(c2.string_col) from functional.alltypestiny join
[jira] [Commented] (IMPALA-7075) Add support for object ownership for Impala
[ https://issues.apache.org/jira/browse/IMPALA-7075?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16564470#comment-16564470 ] Jim Apple commented on IMPALA-7075: --- [~fredyw], it would be helpful if you would add a description to tickets like this one and IMPALA-7377 so that a person could understand what you meant even if you were not available to explain it to them. > Add support for object ownership for Impala > --- > > Key: IMPALA-7075 > URL: https://issues.apache.org/jira/browse/IMPALA-7075 > Project: IMPALA > Issue Type: Epic > Components: Frontend >Reporter: Fredy Wijaya >Assignee: Adam Holley >Priority: Major > Labels: security > -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Commented] (IMPALA-7335) Assertion Failure - test_corrupt_files
[
https://issues.apache.org/jira/browse/IMPALA-7335?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16564464#comment-16564464
]
Philip Zeyliger commented on IMPALA-7335:
-
https://jenkins.impala.io/job/ubuntu-16.04-from-scratch/2822/testReport/junit/query_test.test_scanners/TestParquet/test_parquet_exec_optionbatch_size___0___num_nodes___0___disable_codegen_rows_threshold___0___disable_codegen___False___abort_on_error___1___debug_action___None___exec_single_node_rows_threshold___0table_format__parquet_none_/
seems similar too. We expect an error that doesn't show up.
> Assertion Failure - test_corrupt_files
> --
>
> Key: IMPALA-7335
> URL: https://issues.apache.org/jira/browse/IMPALA-7335
> Project: IMPALA
> Issue Type: Bug
>Affects Versions: Impala 3.1.0
>Reporter: nithya
>Assignee: Pooja Nilangekar
>Priority: Critical
> Labels: broken-build
>
> test_corrupt_files fails
>
> query_test.test_scanners.TestParquet.test_corrupt_files[exec_option:
> \\{'batch_size': 0, 'num_nodes': 0, 'disable_codegen_rows_threshold': 0,
> 'disable_codegen': False, 'abort_on_error': 1, 'debug_action': None,
> 'exec_single_node_rows_threshold': 0} | table_format: parquet/none] (from
> pytest)
>
> {code:java}
> Error Message
> query_test/test_scanners.py:300: in test_corrupt_files
> self.run_test_case('QueryTest/parquet-abort-on-error', vector)
> common/impala_test_suite.py:420: in run_test_case assert False, "Expected
> exception: %s" % expected_str E AssertionError: Expected exception: Column
> metadata states there are 11 values, but read 10 values from column id.
> STACKTRACE
> query_test/test_scanners.py:300: in test_corrupt_files
> self.run_test_case('QueryTest/parquet-abort-on-error', vector)
> common/impala_test_suite.py:420: in run_test_case
> assert False, "Expected exception: %s" % expected_str
> E AssertionError: Expected exception: Column metadata states there are 11
> values, but read 10 values from column id.
> Standard Error
> -- executing against localhost:21000
> use functional_parquet;
> SET batch_size=0;
> SET num_nodes=0;
> SET disable_codegen_rows_threshold=0;
> SET disable_codegen=False;
> SET abort_on_error=0;
> SET exec_single_node_rows_threshold=0;
> -- executing against localhost:21000
> set num_nodes=1;
> -- executing against localhost:21000
> set num_scanner_threads=1;
> -- executing against localhost:21000
> select id, cnt from bad_column_metadata t, (select count(*) cnt from
> t.int_array) v;
> -- executing against localhost:21000
> SET NUM_NODES="0";
> -- executing against localhost:21000
> SET NUM_SCANNER_THREADS="0";
> -- executing against localhost:21000
> set num_nodes=1;
> -- executing against localhost:21000
> set num_scanner_threads=1;
> -- executing against localhost:21000
> select id from bad_column_metadata;
> -- executing against localhost:21000
> SET NUM_NODES="0";
> -- executing against localhost:21000
> SET NUM_SCANNER_THREADS="0";
> -- executing against localhost:21000
> SELECT * from bad_parquet_strings_negative_len;
> -- executing against localhost:21000
> SELECT * from bad_parquet_strings_out_of_bounds;
> -- executing against localhost:21000
> use functional_parquet;
> SET batch_size=0;
> SET num_nodes=0;
> SET disable_codegen_rows_threshold=0;
> SET disable_codegen=False;
> SET abort_on_error=1;
> SET exec_single_node_rows_threshold=0;
> -- executing against localhost:21000
> set num_nodes=1;
> -- executing against localhost:21000
> set num_scanner_threads=1;
> -- executing against localhost:21000
> select id, cnt from bad_column_metadata t, (select count(*) cnt from
> t.int_array) v;
> -- executing against localhost:21000
> SET NUM_NODES="0";
> -- executing against localhost:21000
> SET NUM_SCANNER_THREADS="0";
> -- executing against localhost:21000
> set num_nodes=1;
> -- executing against localhost:21000
> set num_scanner_threads=1;
> -- executing against localhost:21000
> select id from bad_column_metadata;
> -- executing against localhost:21000
> SET NUM_NODES="0";
> -- executing against localhost:21000
> SET NUM_SCANNER_THREADS="0";
> {code}
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Commented] (IMPALA-7378) test_strict_mode failed on an ASAN build: expected "Error converting column: 5 to DOUBLE"
[
https://issues.apache.org/jira/browse/IMPALA-7378?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16564459#comment-16564459
]
Pooja Nilangekar commented on IMPALA-7378:
--
Yes, it looks like the failures in the fragments are not reported to the
coordinator. Let me take a look at the logs to confirm it.
> test_strict_mode failed on an ASAN build: expected "Error converting column:
> 5 to DOUBLE"
> -
>
> Key: IMPALA-7378
> URL: https://issues.apache.org/jira/browse/IMPALA-7378
> Project: IMPALA
> Issue Type: Bug
> Components: Backend
>Affects Versions: Impala 3.1.0
>Reporter: David Knupp
>Assignee: Tim Armstrong
>Priority: Critical
> Labels: broken-build
>
> *Stacktrace*
> {noformat}
> query_test/test_queries.py:159: in test_strict_mode
> self.run_test_case('QueryTest/strict-mode-abort', vector)
> common/impala_test_suite.py:420: in run_test_case
> assert False, "Expected exception: %s" % expected_str
> E AssertionError: Expected exception: Error converting column: 5 to DOUBLE
> {noformat}
> *Standard Error*
> {noformat}
> -- executing against localhost:21000
> use functional;
> SET strict_mode=1;
> SET batch_size=0;
> SET num_nodes=0;
> SET disable_codegen_rows_threshold=0;
> SET disable_codegen=False;
> SET abort_on_error=0;
> SET exec_single_node_rows_threshold=0;
> -- executing against localhost:21000
> select * from overflow;
> -- executing against localhost:21000
> use functional;
> SET strict_mode=1;
> SET batch_size=0;
> SET num_nodes=0;
> SET disable_codegen_rows_threshold=0;
> SET disable_codegen=False;
> SET abort_on_error=1;
> SET exec_single_node_rows_threshold=0;
> -- executing against localhost:21000
> select tinyint_col from overflow;
> -- executing against localhost:21000
> select smallint_col from overflow;
> -- executing against localhost:21000
> select int_col from overflow;
> -- executing against localhost:21000
> select bigint_col from overflow;
> -- executing against localhost:21000
> select float_col from overflow;
> -- executing against localhost:21000
> select double_col from overflow;
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Commented] (IMPALA-7378) test_strict_mode failed on an ASAN build: expected "Error converting column: 5 to DOUBLE"
[
https://issues.apache.org/jira/browse/IMPALA-7378?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16564441#comment-16564441
]
Tim Armstrong commented on IMPALA-7378:
---
[~poojanilangekar] looks a bit like IMPALA-7335 except in a different test. Do
you agree?
> test_strict_mode failed on an ASAN build: expected "Error converting column:
> 5 to DOUBLE"
> -
>
> Key: IMPALA-7378
> URL: https://issues.apache.org/jira/browse/IMPALA-7378
> Project: IMPALA
> Issue Type: Bug
> Components: Backend
>Affects Versions: Impala 3.1.0
>Reporter: David Knupp
>Assignee: Tim Armstrong
>Priority: Critical
> Labels: broken-build
>
> *Stacktrace*
> {noformat}
> query_test/test_queries.py:159: in test_strict_mode
> self.run_test_case('QueryTest/strict-mode-abort', vector)
> common/impala_test_suite.py:420: in run_test_case
> assert False, "Expected exception: %s" % expected_str
> E AssertionError: Expected exception: Error converting column: 5 to DOUBLE
> {noformat}
> *Standard Error*
> {noformat}
> -- executing against localhost:21000
> use functional;
> SET strict_mode=1;
> SET batch_size=0;
> SET num_nodes=0;
> SET disable_codegen_rows_threshold=0;
> SET disable_codegen=False;
> SET abort_on_error=0;
> SET exec_single_node_rows_threshold=0;
> -- executing against localhost:21000
> select * from overflow;
> -- executing against localhost:21000
> use functional;
> SET strict_mode=1;
> SET batch_size=0;
> SET num_nodes=0;
> SET disable_codegen_rows_threshold=0;
> SET disable_codegen=False;
> SET abort_on_error=1;
> SET exec_single_node_rows_threshold=0;
> -- executing against localhost:21000
> select tinyint_col from overflow;
> -- executing against localhost:21000
> select smallint_col from overflow;
> -- executing against localhost:21000
> select int_col from overflow;
> -- executing against localhost:21000
> select bigint_col from overflow;
> -- executing against localhost:21000
> select float_col from overflow;
> -- executing against localhost:21000
> select double_col from overflow;
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Updated] (IMPALA-7378) test_strict_mode failed on an ASAN build: expected "Error converting column: 5 to DOUBLE"
[
https://issues.apache.org/jira/browse/IMPALA-7378?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Tim Armstrong updated IMPALA-7378:
--
Labels: broken-build (was: )
> test_strict_mode failed on an ASAN build: expected "Error converting column:
> 5 to DOUBLE"
> -
>
> Key: IMPALA-7378
> URL: https://issues.apache.org/jira/browse/IMPALA-7378
> Project: IMPALA
> Issue Type: Bug
> Components: Backend
>Affects Versions: Impala 3.1.0
>Reporter: David Knupp
>Assignee: Tim Armstrong
>Priority: Critical
> Labels: broken-build
>
> *Stacktrace*
> {noformat}
> query_test/test_queries.py:159: in test_strict_mode
> self.run_test_case('QueryTest/strict-mode-abort', vector)
> common/impala_test_suite.py:420: in run_test_case
> assert False, "Expected exception: %s" % expected_str
> E AssertionError: Expected exception: Error converting column: 5 to DOUBLE
> {noformat}
> *Standard Error*
> {noformat}
> -- executing against localhost:21000
> use functional;
> SET strict_mode=1;
> SET batch_size=0;
> SET num_nodes=0;
> SET disable_codegen_rows_threshold=0;
> SET disable_codegen=False;
> SET abort_on_error=0;
> SET exec_single_node_rows_threshold=0;
> -- executing against localhost:21000
> select * from overflow;
> -- executing against localhost:21000
> use functional;
> SET strict_mode=1;
> SET batch_size=0;
> SET num_nodes=0;
> SET disable_codegen_rows_threshold=0;
> SET disable_codegen=False;
> SET abort_on_error=1;
> SET exec_single_node_rows_threshold=0;
> -- executing against localhost:21000
> select tinyint_col from overflow;
> -- executing against localhost:21000
> select smallint_col from overflow;
> -- executing against localhost:21000
> select int_col from overflow;
> -- executing against localhost:21000
> select bigint_col from overflow;
> -- executing against localhost:21000
> select float_col from overflow;
> -- executing against localhost:21000
> select double_col from overflow;
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Updated] (IMPALA-7378) test_strict_mode failed on an ASAN build: expected "Error converting column: 5 to DOUBLE"
[
https://issues.apache.org/jira/browse/IMPALA-7378?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Tim Armstrong updated IMPALA-7378:
--
Target Version: Impala 3.1.0
> test_strict_mode failed on an ASAN build: expected "Error converting column:
> 5 to DOUBLE"
> -
>
> Key: IMPALA-7378
> URL: https://issues.apache.org/jira/browse/IMPALA-7378
> Project: IMPALA
> Issue Type: Bug
> Components: Backend
>Affects Versions: Impala 3.1.0
>Reporter: David Knupp
>Assignee: Tim Armstrong
>Priority: Critical
>
> *Stacktrace*
> {noformat}
> query_test/test_queries.py:159: in test_strict_mode
> self.run_test_case('QueryTest/strict-mode-abort', vector)
> common/impala_test_suite.py:420: in run_test_case
> assert False, "Expected exception: %s" % expected_str
> E AssertionError: Expected exception: Error converting column: 5 to DOUBLE
> {noformat}
> *Standard Error*
> {noformat}
> -- executing against localhost:21000
> use functional;
> SET strict_mode=1;
> SET batch_size=0;
> SET num_nodes=0;
> SET disable_codegen_rows_threshold=0;
> SET disable_codegen=False;
> SET abort_on_error=0;
> SET exec_single_node_rows_threshold=0;
> -- executing against localhost:21000
> select * from overflow;
> -- executing against localhost:21000
> use functional;
> SET strict_mode=1;
> SET batch_size=0;
> SET num_nodes=0;
> SET disable_codegen_rows_threshold=0;
> SET disable_codegen=False;
> SET abort_on_error=1;
> SET exec_single_node_rows_threshold=0;
> -- executing against localhost:21000
> select tinyint_col from overflow;
> -- executing against localhost:21000
> select smallint_col from overflow;
> -- executing against localhost:21000
> select int_col from overflow;
> -- executing against localhost:21000
> select bigint_col from overflow;
> -- executing against localhost:21000
> select float_col from overflow;
> -- executing against localhost:21000
> select double_col from overflow;
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Updated] (IMPALA-7378) test_strict_mode failed on an ASAN build: expected "Error converting column: 5 to DOUBLE"
[
https://issues.apache.org/jira/browse/IMPALA-7378?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Tim Armstrong updated IMPALA-7378:
--
Priority: Critical (was: Major)
> test_strict_mode failed on an ASAN build: expected "Error converting column:
> 5 to DOUBLE"
> -
>
> Key: IMPALA-7378
> URL: https://issues.apache.org/jira/browse/IMPALA-7378
> Project: IMPALA
> Issue Type: Bug
> Components: Backend
>Affects Versions: Impala 3.1.0
>Reporter: David Knupp
>Assignee: Tim Armstrong
>Priority: Critical
>
> *Stacktrace*
> {noformat}
> query_test/test_queries.py:159: in test_strict_mode
> self.run_test_case('QueryTest/strict-mode-abort', vector)
> common/impala_test_suite.py:420: in run_test_case
> assert False, "Expected exception: %s" % expected_str
> E AssertionError: Expected exception: Error converting column: 5 to DOUBLE
> {noformat}
> *Standard Error*
> {noformat}
> -- executing against localhost:21000
> use functional;
> SET strict_mode=1;
> SET batch_size=0;
> SET num_nodes=0;
> SET disable_codegen_rows_threshold=0;
> SET disable_codegen=False;
> SET abort_on_error=0;
> SET exec_single_node_rows_threshold=0;
> -- executing against localhost:21000
> select * from overflow;
> -- executing against localhost:21000
> use functional;
> SET strict_mode=1;
> SET batch_size=0;
> SET num_nodes=0;
> SET disable_codegen_rows_threshold=0;
> SET disable_codegen=False;
> SET abort_on_error=1;
> SET exec_single_node_rows_threshold=0;
> -- executing against localhost:21000
> select tinyint_col from overflow;
> -- executing against localhost:21000
> select smallint_col from overflow;
> -- executing against localhost:21000
> select int_col from overflow;
> -- executing against localhost:21000
> select bigint_col from overflow;
> -- executing against localhost:21000
> select float_col from overflow;
> -- executing against localhost:21000
> select double_col from overflow;
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Commented] (IMPALA-7378) test_strict_mode failed on an ASAN build: expected "Error converting column: 5 to DOUBLE"
[
https://issues.apache.org/jira/browse/IMPALA-7378?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16564440#comment-16564440
]
David Knupp commented on IMPALA-7378:
-
Assigning to @tim who had context from when this test was first added. Please
reassign if you're swamped.
> test_strict_mode failed on an ASAN build: expected "Error converting column:
> 5 to DOUBLE"
> -
>
> Key: IMPALA-7378
> URL: https://issues.apache.org/jira/browse/IMPALA-7378
> Project: IMPALA
> Issue Type: Bug
> Components: Backend
>Affects Versions: Impala 3.1.0
>Reporter: David Knupp
>Assignee: Tim Armstrong
>Priority: Major
>
> *Stacktrace*
> {noformat}
> query_test/test_queries.py:159: in test_strict_mode
> self.run_test_case('QueryTest/strict-mode-abort', vector)
> common/impala_test_suite.py:420: in run_test_case
> assert False, "Expected exception: %s" % expected_str
> E AssertionError: Expected exception: Error converting column: 5 to DOUBLE
> {noformat}
> *Standard Error*
> {noformat}
> -- executing against localhost:21000
> use functional;
> SET strict_mode=1;
> SET batch_size=0;
> SET num_nodes=0;
> SET disable_codegen_rows_threshold=0;
> SET disable_codegen=False;
> SET abort_on_error=0;
> SET exec_single_node_rows_threshold=0;
> -- executing against localhost:21000
> select * from overflow;
> -- executing against localhost:21000
> use functional;
> SET strict_mode=1;
> SET batch_size=0;
> SET num_nodes=0;
> SET disable_codegen_rows_threshold=0;
> SET disable_codegen=False;
> SET abort_on_error=1;
> SET exec_single_node_rows_threshold=0;
> -- executing against localhost:21000
> select tinyint_col from overflow;
> -- executing against localhost:21000
> select smallint_col from overflow;
> -- executing against localhost:21000
> select int_col from overflow;
> -- executing against localhost:21000
> select bigint_col from overflow;
> -- executing against localhost:21000
> select float_col from overflow;
> -- executing against localhost:21000
> select double_col from overflow;
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Comment Edited] (IMPALA-7378) test_strict_mode failed on an ASAN build: expected "Error converting column: 5 to DOUBLE"
[
https://issues.apache.org/jira/browse/IMPALA-7378?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16564440#comment-16564440
]
David Knupp edited comment on IMPALA-7378 at 7/31/18 10:41 PM:
---
Assigning to [~tarmstrong] who had context from when this test was first added.
Please reassign if you're swamped.
was (Author: dknupp):
Assigning to @tim who had context from when this test was first added. Please
reassign if you're swamped.
> test_strict_mode failed on an ASAN build: expected "Error converting column:
> 5 to DOUBLE"
> -
>
> Key: IMPALA-7378
> URL: https://issues.apache.org/jira/browse/IMPALA-7378
> Project: IMPALA
> Issue Type: Bug
> Components: Backend
>Affects Versions: Impala 3.1.0
>Reporter: David Knupp
>Assignee: Tim Armstrong
>Priority: Major
>
> *Stacktrace*
> {noformat}
> query_test/test_queries.py:159: in test_strict_mode
> self.run_test_case('QueryTest/strict-mode-abort', vector)
> common/impala_test_suite.py:420: in run_test_case
> assert False, "Expected exception: %s" % expected_str
> E AssertionError: Expected exception: Error converting column: 5 to DOUBLE
> {noformat}
> *Standard Error*
> {noformat}
> -- executing against localhost:21000
> use functional;
> SET strict_mode=1;
> SET batch_size=0;
> SET num_nodes=0;
> SET disable_codegen_rows_threshold=0;
> SET disable_codegen=False;
> SET abort_on_error=0;
> SET exec_single_node_rows_threshold=0;
> -- executing against localhost:21000
> select * from overflow;
> -- executing against localhost:21000
> use functional;
> SET strict_mode=1;
> SET batch_size=0;
> SET num_nodes=0;
> SET disable_codegen_rows_threshold=0;
> SET disable_codegen=False;
> SET abort_on_error=1;
> SET exec_single_node_rows_threshold=0;
> -- executing against localhost:21000
> select tinyint_col from overflow;
> -- executing against localhost:21000
> select smallint_col from overflow;
> -- executing against localhost:21000
> select int_col from overflow;
> -- executing against localhost:21000
> select bigint_col from overflow;
> -- executing against localhost:21000
> select float_col from overflow;
> -- executing against localhost:21000
> select double_col from overflow;
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Created] (IMPALA-7378) test_strict_mode failed on an ASAN build: expected "Error converting column: 5 to DOUBLE"
David Knupp created IMPALA-7378:
---
Summary: test_strict_mode failed on an ASAN build: expected "Error
converting column: 5 to DOUBLE"
Key: IMPALA-7378
URL: https://issues.apache.org/jira/browse/IMPALA-7378
Project: IMPALA
Issue Type: Bug
Components: Backend
Affects Versions: Impala 3.1.0
Reporter: David Knupp
Assignee: Tim Armstrong
*Stacktrace*
{noformat}
query_test/test_queries.py:159: in test_strict_mode
self.run_test_case('QueryTest/strict-mode-abort', vector)
common/impala_test_suite.py:420: in run_test_case
assert False, "Expected exception: %s" % expected_str
E AssertionError: Expected exception: Error converting column: 5 to DOUBLE
{noformat}
*Standard Error*
{noformat}
-- executing against localhost:21000
use functional;
SET strict_mode=1;
SET batch_size=0;
SET num_nodes=0;
SET disable_codegen_rows_threshold=0;
SET disable_codegen=False;
SET abort_on_error=0;
SET exec_single_node_rows_threshold=0;
-- executing against localhost:21000
select * from overflow;
-- executing against localhost:21000
use functional;
SET strict_mode=1;
SET batch_size=0;
SET num_nodes=0;
SET disable_codegen_rows_threshold=0;
SET disable_codegen=False;
SET abort_on_error=1;
SET exec_single_node_rows_threshold=0;
-- executing against localhost:21000
select tinyint_col from overflow;
-- executing against localhost:21000
select smallint_col from overflow;
-- executing against localhost:21000
select int_col from overflow;
-- executing against localhost:21000
select bigint_col from overflow;
-- executing against localhost:21000
select float_col from overflow;
-- executing against localhost:21000
select double_col from overflow;
{noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Updated] (IMPALA-7361) test_heterogeneous_proc_mem_limit: Query aborted: Not enough memory available on host (s3)
[
https://issues.apache.org/jira/browse/IMPALA-7361?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
David Knupp updated IMPALA-7361:
Affects Version/s: Impala 3.1.0
> test_heterogeneous_proc_mem_limit: Query aborted: Not enough memory
> available on host (s3)
> ---
>
> Key: IMPALA-7361
> URL: https://issues.apache.org/jira/browse/IMPALA-7361
> Project: IMPALA
> Issue Type: Bug
> Components: Backend
>Affects Versions: Impala 3.1.0
>Reporter: nithya
>Assignee: Bikramjeet Vig
>Priority: Major
> Labels: broken-build
>
> texttest_heterogeneous_proc_mem_limit custom cluster test fails with the
> following assertion error
> Stacktrace
> {noformat}
> custom_cluster/test_admission_controller.py:514: in
> test_heterogeneous_proc_mem_limit
> assert re.search("Queued reason: Not enough memory available on host
> \S+.Needed "
> E AssertionError: ImpalaBeeswaxException:
> E Query aborted:Admission for query exceeded timeout 200ms in pool
> default-pool. Queued reason: Not enough memory available on host
> impala-ec2-centos74-m5-4xlarge-ondemand-08d6.vpc.cloudera.com:22001.Needed
> 2.00 GB but only 1.00 GB out of 3.00 GB was available.
> E
> E
> E assert None
> E+ where None = ('Queued reason: Not
> enough memory available on host \\S+.Needed 2.00 GB but only 1.00 GB out of
> 2.00 GB was available.', 'ImpalaBeeswaxException:\n Query aborted:Admission
> for query exceeded timeout 200ms in pool default-pool. Queued
> reaso...os74-m5-4xlarge-ondemand-08d6.vpc.cloudera.com:22001.Needed 2.00 GB
> but only 1.00 GB out of 3.00 GB was available.\n\n')
> E+where = re.search
> E+and 'ImpalaBeeswaxException:\n Query aborted:Admission for query
> exceeded timeout 200ms in pool default-pool. Queued
> reaso...os74-m5-4xlarge-ondemand-08d6.vpc.cloudera.com:22001.Needed 2.00 GB
> but only 1.00 GB out of 3.00 GB was available.\n\n' =
> str(ImpalaBeeswaxException())
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Updated] (IMPALA-7361) test_heterogeneous_proc_mem_limit: Query aborted: Not enough memory available on host (s3)
[
https://issues.apache.org/jira/browse/IMPALA-7361?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
David Knupp updated IMPALA-7361:
Summary: test_heterogeneous_proc_mem_limit: Query aborted: Not enough
memory available on host (s3) (was: test_heterogeneous_proc_mem_limit: Query
aborted: Not enough memory available on host)
> test_heterogeneous_proc_mem_limit: Query aborted: Not enough memory
> available on host (s3)
> ---
>
> Key: IMPALA-7361
> URL: https://issues.apache.org/jira/browse/IMPALA-7361
> Project: IMPALA
> Issue Type: Bug
> Components: Backend
>Reporter: nithya
>Assignee: Bikramjeet Vig
>Priority: Major
> Labels: broken-build
>
> texttest_heterogeneous_proc_mem_limit custom cluster test fails with the
> following assertion error
> Stacktrace
> {noformat}
> custom_cluster/test_admission_controller.py:514: in
> test_heterogeneous_proc_mem_limit
> assert re.search("Queued reason: Not enough memory available on host
> \S+.Needed "
> E AssertionError: ImpalaBeeswaxException:
> E Query aborted:Admission for query exceeded timeout 200ms in pool
> default-pool. Queued reason: Not enough memory available on host
> impala-ec2-centos74-m5-4xlarge-ondemand-08d6.vpc.cloudera.com:22001.Needed
> 2.00 GB but only 1.00 GB out of 3.00 GB was available.
> E
> E
> E assert None
> E+ where None = ('Queued reason: Not
> enough memory available on host \\S+.Needed 2.00 GB but only 1.00 GB out of
> 2.00 GB was available.', 'ImpalaBeeswaxException:\n Query aborted:Admission
> for query exceeded timeout 200ms in pool default-pool. Queued
> reaso...os74-m5-4xlarge-ondemand-08d6.vpc.cloudera.com:22001.Needed 2.00 GB
> but only 1.00 GB out of 3.00 GB was available.\n\n')
> E+where = re.search
> E+and 'ImpalaBeeswaxException:\n Query aborted:Admission for query
> exceeded timeout 200ms in pool default-pool. Queued
> reaso...os74-m5-4xlarge-ondemand-08d6.vpc.cloudera.com:22001.Needed 2.00 GB
> but only 1.00 GB out of 3.00 GB was available.\n\n' =
> str(ImpalaBeeswaxException())
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Work started] (IMPALA-7377) Update Sentry for object ownership feature
[ https://issues.apache.org/jira/browse/IMPALA-7377?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on IMPALA-7377 started by Fredy Wijaya. > Update Sentry for object ownership feature > -- > > Key: IMPALA-7377 > URL: https://issues.apache.org/jira/browse/IMPALA-7377 > Project: IMPALA > Issue Type: Sub-task > Components: Infrastructure >Reporter: Fredy Wijaya >Assignee: Fredy Wijaya >Priority: Major > Labels: security > Fix For: Impala 3.0 > > -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Created] (IMPALA-7377) Update Sentry for object ownership feature
Fredy Wijaya created IMPALA-7377: Summary: Update Sentry for object ownership feature Key: IMPALA-7377 URL: https://issues.apache.org/jira/browse/IMPALA-7377 Project: IMPALA Issue Type: Sub-task Components: Infrastructure Reporter: Fredy Wijaya Assignee: Fredy Wijaya Fix For: Impala 3.0 -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Updated] (IMPALA-7361) test_heterogeneous_proc_mem_limit: Query aborted: Not enough memory available on host
[
https://issues.apache.org/jira/browse/IMPALA-7361?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
David Knupp updated IMPALA-7361:
Description:
texttest_heterogeneous_proc_mem_limit custom cluster test fails with the
following assertion error
Stacktrace
{noformat}
custom_cluster/test_admission_controller.py:514: in
test_heterogeneous_proc_mem_limit
assert re.search("Queued reason: Not enough memory available on host
\S+.Needed "
E AssertionError: ImpalaBeeswaxException:
E Query aborted:Admission for query exceeded timeout 200ms in pool
default-pool. Queued reason: Not enough memory available on host
impala-ec2-centos74-m5-4xlarge-ondemand-08d6.vpc.cloudera.com:22001.Needed 2.00
GB but only 1.00 GB out of 3.00 GB was available.
E
E
E assert None
E+ where None = ('Queued reason: Not
enough memory available on host \\S+.Needed 2.00 GB but only 1.00 GB out of
2.00 GB was available.', 'ImpalaBeeswaxException:\n Query aborted:Admission for
query exceeded timeout 200ms in pool default-pool. Queued
reaso...os74-m5-4xlarge-ondemand-08d6.vpc.cloudera.com:22001.Needed 2.00 GB but
only 1.00 GB out of 3.00 GB was available.\n\n')
E+where = re.search
E+and 'ImpalaBeeswaxException:\n Query aborted:Admission for query
exceeded timeout 200ms in pool default-pool. Queued
reaso...os74-m5-4xlarge-ondemand-08d6.vpc.cloudera.com:22001.Needed 2.00 GB but
only 1.00 GB out of 3.00 GB was available.\n\n' = str(ImpalaBeeswaxException())
{noformat}
was:
texttest_heterogeneous_proc_mem_limit custom cluster test fails with the
following assertion error
{preformatted}
Stacktrace
custom_cluster/test_admission_controller.py:514: in
test_heterogeneous_proc_mem_limit
assert re.search("Queued reason: Not enough memory available on host
\S+.Needed "
E AssertionError: ImpalaBeeswaxException:
E Query aborted:Admission for query exceeded timeout 200ms in pool
default-pool. Queued reason: Not enough memory available on host
impala-ec2-centos74-m5-4xlarge-ondemand-08d6.vpc.cloudera.com:22001.Needed 2.00
GB but only 1.00 GB out of 3.00 GB was available.
E
E
E assert None
E+ where None = ('Queued reason: Not
enough memory available on host \\S+.Needed 2.00 GB but only 1.00 GB out of
2.00 GB was available.', 'ImpalaBeeswaxException:\n Query aborted:Admission for
query exceeded timeout 200ms in pool default-pool. Queued
reaso...os74-m5-4xlarge-ondemand-08d6.vpc.cloudera.com:22001.Needed 2.00 GB but
only 1.00 GB out of 3.00 GB was available.\n\n')
E+where = re.search
E+and 'ImpalaBeeswaxException:\n Query aborted:Admission for query
exceeded timeout 200ms in pool default-pool. Queued
reaso...os74-m5-4xlarge-ondemand-08d6.vpc.cloudera.com:22001.Needed 2.00 GB but
only 1.00 GB out of 3.00 GB was available.\n\n' = str(ImpalaBeeswaxException())
{preformatted}
> test_heterogeneous_proc_mem_limit: Query aborted: Not enough memory
> available on host
> --
>
> Key: IMPALA-7361
> URL: https://issues.apache.org/jira/browse/IMPALA-7361
> Project: IMPALA
> Issue Type: Bug
> Components: Backend
>Reporter: nithya
>Assignee: Bikramjeet Vig
>Priority: Major
> Labels: broken-build
>
> texttest_heterogeneous_proc_mem_limit custom cluster test fails with the
> following assertion error
> Stacktrace
> {noformat}
> custom_cluster/test_admission_controller.py:514: in
> test_heterogeneous_proc_mem_limit
> assert re.search("Queued reason: Not enough memory available on host
> \S+.Needed "
> E AssertionError: ImpalaBeeswaxException:
> E Query aborted:Admission for query exceeded timeout 200ms in pool
> default-pool. Queued reason: Not enough memory available on host
> impala-ec2-centos74-m5-4xlarge-ondemand-08d6.vpc.cloudera.com:22001.Needed
> 2.00 GB but only 1.00 GB out of 3.00 GB was available.
> E
> E
> E assert None
> E+ where None = ('Queued reason: Not
> enough memory available on host \\S+.Needed 2.00 GB but only 1.00 GB out of
> 2.00 GB was available.', 'ImpalaBeeswaxException:\n Query aborted:Admission
> for query exceeded timeout 200ms in pool default-pool. Queued
> reaso...os74-m5-4xlarge-ondemand-08d6.vpc.cloudera.com:22001.Needed 2.00 GB
> but only 1.00 GB out of 3.00 GB was available.\n\n')
> E+where = re.search
> E+and 'ImpalaBeeswaxException:\n Query aborted:Admission for query
> exceeded timeout 200ms in pool default-pool. Queued
> reaso...os74-m5-4xlarge-ondemand-08d6.vpc.cloudera.com:22001.Needed 2.00 GB
> but only 1.00 GB out of 3.00 GB was available.\n\n' =
> str(ImpalaBeeswaxException())
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (IMPALA-7361) test_heterogeneous_proc_mem_limit: Query aborted: Not enough memory available on host
[
https://issues.apache.org/jira/browse/IMPALA-7361?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
David Knupp updated IMPALA-7361:
Description:
texttest_heterogeneous_proc_mem_limit custom cluster test fails with the
following assertion error
{preformatted}
Stacktrace
custom_cluster/test_admission_controller.py:514: in
test_heterogeneous_proc_mem_limit
assert re.search("Queued reason: Not enough memory available on host
\S+.Needed "
E AssertionError: ImpalaBeeswaxException:
E Query aborted:Admission for query exceeded timeout 200ms in pool
default-pool. Queued reason: Not enough memory available on host
impala-ec2-centos74-m5-4xlarge-ondemand-08d6.vpc.cloudera.com:22001.Needed 2.00
GB but only 1.00 GB out of 3.00 GB was available.
E
E
E assert None
E+ where None = ('Queued reason: Not
enough memory available on host \\S+.Needed 2.00 GB but only 1.00 GB out of
2.00 GB was available.', 'ImpalaBeeswaxException:\n Query aborted:Admission for
query exceeded timeout 200ms in pool default-pool. Queued
reaso...os74-m5-4xlarge-ondemand-08d6.vpc.cloudera.com:22001.Needed 2.00 GB but
only 1.00 GB out of 3.00 GB was available.\n\n')
E+where = re.search
E+and 'ImpalaBeeswaxException:\n Query aborted:Admission for query
exceeded timeout 200ms in pool default-pool. Queued
reaso...os74-m5-4xlarge-ondemand-08d6.vpc.cloudera.com:22001.Needed 2.00 GB but
only 1.00 GB out of 3.00 GB was available.\n\n' = str(ImpalaBeeswaxException())
{preformatted}
was:
test_heterogeneous_proc_mem_limit fails with the following assertion error
{code:java}
AssertionError: ImpalaBeeswaxException: Query aborted:Admission for query
exceeded timeout 200ms in pool default-pool. Queued reason: Not enough memory
available on host :22001.Needed 2.00 GB but only 1.00 GB out of 3.00
GB was available. assert None + where None = ('Queued reason: Not enough memory available on host \\S+.Needed
2.00 GB but only 1.00 GB out of 2.00 GB was available.',
'ImpalaBeeswaxException:\n Query aborted:Admission for query exceeded timeout
200ms in pool default-pool. Queued reaso...:22001.Needed 2.00 GB but
only 1.00 GB out of 3.00 GB was available.\n\n') + where = re.search + and 'ImpalaBeeswaxException:\n Query
aborted:Admission for query exceeded timeout 200ms in pool default-pool. Queued
reaso...:22001.Needed 2.00 GB but only 1.00 GB out of 3.00 GB was
available.\n\n' = str(ImpalaBeeswaxException())
{code}
stack trace
{code:java}
*Stacktrace*
custom_cluster/test_admission_controller.py:514: in
test_heterogeneous_proc_mem_limit
assert re.search("Queued reason: Not enough memory available on host
\S+.Needed "
E AssertionError: ImpalaBeeswaxException:
E Query aborted:Admission for query exceeded timeout 200ms in pool
default-pool. Queued reason: Not enough memory available on host
:22001.Needed 2.00 GB but only 1.00 GB out of 3.00 GB was available.
E
E
E assert None
E + where None = ('Queued reason: Not
enough memory available on host \\S+.Needed 2.00 GB but only 1.00 GB out of
2.00 GB was available.', 'ImpalaBeeswaxException:\n Query aborted:Admission for
query exceeded timeout 200ms in pool default-pool. Queued
reaso...:22001.Needed 2.00 GB but only 1.00 GB out of 3.00 GB was
available.\n\n')
E + where = re.search
E + and 'ImpalaBeeswaxException:\n Query aborted:Admission for query
exceeded timeout 200ms in pool default-pool. Queued
reaso...:22001.Needed 2.00 GB but only 1.00 GB out of 3.00 GB was
available.\n\n' = str(ImpalaBeeswaxException())
*Standard Error*
08:55:51 MainThread: Starting State Store logging to
/data/jenkins/workspace/impala-asf-master-core-s3/repos/Impala/logs/custom_cluster_tests/statestored.INFO
08:55:52 MainThread: Starting Catalog Service logging to
/data/jenkins/workspace/impala-asf-master-core-s3/repos/Impala/logs/custom_cluster_tests/catalogd.INFO
08:55:53 MainThread: Starting Impala Daemon logging to
/data/jenkins/workspace/impala-asf-master-core-s3/repos/Impala/logs/custom_cluster_tests/impalad.INFO
08:55:54 MainThread: Starting Impala Daemon logging to
/data/jenkins/workspace/impala-asf-master-core-s3/repos/Impala/logs/custom_cluster_tests/impalad_node1.INFO
08:55:55 MainThread: Starting Impala Daemon logging to
/data/jenkins/workspace/impala-asf-master-core-s3/repos/Impala/logs/custom_cluster_tests/impalad_node2.INFO
08:55:58 MainThread: Found 3 impalad/1 statestored/1 catalogd process(es)
08:55:58 MainThread: Getting num_known_live_backends from :25000
08:55:58 MainThread: Waiting for num_known_live_backends=3. Current value: 0
08:55:59 MainThread: Getting num_known_live_backends from:25000
08:55:59 MainThread: Waiting for num_known_live_backends=3. Current value: 1
08:56:00 MainThread: Getting num_known_live_backends from :25000
08:56:00 MainThread: Waiting for
[jira] [Updated] (IMPALA-7361) test_heterogeneous_proc_mem_limit: Query aborted: Not enough memory available on host
[
https://issues.apache.org/jira/browse/IMPALA-7361?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
David Knupp updated IMPALA-7361:
Summary: test_heterogeneous_proc_mem_limit: Query aborted: Not enough
memory available on host (was: test_heterogeneous_proc_mem_limit: Query
aborted: )
> test_heterogeneous_proc_mem_limit: Query aborted: Not enough memory
> available on host
> --
>
> Key: IMPALA-7361
> URL: https://issues.apache.org/jira/browse/IMPALA-7361
> Project: IMPALA
> Issue Type: Bug
> Components: Backend
>Reporter: nithya
>Assignee: Bikramjeet Vig
>Priority: Major
> Labels: broken-build
>
> test_heterogeneous_proc_mem_limit fails with the following assertion error
>
> {code:java}
> AssertionError: ImpalaBeeswaxException: Query aborted:Admission for query
> exceeded timeout 200ms in pool default-pool. Queued reason: Not enough memory
> available on host :22001.Needed 2.00 GB but only 1.00 GB out of
> 3.00 GB was available. assert None + where None = 0x7f2b4a67c5f0>('Queued reason: Not enough memory available on host
> \\S+.Needed 2.00 GB but only 1.00 GB out of 2.00 GB was available.',
> 'ImpalaBeeswaxException:\n Query aborted:Admission for query exceeded timeout
> 200ms in pool default-pool. Queued reaso...:22001.Needed 2.00 GB
> but only 1.00 GB out of 3.00 GB was available.\n\n') + where search at 0x7f2b4a67c5f0> = re.search + and 'ImpalaBeeswaxException:\n
> Query aborted:Admission for query exceeded timeout 200ms in pool
> default-pool. Queued reaso...:22001.Needed 2.00 GB but only 1.00 GB
> out of 3.00 GB was available.\n\n' = str(ImpalaBeeswaxException())
> {code}
>
> stack trace
> {code:java}
> *Stacktrace*
> custom_cluster/test_admission_controller.py:514: in
> test_heterogeneous_proc_mem_limit
> assert re.search("Queued reason: Not enough memory available on host
> \S+.Needed "
> E AssertionError: ImpalaBeeswaxException:
> E Query aborted:Admission for query exceeded timeout 200ms in pool
> default-pool. Queued reason: Not enough memory available on host
> :22001.Needed 2.00 GB but only 1.00 GB out of 3.00 GB was available.
> E
> E
> E assert None
> E + where None = ('Queued reason: Not
> enough memory available on host \\S+.Needed 2.00 GB but only 1.00 GB out of
> 2.00 GB was available.', 'ImpalaBeeswaxException:\n Query aborted:Admission
> for query exceeded timeout 200ms in pool default-pool. Queued
> reaso...:22001.Needed 2.00 GB but only 1.00 GB out of 3.00 GB was
> available.\n\n')
> E + where = re.search
> E + and 'ImpalaBeeswaxException:\n Query aborted:Admission for query
> exceeded timeout 200ms in pool default-pool. Queued
> reaso...:22001.Needed 2.00 GB but only 1.00 GB out of 3.00 GB was
> available.\n\n' = str(ImpalaBeeswaxException())
> *Standard Error*
> 08:55:51 MainThread: Starting State Store logging to
> /data/jenkins/workspace/impala-asf-master-core-s3/repos/Impala/logs/custom_cluster_tests/statestored.INFO
> 08:55:52 MainThread: Starting Catalog Service logging to
> /data/jenkins/workspace/impala-asf-master-core-s3/repos/Impala/logs/custom_cluster_tests/catalogd.INFO
> 08:55:53 MainThread: Starting Impala Daemon logging to
> /data/jenkins/workspace/impala-asf-master-core-s3/repos/Impala/logs/custom_cluster_tests/impalad.INFO
> 08:55:54 MainThread: Starting Impala Daemon logging to
> /data/jenkins/workspace/impala-asf-master-core-s3/repos/Impala/logs/custom_cluster_tests/impalad_node1.INFO
> 08:55:55 MainThread: Starting Impala Daemon logging to
> /data/jenkins/workspace/impala-asf-master-core-s3/repos/Impala/logs/custom_cluster_tests/impalad_node2.INFO
> 08:55:58 MainThread: Found 3 impalad/1 statestored/1 catalogd process(es)
> 08:55:58 MainThread: Getting num_known_live_backends from :25000
> 08:55:58 MainThread: Waiting for num_known_live_backends=3. Current value: 0
> 08:55:59 MainThread: Getting num_known_live_backends from:25000
> 08:55:59 MainThread: Waiting for num_known_live_backends=3. Current value: 1
> 08:56:00 MainThread: Getting num_known_live_backends from :25000
> 08:56:00 MainThread: Waiting for num_known_live_backends=3. Current value: 2
> 08:56:01 MainThread: Getting num_known_live_backends from :25000
> 08:56:01 MainThread: num_known_live_backends has reached value: 3
> 08:56:01 MainThread: Getting num_known_live_backends from :25001
> 08:56:01 MainThread: num_known_live_backends has reached value: 3
> 08:56:01 MainThread: Getting num_known_live_backends from :25002
> 08:56:01 MainThread: num_known_live_backends has reached value: 3
> 08:56:01 MainThread: Impala Cluster Running with 3 nodes (3 coordinators, 3
> executors).
> MainThread: Found 3 impalad/1 statestored/1 catalogd
[jira] [Updated] (IMPALA-7361) test_heterogeneous_proc_mem_limit: Query aborted:
[
https://issues.apache.org/jira/browse/IMPALA-7361?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
David Knupp updated IMPALA-7361:
Summary: test_heterogeneous_proc_mem_limit: Query aborted: (was:
test_heterogeneous_proc_mem_limit: Assertion Failure)
> test_heterogeneous_proc_mem_limit: Query aborted:
> ---
>
> Key: IMPALA-7361
> URL: https://issues.apache.org/jira/browse/IMPALA-7361
> Project: IMPALA
> Issue Type: Bug
> Components: Backend
>Reporter: nithya
>Assignee: Bikramjeet Vig
>Priority: Major
> Labels: broken-build
>
> test_heterogeneous_proc_mem_limit fails with the following assertion error
>
> {code:java}
> AssertionError: ImpalaBeeswaxException: Query aborted:Admission for query
> exceeded timeout 200ms in pool default-pool. Queued reason: Not enough memory
> available on host :22001.Needed 2.00 GB but only 1.00 GB out of
> 3.00 GB was available. assert None + where None = 0x7f2b4a67c5f0>('Queued reason: Not enough memory available on host
> \\S+.Needed 2.00 GB but only 1.00 GB out of 2.00 GB was available.',
> 'ImpalaBeeswaxException:\n Query aborted:Admission for query exceeded timeout
> 200ms in pool default-pool. Queued reaso...:22001.Needed 2.00 GB
> but only 1.00 GB out of 3.00 GB was available.\n\n') + where search at 0x7f2b4a67c5f0> = re.search + and 'ImpalaBeeswaxException:\n
> Query aborted:Admission for query exceeded timeout 200ms in pool
> default-pool. Queued reaso...:22001.Needed 2.00 GB but only 1.00 GB
> out of 3.00 GB was available.\n\n' = str(ImpalaBeeswaxException())
> {code}
>
> stack trace
> {code:java}
> *Stacktrace*
> custom_cluster/test_admission_controller.py:514: in
> test_heterogeneous_proc_mem_limit
> assert re.search("Queued reason: Not enough memory available on host
> \S+.Needed "
> E AssertionError: ImpalaBeeswaxException:
> E Query aborted:Admission for query exceeded timeout 200ms in pool
> default-pool. Queued reason: Not enough memory available on host
> :22001.Needed 2.00 GB but only 1.00 GB out of 3.00 GB was available.
> E
> E
> E assert None
> E + where None = ('Queued reason: Not
> enough memory available on host \\S+.Needed 2.00 GB but only 1.00 GB out of
> 2.00 GB was available.', 'ImpalaBeeswaxException:\n Query aborted:Admission
> for query exceeded timeout 200ms in pool default-pool. Queued
> reaso...:22001.Needed 2.00 GB but only 1.00 GB out of 3.00 GB was
> available.\n\n')
> E + where = re.search
> E + and 'ImpalaBeeswaxException:\n Query aborted:Admission for query
> exceeded timeout 200ms in pool default-pool. Queued
> reaso...:22001.Needed 2.00 GB but only 1.00 GB out of 3.00 GB was
> available.\n\n' = str(ImpalaBeeswaxException())
> *Standard Error*
> 08:55:51 MainThread: Starting State Store logging to
> /data/jenkins/workspace/impala-asf-master-core-s3/repos/Impala/logs/custom_cluster_tests/statestored.INFO
> 08:55:52 MainThread: Starting Catalog Service logging to
> /data/jenkins/workspace/impala-asf-master-core-s3/repos/Impala/logs/custom_cluster_tests/catalogd.INFO
> 08:55:53 MainThread: Starting Impala Daemon logging to
> /data/jenkins/workspace/impala-asf-master-core-s3/repos/Impala/logs/custom_cluster_tests/impalad.INFO
> 08:55:54 MainThread: Starting Impala Daemon logging to
> /data/jenkins/workspace/impala-asf-master-core-s3/repos/Impala/logs/custom_cluster_tests/impalad_node1.INFO
> 08:55:55 MainThread: Starting Impala Daemon logging to
> /data/jenkins/workspace/impala-asf-master-core-s3/repos/Impala/logs/custom_cluster_tests/impalad_node2.INFO
> 08:55:58 MainThread: Found 3 impalad/1 statestored/1 catalogd process(es)
> 08:55:58 MainThread: Getting num_known_live_backends from :25000
> 08:55:58 MainThread: Waiting for num_known_live_backends=3. Current value: 0
> 08:55:59 MainThread: Getting num_known_live_backends from:25000
> 08:55:59 MainThread: Waiting for num_known_live_backends=3. Current value: 1
> 08:56:00 MainThread: Getting num_known_live_backends from :25000
> 08:56:00 MainThread: Waiting for num_known_live_backends=3. Current value: 2
> 08:56:01 MainThread: Getting num_known_live_backends from :25000
> 08:56:01 MainThread: num_known_live_backends has reached value: 3
> 08:56:01 MainThread: Getting num_known_live_backends from :25001
> 08:56:01 MainThread: num_known_live_backends has reached value: 3
> 08:56:01 MainThread: Getting num_known_live_backends from :25002
> 08:56:01 MainThread: num_known_live_backends has reached value: 3
> 08:56:01 MainThread: Impala Cluster Running with 3 nodes (3 coordinators, 3
> executors).
> MainThread: Found 3 impalad/1 statestored/1 catalogd process(es)
> MainThread: Getting metric: statestore.live-backends from :25010
> MainThread: Metric
[jira] [Updated] (IMPALA-7361) test_heterogeneous_proc_mem_limit: Assertion Failure
[
https://issues.apache.org/jira/browse/IMPALA-7361?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
David Knupp updated IMPALA-7361:
Summary: test_heterogeneous_proc_mem_limit: Assertion Failure (was:
test_heterogeneous_proc_mem_limit - Assertion Failure)
> test_heterogeneous_proc_mem_limit: Assertion Failure
> -
>
> Key: IMPALA-7361
> URL: https://issues.apache.org/jira/browse/IMPALA-7361
> Project: IMPALA
> Issue Type: Bug
> Components: Backend
>Reporter: nithya
>Assignee: Bikramjeet Vig
>Priority: Major
> Labels: broken-build
>
> test_heterogeneous_proc_mem_limit fails with the following assertion error
>
> {code:java}
> AssertionError: ImpalaBeeswaxException: Query aborted:Admission for query
> exceeded timeout 200ms in pool default-pool. Queued reason: Not enough memory
> available on host :22001.Needed 2.00 GB but only 1.00 GB out of
> 3.00 GB was available. assert None + where None = 0x7f2b4a67c5f0>('Queued reason: Not enough memory available on host
> \\S+.Needed 2.00 GB but only 1.00 GB out of 2.00 GB was available.',
> 'ImpalaBeeswaxException:\n Query aborted:Admission for query exceeded timeout
> 200ms in pool default-pool. Queued reaso...:22001.Needed 2.00 GB
> but only 1.00 GB out of 3.00 GB was available.\n\n') + where search at 0x7f2b4a67c5f0> = re.search + and 'ImpalaBeeswaxException:\n
> Query aborted:Admission for query exceeded timeout 200ms in pool
> default-pool. Queued reaso...:22001.Needed 2.00 GB but only 1.00 GB
> out of 3.00 GB was available.\n\n' = str(ImpalaBeeswaxException())
> {code}
>
> stack trace
> {code:java}
> *Stacktrace*
> custom_cluster/test_admission_controller.py:514: in
> test_heterogeneous_proc_mem_limit
> assert re.search("Queued reason: Not enough memory available on host
> \S+.Needed "
> E AssertionError: ImpalaBeeswaxException:
> E Query aborted:Admission for query exceeded timeout 200ms in pool
> default-pool. Queued reason: Not enough memory available on host
> :22001.Needed 2.00 GB but only 1.00 GB out of 3.00 GB was available.
> E
> E
> E assert None
> E + where None = ('Queued reason: Not
> enough memory available on host \\S+.Needed 2.00 GB but only 1.00 GB out of
> 2.00 GB was available.', 'ImpalaBeeswaxException:\n Query aborted:Admission
> for query exceeded timeout 200ms in pool default-pool. Queued
> reaso...:22001.Needed 2.00 GB but only 1.00 GB out of 3.00 GB was
> available.\n\n')
> E + where = re.search
> E + and 'ImpalaBeeswaxException:\n Query aborted:Admission for query
> exceeded timeout 200ms in pool default-pool. Queued
> reaso...:22001.Needed 2.00 GB but only 1.00 GB out of 3.00 GB was
> available.\n\n' = str(ImpalaBeeswaxException())
> *Standard Error*
> 08:55:51 MainThread: Starting State Store logging to
> /data/jenkins/workspace/impala-asf-master-core-s3/repos/Impala/logs/custom_cluster_tests/statestored.INFO
> 08:55:52 MainThread: Starting Catalog Service logging to
> /data/jenkins/workspace/impala-asf-master-core-s3/repos/Impala/logs/custom_cluster_tests/catalogd.INFO
> 08:55:53 MainThread: Starting Impala Daemon logging to
> /data/jenkins/workspace/impala-asf-master-core-s3/repos/Impala/logs/custom_cluster_tests/impalad.INFO
> 08:55:54 MainThread: Starting Impala Daemon logging to
> /data/jenkins/workspace/impala-asf-master-core-s3/repos/Impala/logs/custom_cluster_tests/impalad_node1.INFO
> 08:55:55 MainThread: Starting Impala Daemon logging to
> /data/jenkins/workspace/impala-asf-master-core-s3/repos/Impala/logs/custom_cluster_tests/impalad_node2.INFO
> 08:55:58 MainThread: Found 3 impalad/1 statestored/1 catalogd process(es)
> 08:55:58 MainThread: Getting num_known_live_backends from :25000
> 08:55:58 MainThread: Waiting for num_known_live_backends=3. Current value: 0
> 08:55:59 MainThread: Getting num_known_live_backends from:25000
> 08:55:59 MainThread: Waiting for num_known_live_backends=3. Current value: 1
> 08:56:00 MainThread: Getting num_known_live_backends from :25000
> 08:56:00 MainThread: Waiting for num_known_live_backends=3. Current value: 2
> 08:56:01 MainThread: Getting num_known_live_backends from :25000
> 08:56:01 MainThread: num_known_live_backends has reached value: 3
> 08:56:01 MainThread: Getting num_known_live_backends from :25001
> 08:56:01 MainThread: num_known_live_backends has reached value: 3
> 08:56:01 MainThread: Getting num_known_live_backends from :25002
> 08:56:01 MainThread: num_known_live_backends has reached value: 3
> 08:56:01 MainThread: Impala Cluster Running with 3 nodes (3 coordinators, 3
> executors).
> MainThread: Found 3 impalad/1 statestored/1 catalogd process(es)
> MainThread: Getting metric: statestore.live-backends from :25010
> MainThread: Metric
[jira] [Commented] (IMPALA-7376) Impala hits a DCHECK if a fragment instance fails to initialize the filter bank
[ https://issues.apache.org/jira/browse/IMPALA-7376?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16564386#comment-16564386 ] Sailesh Mukil commented on IMPALA-7376: --- CC: [~kwho] > Impala hits a DCHECK if a fragment instance fails to initialize the filter > bank > --- > > Key: IMPALA-7376 > URL: https://issues.apache.org/jira/browse/IMPALA-7376 > Project: IMPALA > Issue Type: Bug > Components: Backend >Reporter: Sailesh Mukil >Assignee: Sailesh Mukil >Priority: Critical > Labels: crash > > While Prepare()-ing a fragment instance, if we fail to initialize the runtime > filter bank, we will exit FIS::Prepare() without acquiring a thread token > (AcquireThreadToken()): > https://github.com/apache/impala/blob/316b17ac55adb3d1deeb1289b4045688269b201d/be/src/runtime/fragment-instance-state.cc#L135-L139 > FIS::Finalize() is called *always* regardless of whether the fragment > instance succeeded or failed. And FIS::Finalize() tries to > ReleaseThreadToken() even though it might not have gotten acquired: > https://github.com/apache/impala/blob/316b17ac55adb3d1deeb1289b4045688269b201d/be/src/runtime/fragment-instance-state.cc#L464 > , causing a DCHECK to be hit. > This was found while I was adding global debug actions (IMPALA-7046) to the > FIS. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Created] (IMPALA-7376) Impala hits a DCHECK if a fragment instance fails to initialize the filter bank
Sailesh Mukil created IMPALA-7376: - Summary: Impala hits a DCHECK if a fragment instance fails to initialize the filter bank Key: IMPALA-7376 URL: https://issues.apache.org/jira/browse/IMPALA-7376 Project: IMPALA Issue Type: Bug Components: Backend Reporter: Sailesh Mukil Assignee: Sailesh Mukil While Prepare()-ing a fragment instance, if we fail to initialize the runtime filter bank, we will exit FIS::Prepare() without acquiring a thread token (AcquireThreadToken()): https://github.com/apache/impala/blob/316b17ac55adb3d1deeb1289b4045688269b201d/be/src/runtime/fragment-instance-state.cc#L135-L139 FIS::Finalize() is called *always* regardless of whether the fragment instance succeeded or failed. And FIS::Finalize() tries to ReleaseThreadToken() even though it might not have gotten acquired: https://github.com/apache/impala/blob/316b17ac55adb3d1deeb1289b4045688269b201d/be/src/runtime/fragment-instance-state.cc#L464 , causing a DCHECK to be hit. This was found while I was adding global debug actions (IMPALA-7046) to the FIS. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Closed] (IMPALA-7373) Create a script to generate a JUnitXML symptom.
[ https://issues.apache.org/jira/browse/IMPALA-7373?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lars Volker closed IMPALA-7373. --- Resolution: Invalid This seems to be the wrong place to track. > Create a script to generate a JUnitXML symptom. > > > Key: IMPALA-7373 > URL: https://issues.apache.org/jira/browse/IMPALA-7373 > Project: IMPALA > Issue Type: Improvement >Reporter: Lars Volker >Assignee: David Knupp >Priority: Major > > A symptom can contain a chunk of a log file, a starting timestamp, and a > duration. See the xml files generated by the pytests. See the python package > junit-xml. A symptom can exist for success or failure, and in general, it is > better to always generate a symptom. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Work started] (IMPALA-7354) Validate memory estimates for standard workloads in planner tests
[ https://issues.apache.org/jira/browse/IMPALA-7354?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on IMPALA-7354 started by Tim Armstrong. - > Validate memory estimates for standard workloads in planner tests > - > > Key: IMPALA-7354 > URL: https://issues.apache.org/jira/browse/IMPALA-7354 > Project: IMPALA > Issue Type: Sub-task > Components: Frontend >Reporter: Tim Armstrong >Assignee: Tim Armstrong >Priority: Major > Labels: resource-management > > Currently we have targeted tests for estimates of individual plan nodes, but > it would be good to get coverage of some more interesting query shapes. We > should verify the total memory estimate produced by the planner for key > workloads - TPC-DS, TPC-H, TPC-H Nested and TPC-H Kudu -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Created] (IMPALA-7374) Impala Doc: Doc DATE type
Alex Rodoni created IMPALA-7374: --- Summary: Impala Doc: Doc DATE type Key: IMPALA-7374 URL: https://issues.apache.org/jira/browse/IMPALA-7374 Project: IMPALA Issue Type: Sub-task Components: Docs Reporter: Alex Rodoni Assignee: Alex Rodoni -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Comment Edited] (IMPALA-7335) Assertion Failure - test_corrupt_files
[
https://issues.apache.org/jira/browse/IMPALA-7335?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16562750#comment-16562750
]
Pooja Nilangekar edited comment on IMPALA-7335 at 7/31/18 6:53 PM:
---
This appears to be a flaky issue. From the cluster logs, it was not clear what
the cause of failure was.
Since the file is not generated on the fly, it can't be a case of failure due
to errors while generating the file. Additionally, the code path to determine
invalid metadata is deterministic and should be hit as long as the file
contains fewer values than expected. In the cluster logs, the test failed
because all fragments completed successfully.
I tried to recreate the error by running the test in an infinite loop with the
same build parameters. At the end of a 24 hour run, it completed over 1
successful iterations without any failure. We can wait and see if the issue is
reproduced in the next week or so. If not, we can close it until it reappears.
was (Author: poojanilangekar):
This appears to be a flaky issue. From the cluster logs, it was not clear what
the cause of failure was.
Since the file is not generated on the fly, it can't be a case of failure due
to errors while generating the file. Additionally, the code path to determine
invalid metadata is deterministic and should be hit as long as the file
contains fewer values than expected. In the cluster logs, the test failed
because all fragments completed successfully.
I tried to recreate the error by running the test in an infinite loop with the
same build parameters. As of now, it has had over 3700 successful iterations
without any failure. We can wait and see if the issue is reproduced in the next
week or so. If not, we can close it until it reappears.
> Assertion Failure - test_corrupt_files
> --
>
> Key: IMPALA-7335
> URL: https://issues.apache.org/jira/browse/IMPALA-7335
> Project: IMPALA
> Issue Type: Bug
>Affects Versions: Impala 3.1.0
>Reporter: nithya
>Assignee: Pooja Nilangekar
>Priority: Critical
> Labels: broken-build
>
> test_corrupt_files fails
>
> query_test.test_scanners.TestParquet.test_corrupt_files[exec_option:
> \\{'batch_size': 0, 'num_nodes': 0, 'disable_codegen_rows_threshold': 0,
> 'disable_codegen': False, 'abort_on_error': 1, 'debug_action': None,
> 'exec_single_node_rows_threshold': 0} | table_format: parquet/none] (from
> pytest)
>
> {code:java}
> Error Message
> query_test/test_scanners.py:300: in test_corrupt_files
> self.run_test_case('QueryTest/parquet-abort-on-error', vector)
> common/impala_test_suite.py:420: in run_test_case assert False, "Expected
> exception: %s" % expected_str E AssertionError: Expected exception: Column
> metadata states there are 11 values, but read 10 values from column id.
> STACKTRACE
> query_test/test_scanners.py:300: in test_corrupt_files
> self.run_test_case('QueryTest/parquet-abort-on-error', vector)
> common/impala_test_suite.py:420: in run_test_case
> assert False, "Expected exception: %s" % expected_str
> E AssertionError: Expected exception: Column metadata states there are 11
> values, but read 10 values from column id.
> Standard Error
> -- executing against localhost:21000
> use functional_parquet;
> SET batch_size=0;
> SET num_nodes=0;
> SET disable_codegen_rows_threshold=0;
> SET disable_codegen=False;
> SET abort_on_error=0;
> SET exec_single_node_rows_threshold=0;
> -- executing against localhost:21000
> set num_nodes=1;
> -- executing against localhost:21000
> set num_scanner_threads=1;
> -- executing against localhost:21000
> select id, cnt from bad_column_metadata t, (select count(*) cnt from
> t.int_array) v;
> -- executing against localhost:21000
> SET NUM_NODES="0";
> -- executing against localhost:21000
> SET NUM_SCANNER_THREADS="0";
> -- executing against localhost:21000
> set num_nodes=1;
> -- executing against localhost:21000
> set num_scanner_threads=1;
> -- executing against localhost:21000
> select id from bad_column_metadata;
> -- executing against localhost:21000
> SET NUM_NODES="0";
> -- executing against localhost:21000
> SET NUM_SCANNER_THREADS="0";
> -- executing against localhost:21000
> SELECT * from bad_parquet_strings_negative_len;
> -- executing against localhost:21000
> SELECT * from bad_parquet_strings_out_of_bounds;
> -- executing against localhost:21000
> use functional_parquet;
> SET batch_size=0;
> SET num_nodes=0;
> SET disable_codegen_rows_threshold=0;
> SET disable_codegen=False;
> SET abort_on_error=1;
> SET exec_single_node_rows_threshold=0;
> -- executing against localhost:21000
> set num_nodes=1;
> -- executing against localhost:21000
> set num_scanner_threads=1;
> -- executing against localhost:21000
> select id, cnt from bad_column_metadata t,
[jira] [Created] (IMPALA-7373) Create a script to generate a JUnitXML symptom.
Lars Volker created IMPALA-7373: --- Summary: Create a script to generate a JUnitXML symptom. Key: IMPALA-7373 URL: https://issues.apache.org/jira/browse/IMPALA-7373 Project: IMPALA Issue Type: Improvement Reporter: Lars Volker Assignee: David Knupp A symptom can contain a chunk of a log file, a starting timestamp, and a duration. See the xml files generated by the pytests. See the python package junit-xml. A symptom can exist for success or failure, and in general, it is better to always generate a symptom. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Created] (IMPALA-7372) Generate Jenkins Symptoms for Common Problems
Lars Volker created IMPALA-7372: --- Summary: Generate Jenkins Symptoms for Common Problems Key: IMPALA-7372 URL: https://issues.apache.org/jira/browse/IMPALA-7372 Project: IMPALA Issue Type: Epic Components: Infrastructure Affects Versions: Impala 3.0 Reporter: Lars Volker Common failures currently don't emit proper Jenkins Symptoms, making build failure triage harder than it should be. This Epic tracks work on adding more symptoms throughout our test execution. A symptom can contain a chunk of a log file, a starting timestamp, and a duration. See the xml files generated by the pytests. See the python package junit-xml. A symptom can exist for success or failure, and in general, it is better to always generate a symptom. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Commented] (IMPALA-7364) Upgrade RapidJson to the latest version
[ https://issues.apache.org/jira/browse/IMPALA-7364?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16564169#comment-16564169 ] Tim Armstrong commented on IMPALA-7364: --- How about I make the native-toolchain change to add the new version since it'll be easier for me to test? > Upgrade RapidJson to the latest version > --- > > Key: IMPALA-7364 > URL: https://issues.apache.org/jira/browse/IMPALA-7364 > Project: IMPALA > Issue Type: Improvement >Reporter: Quanlong Huang >Priority: Major > > Currently, we're using rapidjson-0.11. It's old enough and can't show > detailed parse errors. In the implementation of built-in JSON function in > IMPALA-376, we can't show warnings on how we fail to parse a json string and > where is the error location like this: > http://rapidjson.org/group___r_a_p_i_d_j_s_o_n___e_r_r_o_r_s.html#structrapidjson_1_1_parse_result > This JIRA aim to upgrade the version of RapidJson to v1.1. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Commented] (IMPALA-7364) Upgrade RapidJson to the latest version
[
https://issues.apache.org/jira/browse/IMPALA-7364?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16564152#comment-16564152
]
Tim Armstrong commented on IMPALA-7364:
---
[~stiga-huang] yeah the source S3 buckets are managed by Cloudera but you can
build locally. We do code reviews on gerrit.cloudera.org under the
native-toolchain project. We need to improve the docs on how to contribute.
I uploaded rapidjson-1.1.0.zip to the source bucket:
{noformat}
$ sha1sum rapidjson-1.1.0.zip
0fe7b4f7b83df4b3d517f4a202f3a383af7a0818 rapidjson-1.1.0.zip
{noformat}
I think building it with "./build.sh rapidjson 1.1.0" might just work with no
modifications since it's a header-only library
> Upgrade RapidJson to the latest version
> ---
>
> Key: IMPALA-7364
> URL: https://issues.apache.org/jira/browse/IMPALA-7364
> Project: IMPALA
> Issue Type: Improvement
>Reporter: Quanlong Huang
>Priority: Major
>
> Currently, we're using rapidjson-0.11. It's old enough and can't show
> detailed parse errors. In the implementation of built-in JSON function in
> IMPALA-376, we can't show warnings on how we fail to parse a json string and
> where is the error location like this:
> http://rapidjson.org/group___r_a_p_i_d_j_s_o_n___e_r_r_o_r_s.html#structrapidjson_1_1_parse_result
> This JIRA aim to upgrade the version of RapidJson to v1.1.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Closed] (IMPALA-7222) [DOCS] authorization_proxy_user_config needs clarification
[
https://issues.apache.org/jira/browse/IMPALA-7222?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alex Rodoni closed IMPALA-7222.
---
Resolution: Fixed
Fix Version/s: Impala 3.1.0
> [DOCS] authorization_proxy_user_config needs clarification
> --
>
> Key: IMPALA-7222
> URL: https://issues.apache.org/jira/browse/IMPALA-7222
> Project: IMPALA
> Issue Type: Bug
> Components: Docs
>Reporter: Zsombor Fedor
>Assignee: Alex Rodoni
>Priority: Minor
> Fix For: Impala 3.1.0
>
>
> Please refer to the following Impala documentation:
> [https://impala.apache.org/docs/build3x/html/topics/impala_delegation.html]
>
> The following clarifications needed for better understanding:
> When using this option --authorized_proxy_user_config= 'user1=user2' :
> * authentication is happening based on the user on the left hand side
> (_user1_)
> * authorization is happening based on the right hand side user(s) (_user2_)
> * you can list the users to enable the delegation for them using the
> delimiter stated in authorized_proxy_user_config_delimiter switch (default:
> ",") eg.: _user1_=_user2_,_user3_,_user4_ or enable for any user by *. More
> entries delimited by ";" (_user1_=_user2_;_user3_=_user4_)
> * it is not straightforward (at least it wasn't for me) that the delegation
> doesn't happen automatically when connecting with _user1,_ the client must be
> able to provide delegated username when opening the session (via
> "DelegationUID"). ((_user2_ in this case))
> * it is not necessary for _user1_ to have the permission to access/edit files
> * it is not necessary for _user2_ to have access to the service via Kerberos
> * delegated username must exist in the OS to be able to match the permissions
> * in Impala user() will be _user1_ and effective_user() will be _user2_
> * {color:#00}it is a security matter in the client to prevent
> unauthorized access for the delegate-able users{color}
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Commented] (IMPALA-6214) Determine and warn about stuck fragment instances
[ https://issues.apache.org/jira/browse/IMPALA-6214?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16564064#comment-16564064 ] Lars Volker commented on IMPALA-6214: - One place for this check could be in or around the loop in FragmentInstanceState::ExecInternal(). We need to come up with a way to expose this on a per-query basis, either on a status page or (preferably) in the profile, so that other tools can parse this more easily. In addition, we should think about how we can prevent false posiitives, i.e. only report an instance that blocks, but is not blocked by any of its inputs. > Determine and warn about stuck fragment instances > - > > Key: IMPALA-6214 > URL: https://issues.apache.org/jira/browse/IMPALA-6214 > Project: IMPALA > Issue Type: Bug > Components: Backend >Reporter: Lars Volker >Assignee: Pranay Singh >Priority: Major > Labels: observability, supportability > > It would be great to have a programmatic way to determine if a fragment > instance is hung by checking if it’s producing rows periodically. A fragment > instance can appear to be not making progress because its input operator / > fragment may be hung (e.g.the probe side of a join will not be able to make > much progress until the build side is done and the build side itself could be > another chain of joins). It'd be much easier to resolve this dependency chain > programmatically to find the root of the cascade of delay. > Details of algorithm are still unclear. It may be easier to include exec node > states in query profile and analyze those, but this probably requires taking > multiple snapshots of the query profiles over time. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Assigned] (IMPALA-7371) TestInsertQueries.test_insert fails on S3 with 0 rows returned
[
https://issues.apache.org/jira/browse/IMPALA-7371?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
David Knupp reassigned IMPALA-7371:
---
Assignee: Sailesh Mukil (was: David Knupp)
> TestInsertQueries.test_insert fails on S3 with 0 rows returned
> --
>
> Key: IMPALA-7371
> URL: https://issues.apache.org/jira/browse/IMPALA-7371
> Project: IMPALA
> Issue Type: Bug
> Components: Backend
>Affects Versions: Impala 3.1.0
>Reporter: David Knupp
>Assignee: Sailesh Mukil
>Priority: Critical
>
> Stacktrace
> {noformat}
> query_test/test_insert.py:118: in test_insert
> multiple_impalad=vector.get_value('exec_option')['sync_ddl'] == 1)
> /data/jenkins/workspace/impala-cdh6.0.x-core-s3/repos/Impala/tests/common/impala_test_suite.py:426:
> in run_test_case
> self.__verify_results_and_errors(vector, test_section, result, use_db)
> /data/jenkins/workspace/impala-cdh6.0.x-core-s3/repos/Impala/tests/common/impala_test_suite.py:299:
> in __verify_results_and_errors
> replace_filenames_with_placeholder)
> /data/jenkins/workspace/impala-cdh6.0.x-core-s3/repos/Impala/tests/common/test_result_verifier.py:434:
> in verify_raw_results
> VERIFIER_MAP[verifier](expected, actual)
> /data/jenkins/workspace/impala-cdh6.0.x-core-s3/repos/Impala/tests/common/test_result_verifier.py:261:
> in verify_query_result_is_equal
> assert expected_results == actual_results
> E assert Comparing QueryTestResults (expected vs actual):
> E 75,false,0,0,0,0,0,0,'04/01/09','0' != None
> E 76,true,1,1,1,10,1.10023841858,10.1,'04/01/09','1' != None
> E 77,false,2,2,2,20,2.20047683716,20.2,'04/01/09','2' != None
> E 78,true,3,3,3,30,3.29952316284,30.3,'04/01/09','3' != None
> E 79,false,4,4,4,40,4.40095367432,40.4,'04/01/09','4' != None
> E 80,true,5,5,5,50,5.5,50.5,'04/01/09','5' != None
> E 81,false,6,6,6,60,6.59904632568,60.6,'04/01/09','6' != None
> E 82,true,7,7,7,70,7.69809265137,70.7,'04/01/09','7' != None
> E 83,false,8,8,8,80,8.80190734863,80.8,'04/01/09','8' != None
> E 84,true,9,9,9,90,9.89618530273,90.91,'04/01/09','9' !=
> None
> E 85,false,0,0,0,0,0,0,'04/02/09','0' != None
> E 86,true,1,1,1,10,1.10023841858,10.1,'04/02/09','1' != None
> E 87,false,2,2,2,20,2.20047683716,20.2,'04/02/09','2' != None
> E 88,true,3,3,3,30,3.29952316284,30.3,'04/02/09','3' != None
> E 89,false,4,4,4,40,4.40095367432,40.4,'04/02/09','4' != None
> E 90,true,5,5,5,50,5.5,50.5,'04/02/09','5' != None
> E 91,false,6,6,6,60,6.59904632568,60.6,'04/02/09','6' != None
> E 92,true,7,7,7,70,7.69809265137,70.7,'04/02/09','7' != None
> E 93,false,8,8,8,80,8.80190734863,80.8,'04/02/09','8' != None
> E 94,true,9,9,9,90,9.89618530273,90.91,'04/02/09','9' !=
> None
> E 95,false,0,0,0,0,0,0,'04/03/09','0' != None
> E 96,true,1,1,1,10,1.10023841858,10.1,'04/03/09','1' != None
> E 97,false,2,2,2,20,2.20047683716,20.2,'04/03/09','2' != None
> E 98,true,3,3,3,30,3.29952316284,30.3,'04/03/09','3' != None
> E 99,false,4,4,4,40,4.40095367432,40.4,'04/03/09','4' != None
> E Number of rows returned (expected vs actual): 25 != 0
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Assigned] (IMPALA-7371) TestInsertQueries.test_insert fails on S3 with 0 rows returned
[
https://issues.apache.org/jira/browse/IMPALA-7371?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
David Knupp reassigned IMPALA-7371:
---
Assignee: David Knupp
> TestInsertQueries.test_insert fails on S3 with 0 rows returned
> --
>
> Key: IMPALA-7371
> URL: https://issues.apache.org/jira/browse/IMPALA-7371
> Project: IMPALA
> Issue Type: Bug
> Components: Backend
>Affects Versions: Impala 3.1.0
>Reporter: David Knupp
>Assignee: David Knupp
>Priority: Critical
>
> Stacktrace
> {noformat}
> query_test/test_insert.py:118: in test_insert
> multiple_impalad=vector.get_value('exec_option')['sync_ddl'] == 1)
> /data/jenkins/workspace/impala-cdh6.0.x-core-s3/repos/Impala/tests/common/impala_test_suite.py:426:
> in run_test_case
> self.__verify_results_and_errors(vector, test_section, result, use_db)
> /data/jenkins/workspace/impala-cdh6.0.x-core-s3/repos/Impala/tests/common/impala_test_suite.py:299:
> in __verify_results_and_errors
> replace_filenames_with_placeholder)
> /data/jenkins/workspace/impala-cdh6.0.x-core-s3/repos/Impala/tests/common/test_result_verifier.py:434:
> in verify_raw_results
> VERIFIER_MAP[verifier](expected, actual)
> /data/jenkins/workspace/impala-cdh6.0.x-core-s3/repos/Impala/tests/common/test_result_verifier.py:261:
> in verify_query_result_is_equal
> assert expected_results == actual_results
> E assert Comparing QueryTestResults (expected vs actual):
> E 75,false,0,0,0,0,0,0,'04/01/09','0' != None
> E 76,true,1,1,1,10,1.10023841858,10.1,'04/01/09','1' != None
> E 77,false,2,2,2,20,2.20047683716,20.2,'04/01/09','2' != None
> E 78,true,3,3,3,30,3.29952316284,30.3,'04/01/09','3' != None
> E 79,false,4,4,4,40,4.40095367432,40.4,'04/01/09','4' != None
> E 80,true,5,5,5,50,5.5,50.5,'04/01/09','5' != None
> E 81,false,6,6,6,60,6.59904632568,60.6,'04/01/09','6' != None
> E 82,true,7,7,7,70,7.69809265137,70.7,'04/01/09','7' != None
> E 83,false,8,8,8,80,8.80190734863,80.8,'04/01/09','8' != None
> E 84,true,9,9,9,90,9.89618530273,90.91,'04/01/09','9' !=
> None
> E 85,false,0,0,0,0,0,0,'04/02/09','0' != None
> E 86,true,1,1,1,10,1.10023841858,10.1,'04/02/09','1' != None
> E 87,false,2,2,2,20,2.20047683716,20.2,'04/02/09','2' != None
> E 88,true,3,3,3,30,3.29952316284,30.3,'04/02/09','3' != None
> E 89,false,4,4,4,40,4.40095367432,40.4,'04/02/09','4' != None
> E 90,true,5,5,5,50,5.5,50.5,'04/02/09','5' != None
> E 91,false,6,6,6,60,6.59904632568,60.6,'04/02/09','6' != None
> E 92,true,7,7,7,70,7.69809265137,70.7,'04/02/09','7' != None
> E 93,false,8,8,8,80,8.80190734863,80.8,'04/02/09','8' != None
> E 94,true,9,9,9,90,9.89618530273,90.91,'04/02/09','9' !=
> None
> E 95,false,0,0,0,0,0,0,'04/03/09','0' != None
> E 96,true,1,1,1,10,1.10023841858,10.1,'04/03/09','1' != None
> E 97,false,2,2,2,20,2.20047683716,20.2,'04/03/09','2' != None
> E 98,true,3,3,3,30,3.29952316284,30.3,'04/03/09','3' != None
> E 99,false,4,4,4,40,4.40095367432,40.4,'04/03/09','4' != None
> E Number of rows returned (expected vs actual): 25 != 0
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Commented] (IMPALA-7209) Disallow self referencing ALTER VIEW statments
[
https://issues.apache.org/jira/browse/IMPALA-7209?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16564039#comment-16564039
]
Pooja Nilangekar commented on IMPALA-7209:
--
[~tarmstrong] Done.
> Disallow self referencing ALTER VIEW statments
> --
>
> Key: IMPALA-7209
> URL: https://issues.apache.org/jira/browse/IMPALA-7209
> Project: IMPALA
> Issue Type: Bug
> Components: Frontend
>Reporter: Pooja Nilangekar
>Assignee: Pooja Nilangekar
>Priority: Major
> Fix For: Impala 3.1.0
>
>
> Currently, an ALTER VIEW statement accepts self referencing definitions.
> However, upon querying the altered view, the analyzer is unable to find the
> reference and hence throws a StackOverflowError: null error.
> The expected behavior would be to throw an AnalysisException while executing
> the alter view statement.
>
> Example:
>
> {code:java}
> [localhost:21000] default> create view foo as select * from
> functional.alltypes;
> Query: create view foo as select * from functional.alltypes
> Query submitted at: 2018-07-03 11:36:48 (Coordinator:
> http://pooja-OptiPlex-7040:25000)
> Query progress can be monitored at:
> http://pooja-OptiPlex-7040:25000/query_plan?query_id=614e03efbcb4d8b1:586a4bad
> ++
> | summary |
> ++
> | View has been created. |
> ++
> Fetched 1 row(s) in 0.26s
> [localhost:21000] default> alter view foo as select * from foo;
> Query: alter view foo as select * from foo
> ++
> | summary |
> ++
> | View has been altered. |
> ++
> Fetched 1 row(s) in 5.65s
> [localhost:21000] default> select * from foo;
> Query: select * from foo
> Query submitted at: 2018-07-03 11:37:12 (Coordinator:
> http://pooja-OptiPlex-7040:25000)
> ERROR: StackOverflowError: null
> {code}
>
> The select statement on the view fails because the analyzer can't resolve its
> reference. Other databases return failure during the alter view statement
> because stating.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Updated] (IMPALA-7209) Disallow self referencing ALTER VIEW statments
[
https://issues.apache.org/jira/browse/IMPALA-7209?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Pooja Nilangekar updated IMPALA-7209:
-
Fix Version/s: Impala 3.1.0
> Disallow self referencing ALTER VIEW statments
> --
>
> Key: IMPALA-7209
> URL: https://issues.apache.org/jira/browse/IMPALA-7209
> Project: IMPALA
> Issue Type: Bug
> Components: Frontend
>Reporter: Pooja Nilangekar
>Assignee: Pooja Nilangekar
>Priority: Major
> Fix For: Impala 3.1.0
>
>
> Currently, an ALTER VIEW statement accepts self referencing definitions.
> However, upon querying the altered view, the analyzer is unable to find the
> reference and hence throws a StackOverflowError: null error.
> The expected behavior would be to throw an AnalysisException while executing
> the alter view statement.
>
> Example:
>
> {code:java}
> [localhost:21000] default> create view foo as select * from
> functional.alltypes;
> Query: create view foo as select * from functional.alltypes
> Query submitted at: 2018-07-03 11:36:48 (Coordinator:
> http://pooja-OptiPlex-7040:25000)
> Query progress can be monitored at:
> http://pooja-OptiPlex-7040:25000/query_plan?query_id=614e03efbcb4d8b1:586a4bad
> ++
> | summary |
> ++
> | View has been created. |
> ++
> Fetched 1 row(s) in 0.26s
> [localhost:21000] default> alter view foo as select * from foo;
> Query: alter view foo as select * from foo
> ++
> | summary |
> ++
> | View has been altered. |
> ++
> Fetched 1 row(s) in 5.65s
> [localhost:21000] default> select * from foo;
> Query: select * from foo
> Query submitted at: 2018-07-03 11:37:12 (Coordinator:
> http://pooja-OptiPlex-7040:25000)
> ERROR: StackOverflowError: null
> {code}
>
> The select statement on the view fails because the analyzer can't resolve its
> reference. Other databases return failure during the alter view statement
> because stating.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Created] (IMPALA-7371) TestInsertQueries.test_insert fails on S3 with 0 rows returned
David Knupp created IMPALA-7371:
---
Summary: TestInsertQueries.test_insert fails on S3 with 0 rows
returned
Key: IMPALA-7371
URL: https://issues.apache.org/jira/browse/IMPALA-7371
Project: IMPALA
Issue Type: Bug
Components: Backend
Affects Versions: Impala 3.1.0
Reporter: David Knupp
Stacktrace
{noformat}
query_test/test_insert.py:118: in test_insert
multiple_impalad=vector.get_value('exec_option')['sync_ddl'] == 1)
/data/jenkins/workspace/impala-cdh6.0.x-core-s3/repos/Impala/tests/common/impala_test_suite.py:426:
in run_test_case
self.__verify_results_and_errors(vector, test_section, result, use_db)
/data/jenkins/workspace/impala-cdh6.0.x-core-s3/repos/Impala/tests/common/impala_test_suite.py:299:
in __verify_results_and_errors
replace_filenames_with_placeholder)
/data/jenkins/workspace/impala-cdh6.0.x-core-s3/repos/Impala/tests/common/test_result_verifier.py:434:
in verify_raw_results
VERIFIER_MAP[verifier](expected, actual)
/data/jenkins/workspace/impala-cdh6.0.x-core-s3/repos/Impala/tests/common/test_result_verifier.py:261:
in verify_query_result_is_equal
assert expected_results == actual_results
E assert Comparing QueryTestResults (expected vs actual):
E 75,false,0,0,0,0,0,0,'04/01/09','0' != None
E 76,true,1,1,1,10,1.10023841858,10.1,'04/01/09','1' != None
E 77,false,2,2,2,20,2.20047683716,20.2,'04/01/09','2' != None
E 78,true,3,3,3,30,3.29952316284,30.3,'04/01/09','3' != None
E 79,false,4,4,4,40,4.40095367432,40.4,'04/01/09','4' != None
E 80,true,5,5,5,50,5.5,50.5,'04/01/09','5' != None
E 81,false,6,6,6,60,6.59904632568,60.6,'04/01/09','6' != None
E 82,true,7,7,7,70,7.69809265137,70.7,'04/01/09','7' != None
E 83,false,8,8,8,80,8.80190734863,80.8,'04/01/09','8' != None
E 84,true,9,9,9,90,9.89618530273,90.91,'04/01/09','9' !=
None
E 85,false,0,0,0,0,0,0,'04/02/09','0' != None
E 86,true,1,1,1,10,1.10023841858,10.1,'04/02/09','1' != None
E 87,false,2,2,2,20,2.20047683716,20.2,'04/02/09','2' != None
E 88,true,3,3,3,30,3.29952316284,30.3,'04/02/09','3' != None
E 89,false,4,4,4,40,4.40095367432,40.4,'04/02/09','4' != None
E 90,true,5,5,5,50,5.5,50.5,'04/02/09','5' != None
E 91,false,6,6,6,60,6.59904632568,60.6,'04/02/09','6' != None
E 92,true,7,7,7,70,7.69809265137,70.7,'04/02/09','7' != None
E 93,false,8,8,8,80,8.80190734863,80.8,'04/02/09','8' != None
E 94,true,9,9,9,90,9.89618530273,90.91,'04/02/09','9' !=
None
E 95,false,0,0,0,0,0,0,'04/03/09','0' != None
E 96,true,1,1,1,10,1.10023841858,10.1,'04/03/09','1' != None
E 97,false,2,2,2,20,2.20047683716,20.2,'04/03/09','2' != None
E 98,true,3,3,3,30,3.29952316284,30.3,'04/03/09','3' != None
E 99,false,4,4,4,40,4.40095367432,40.4,'04/03/09','4' != None
E Number of rows returned (expected vs actual): 25 != 0
{noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Commented] (IMPALA-7209) Disallow self referencing ALTER VIEW statments
[
https://issues.apache.org/jira/browse/IMPALA-7209?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16564002#comment-16564002
]
Tim Armstrong commented on IMPALA-7209:
---
[~poojanilangekar] can you set the fix version?
> Disallow self referencing ALTER VIEW statments
> --
>
> Key: IMPALA-7209
> URL: https://issues.apache.org/jira/browse/IMPALA-7209
> Project: IMPALA
> Issue Type: Bug
> Components: Frontend
>Reporter: Pooja Nilangekar
>Assignee: Pooja Nilangekar
>Priority: Major
>
> Currently, an ALTER VIEW statement accepts self referencing definitions.
> However, upon querying the altered view, the analyzer is unable to find the
> reference and hence throws a StackOverflowError: null error.
> The expected behavior would be to throw an AnalysisException while executing
> the alter view statement.
>
> Example:
>
> {code:java}
> [localhost:21000] default> create view foo as select * from
> functional.alltypes;
> Query: create view foo as select * from functional.alltypes
> Query submitted at: 2018-07-03 11:36:48 (Coordinator:
> http://pooja-OptiPlex-7040:25000)
> Query progress can be monitored at:
> http://pooja-OptiPlex-7040:25000/query_plan?query_id=614e03efbcb4d8b1:586a4bad
> ++
> | summary |
> ++
> | View has been created. |
> ++
> Fetched 1 row(s) in 0.26s
> [localhost:21000] default> alter view foo as select * from foo;
> Query: alter view foo as select * from foo
> ++
> | summary |
> ++
> | View has been altered. |
> ++
> Fetched 1 row(s) in 5.65s
> [localhost:21000] default> select * from foo;
> Query: select * from foo
> Query submitted at: 2018-07-03 11:37:12 (Coordinator:
> http://pooja-OptiPlex-7040:25000)
> ERROR: StackOverflowError: null
> {code}
>
> The select statement on the view fails because the analyzer can't resolve its
> reference. Other databases return failure during the alter view statement
> because stating.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Commented] (IMPALA-6671) Metadata operations that modify a table blocks topic updates for other unrelated operations
[
https://issues.apache.org/jira/browse/IMPALA-6671?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16563974#comment-16563974
]
Todd Lipcon commented on IMPALA-6671:
-
Been brainstorming a possible route to fixing this...
Currently, the flow for operations that mutate a table object is:
{code}
lock version number (write)
| lock table
| | increment catalog version number and remember the newVersion
|-+-unlock version number
|
| modify table in place
| set version number of table to newVersion
unlock table
{code}
The "update collector" thread looks something like:
{code}
fromVersion = last version sent out
lock versionLock
toVersion = get version number
unlock versionLock
foreach table:
if table version > toVersion:
continue (skip the table)
lock table (blocking here causes IMPALA-6671)
if the version number is within the range we are collecting, copy it as
thrift into the delta
unlock table
{code}
Essentially, when the update collector hits a table which is locked (being
updated) it doesn't know whether the "newVersion" that it will have post-lock
is within the range to be collected or not, because that "newVersion" is only
stored within a local variable of the mutating thread. Additionally, it can't
_affect_ the new version. If it were to just do a "trylock" and skip this table
on failure, then in the next collection loop, it would incorrectly determine
that it already sent this table, and we'd end up with a case where we miss
propagating an update -- no good.
So, I've been toying with an idea loosely inspired by CockroachDB's handling of
read-write conflicts. Consider the "write" transaction here to be the mutating
thread, and the "read" transaction to be the "collector" thread. In the current
design, the writing transaction is essentially assigning a timestamp at its
start. But, there is no requirement that it commit with that same timestamp
that it started with -- it would be equally happy with any higher timestamp.
So, what if we have the read transaction "push" the write transaction's commit
timestamp out of the range to be read, essentially deferring it into the next
collection loop. The flow would look something like:
Mutator:
{code}
lock version number (write)
| lock table
| | table.pendingWriteVersion = catalog.incrementVersionNumber()
|-+-unlock version number
|
| modify table in place
| table.setVersionNumber(table.pendingWriteVersion)
| table.pendingWriteVersion = -1
unlock table
{code}
Update collector:
{code}
fromVersion = last version sent out
lock versionLock
toVersion = get version number
unlock versionLock
foreach table:
if table version > toVersion:
continue (skip the table)
try to lock table:
if successful, handle normally
else:
# push the write transaction to commit in the next update
table.pendingWriteVersion = max(table.pendingWriteVersion, toVersion + 1)
{code}
The pseudocode as written above has some races, since we need to ensure that
the lock and the 'pendingWriteVersion', etc, are operated upon atomically, but
I think that's pretty solvable. We also need to have similar treatment of the
"max skipped updates" thing to ensure that a constantly-refreshed table
_eventually_ gets sent out, but we already have that code path today.
Thoughts?
> Metadata operations that modify a table blocks topic updates for other
> unrelated operations
> ---
>
> Key: IMPALA-6671
> URL: https://issues.apache.org/jira/browse/IMPALA-6671
> Project: IMPALA
> Issue Type: Bug
> Components: Catalog
>Affects Versions: Impala 2.10.0, Impala 2.11.0, Impala 2.12.0
>Reporter: Mostafa Mokhtar
>Priority: Critical
> Labels: catalog-server, perfomance
>
> Metadata operations that mutate the state of a table like "compute stats foo"
> or "alter recover partitions" block topic updates for read only operations
> against unrelated tables as "describe bar".
> Thread for blocked operation
> {code}
> "Thread-7" prio=10 tid=0x11613000 nid=0x21b3b waiting on condition
> [0x7f5f2ef52000]
>java.lang.Thread.State: WAITING (parking)
> at sun.misc.Unsafe.park(Native Method)
> - parking to wait for <0x7f6f57ff0240> (a
> java.util.concurrent.locks.ReentrantLock$NonfairSync)
> at
[jira] [Updated] (IMPALA-7347) Assertion Failure - test_show_create_table
[
https://issues.apache.org/jira/browse/IMPALA-7347?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
David Knupp updated IMPALA-7347:
Affects Version/s: (was: Impala 3.0)
Impala 3.1.0
> Assertion Failure - test_show_create_table
> ---
>
> Key: IMPALA-7347
> URL: https://issues.apache.org/jira/browse/IMPALA-7347
> Project: IMPALA
> Issue Type: Bug
>Affects Versions: Impala 3.1.0
>Reporter: nithya
>Assignee: Tianyi Wang
>Priority: Blocker
> Labels: build-failure
>
> test_show_create_table in metadata/test_show_create_table.py is failing with
> the following assertion error
> {noformat}
> metadata/test_show_create_table.py:58: in test_show_create_table
> unique_database)
> metadata/test_show_create_table.py:106: in __run_show_create_table_test_case
> self.__compare_result(expected_result, create_table_result)
> metadata/test_show_create_table.py:134: in __compare_result
> assert expected_tbl_props == actual_tbl_props
> E assert {} == {'numFilesErasureCoded': '0'}
> E Right contains more items:
> E {'numFilesErasureCoded': '0'}
> E Use -v to get the full diff
> {noformat}
>
> It appears that table property "numFilesErasureCoded" is showing up in table
> properties.
> Either the test needs updating or a bug.
>
> {code}
> h3. Error Message
> metadata/test_show_create_table.py:58: in test_show_create_table
> unique_database) metadata/test_show_create_table.py:106: in
> __run_show_create_table_test_case self.__compare_result(expected_result,
> create_table_result) metadata/test_show_create_table.py:134: in
> __compare_result assert expected_tbl_props == actual_tbl_props E assert {} ==
> \{'numFilesErasureCoded': '0'} E Right contains more items: E
> \{'numFilesErasureCoded': '0'} E Use -v to get the full diff
> {code}
>
> ---
> {code}
> h3. Standard Error
> -- connecting to: localhost:21000 SET sync_ddl=False; -- executing against
> localhost:21000 DROP DATABASE IF EXISTS `test_show_create_table_f1598d0b`
> CASCADE; SET sync_ddl=False; -- executing against localhost:21000 CREATE
> DATABASE `test_show_create_table_f1598d0b`; MainThread: Created database
> "test_show_create_table_f1598d0b" for test ID
> "metadata/test_show_create_table.py::TestShowCreateTable::()::test_show_create_table[table_format:
> text/none]" -- executing against localhost:21000 CREATE TABLE
> test_show_create_table_f1598d0b.test1 ( id INT ) STORED AS TEXTFILE; --
> executing against localhost:21000 show create table
> test_show_create_table_f1598d0b.test1; -- executing against localhost:21000
> drop table test_show_create_table_f1598d0b.test1; -- executing against
> localhost:21000 CREATE TABLE test_show_create_table_f1598d0b.test1 (id INT)
> STORED AS TEXTFILE LOCATION
> 'hdfs://localhost:20500/test-warehouse/test_show_create_table_f1598d0b.db/test1';
> -- executing against localhost:21000 show create table
> test_show_create_table_f1598d0b.test1; -- executing against localhost:21000
> drop table test_show_create_table_f1598d0b.test1; -- executing against
> localhost:21000 CREATE TABLE test_show_create_table_f1598d0b.test2 ( year
> INT, month INT, id INT COMMENT 'Add a comment', bool_col BOOLEAN, tinyint_col
> TINYINT, smallint_col SMALLINT, int_col INT, bigint_col BIGINT, float_col
> FLOAT, double_col DOUBLE, date_string_col STRING, string_col STRING,
> timestamp_col TIMESTAMP ) STORED AS TEXTFILE; -- executing against
> localhost:21000 show create table test_show_create_table_f1598d0b.test2; --
> executing against localhost:21000 drop table
> test_show_create_table_f1598d0b.test2; -- executing against localhost:21000
> CREATE TABLE test_show_create_table_f1598d0b.test2 (year INT, month INT, id
> INT COMMENT 'Add a comment', bool_col BOOLEAN, tinyint_col TINYINT,
> smallint_col SMALLINT, int_col INT, bigint_col BIGINT, float_col FLOAT,
> double_col DOUBLE, date_string_col STRING, string_col STRING, timestamp_col
> TIMESTAMP) STORED AS TEXTFILE LOCATION
> 'hdfs://localhost:20500/test-warehouse/test_show_create_table_f1598d0b.db/test2';
> -- executing against localhost:21000 show create table
> test_show_create_table_f1598d0b.test2; -- executing against localhost:21000
> drop table test_show_create_table_f1598d0b.test2; -- executing against
> localhost:21000 CREATE TABLE test_show_create_table_f1598d0b.test3 ( year
> INT, month INT, id INT COMMENT 'Add a comment', bool_col BOOLEAN, tinyint_col
> TINYINT, smallint_col SMALLINT, int_col INT, bigint_col BIGINT, float_col
> FLOAT, double_col DOUBLE, date_string_col STRING, string_col STRING,
> timestamp_col TIMESTAMP ) PARTITIONED BY ( x INT, y INT, a BOOLEAN ) COMMENT
> 'This is a test' STORED AS TEXTFILE; -- executing against localhost:21000
> show create table
[jira] [Updated] (IMPALA-7347) Assertion Failure - test_show_create_table
[
https://issues.apache.org/jira/browse/IMPALA-7347?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
David Knupp updated IMPALA-7347:
Issue Type: Bug (was: Test)
> Assertion Failure - test_show_create_table
> ---
>
> Key: IMPALA-7347
> URL: https://issues.apache.org/jira/browse/IMPALA-7347
> Project: IMPALA
> Issue Type: Bug
>Affects Versions: Impala 3.0
>Reporter: nithya
>Assignee: Tianyi Wang
>Priority: Blocker
> Labels: build-failure
>
> test_show_create_table in metadata/test_show_create_table.py is failing with
> the following assertion error
> {noformat}
> metadata/test_show_create_table.py:58: in test_show_create_table
> unique_database)
> metadata/test_show_create_table.py:106: in __run_show_create_table_test_case
> self.__compare_result(expected_result, create_table_result)
> metadata/test_show_create_table.py:134: in __compare_result
> assert expected_tbl_props == actual_tbl_props
> E assert {} == {'numFilesErasureCoded': '0'}
> E Right contains more items:
> E {'numFilesErasureCoded': '0'}
> E Use -v to get the full diff
> {noformat}
>
> It appears that table property "numFilesErasureCoded" is showing up in table
> properties.
> Either the test needs updating or a bug.
>
> {code}
> h3. Error Message
> metadata/test_show_create_table.py:58: in test_show_create_table
> unique_database) metadata/test_show_create_table.py:106: in
> __run_show_create_table_test_case self.__compare_result(expected_result,
> create_table_result) metadata/test_show_create_table.py:134: in
> __compare_result assert expected_tbl_props == actual_tbl_props E assert {} ==
> \{'numFilesErasureCoded': '0'} E Right contains more items: E
> \{'numFilesErasureCoded': '0'} E Use -v to get the full diff
> {code}
>
> ---
> {code}
> h3. Standard Error
> -- connecting to: localhost:21000 SET sync_ddl=False; -- executing against
> localhost:21000 DROP DATABASE IF EXISTS `test_show_create_table_f1598d0b`
> CASCADE; SET sync_ddl=False; -- executing against localhost:21000 CREATE
> DATABASE `test_show_create_table_f1598d0b`; MainThread: Created database
> "test_show_create_table_f1598d0b" for test ID
> "metadata/test_show_create_table.py::TestShowCreateTable::()::test_show_create_table[table_format:
> text/none]" -- executing against localhost:21000 CREATE TABLE
> test_show_create_table_f1598d0b.test1 ( id INT ) STORED AS TEXTFILE; --
> executing against localhost:21000 show create table
> test_show_create_table_f1598d0b.test1; -- executing against localhost:21000
> drop table test_show_create_table_f1598d0b.test1; -- executing against
> localhost:21000 CREATE TABLE test_show_create_table_f1598d0b.test1 (id INT)
> STORED AS TEXTFILE LOCATION
> 'hdfs://localhost:20500/test-warehouse/test_show_create_table_f1598d0b.db/test1';
> -- executing against localhost:21000 show create table
> test_show_create_table_f1598d0b.test1; -- executing against localhost:21000
> drop table test_show_create_table_f1598d0b.test1; -- executing against
> localhost:21000 CREATE TABLE test_show_create_table_f1598d0b.test2 ( year
> INT, month INT, id INT COMMENT 'Add a comment', bool_col BOOLEAN, tinyint_col
> TINYINT, smallint_col SMALLINT, int_col INT, bigint_col BIGINT, float_col
> FLOAT, double_col DOUBLE, date_string_col STRING, string_col STRING,
> timestamp_col TIMESTAMP ) STORED AS TEXTFILE; -- executing against
> localhost:21000 show create table test_show_create_table_f1598d0b.test2; --
> executing against localhost:21000 drop table
> test_show_create_table_f1598d0b.test2; -- executing against localhost:21000
> CREATE TABLE test_show_create_table_f1598d0b.test2 (year INT, month INT, id
> INT COMMENT 'Add a comment', bool_col BOOLEAN, tinyint_col TINYINT,
> smallint_col SMALLINT, int_col INT, bigint_col BIGINT, float_col FLOAT,
> double_col DOUBLE, date_string_col STRING, string_col STRING, timestamp_col
> TIMESTAMP) STORED AS TEXTFILE LOCATION
> 'hdfs://localhost:20500/test-warehouse/test_show_create_table_f1598d0b.db/test2';
> -- executing against localhost:21000 show create table
> test_show_create_table_f1598d0b.test2; -- executing against localhost:21000
> drop table test_show_create_table_f1598d0b.test2; -- executing against
> localhost:21000 CREATE TABLE test_show_create_table_f1598d0b.test3 ( year
> INT, month INT, id INT COMMENT 'Add a comment', bool_col BOOLEAN, tinyint_col
> TINYINT, smallint_col SMALLINT, int_col INT, bigint_col BIGINT, float_col
> FLOAT, double_col DOUBLE, date_string_col STRING, string_col STRING,
> timestamp_col TIMESTAMP ) PARTITIONED BY ( x INT, y INT, a BOOLEAN ) COMMENT
> 'This is a test' STORED AS TEXTFILE; -- executing against localhost:21000
> show create table test_show_create_table_f1598d0b.test3; -- executing against
>
[jira] [Resolved] (IMPALA-7209) Disallow self referencing ALTER VIEW statments
[
https://issues.apache.org/jira/browse/IMPALA-7209?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Pooja Nilangekar resolved IMPALA-7209.
--
Resolution: Fixed
> Disallow self referencing ALTER VIEW statments
> --
>
> Key: IMPALA-7209
> URL: https://issues.apache.org/jira/browse/IMPALA-7209
> Project: IMPALA
> Issue Type: Bug
> Components: Frontend
>Reporter: Pooja Nilangekar
>Assignee: Pooja Nilangekar
>Priority: Major
>
> Currently, an ALTER VIEW statement accepts self referencing definitions.
> However, upon querying the altered view, the analyzer is unable to find the
> reference and hence throws a StackOverflowError: null error.
> The expected behavior would be to throw an AnalysisException while executing
> the alter view statement.
>
> Example:
>
> {code:java}
> [localhost:21000] default> create view foo as select * from
> functional.alltypes;
> Query: create view foo as select * from functional.alltypes
> Query submitted at: 2018-07-03 11:36:48 (Coordinator:
> http://pooja-OptiPlex-7040:25000)
> Query progress can be monitored at:
> http://pooja-OptiPlex-7040:25000/query_plan?query_id=614e03efbcb4d8b1:586a4bad
> ++
> | summary |
> ++
> | View has been created. |
> ++
> Fetched 1 row(s) in 0.26s
> [localhost:21000] default> alter view foo as select * from foo;
> Query: alter view foo as select * from foo
> ++
> | summary |
> ++
> | View has been altered. |
> ++
> Fetched 1 row(s) in 5.65s
> [localhost:21000] default> select * from foo;
> Query: select * from foo
> Query submitted at: 2018-07-03 11:37:12 (Coordinator:
> http://pooja-OptiPlex-7040:25000)
> ERROR: StackOverflowError: null
> {code}
>
> The select statement on the view fails because the analyzer can't resolve its
> reference. Other databases return failure during the alter view statement
> because stating.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Resolved] (IMPALA-7329) Blacklist CDH Maven snapshots repository
[ https://issues.apache.org/jira/browse/IMPALA-7329?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Fredy Wijaya resolved IMPALA-7329. -- Resolution: Fixed Fix Version/s: Impala 3.1.0 > Blacklist CDH Maven snapshots repository > > > Key: IMPALA-7329 > URL: https://issues.apache.org/jira/browse/IMPALA-7329 > Project: IMPALA > Issue Type: Bug > Components: Infrastructure >Affects Versions: Impala 3.0 >Reporter: Fredy Wijaya >Assignee: Fredy Wijaya >Priority: Major > Fix For: Impala 3.1.0 > > > The Impala development environment bootstrapping depends on CDH Maven > snapshots which transitively pulls the dependencies from other repositories > causing the build to be non-reproducible, such as the issue in > https://issues.apache.org/jira/browse/IMPALA-7316. In order to fix this > issue, we need to blacklist a certain repository so that Impala does not > accidentally downloads the latest snapshot for its dependencies. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Updated] (IMPALA-7368) Add initial support for DATE type
[ https://issues.apache.org/jira/browse/IMPALA-7368?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Attila Jeges updated IMPALA-7368: - Description: DATE values describe a particular year/month/day, in the form -MM-DD. For example, DATE '2013-01-01'. Date types do not have a time of day component. The range of values supported for the Date type is -01-01 to -12-31. The initial DATE type support should incluide the following changes: - new internal type - text scanner/writer - casting between DATE and other types - codegen infrastructure for expression evaluation - "IS [NOT] NULL" and "[NOT] IN" predicates - common comparison operators - conditional functions - infrastructure changes for builtin scalar functions. - support partitioning. These items are tightly coupled and it makes sense to implement them in one change-set. was: DATE values describe a particular year/month/day, in the form -MM-DD. For example, DATE '2013-01-01'. Date types do not have a time of day component. The range of values supported for the Date type is -01-01 to -12-31. The initial DATE type support should incluide the following changes: - new internal type - text scanner/writer - casting between DATE and other types - codegen infrastructure for expression evaluation - "IS [NOT] NULL" and "[NOT] IN" predicates - common comparison operators - conditional functions - infrastructure changes for builtin scalar functions. These items are tightly coupled and it makes sense to implement them in one change-set. > Add initial support for DATE type > - > > Key: IMPALA-7368 > URL: https://issues.apache.org/jira/browse/IMPALA-7368 > Project: IMPALA > Issue Type: Sub-task >Reporter: Attila Jeges >Assignee: Attila Jeges >Priority: Major > > DATE values describe a particular year/month/day, in the form -MM-DD. > For example, DATE '2013-01-01'. Date types do not have a time of day > component. The range of values supported for the Date type is -01-01 to > -12-31. > The initial DATE type support should incluide the following changes: > - new internal type > - text scanner/writer > - casting between DATE and other types > - codegen infrastructure for expression evaluation > - "IS [NOT] NULL" and "[NOT] IN" predicates > - common comparison operators > - conditional functions > - infrastructure changes for builtin scalar functions. > - support partitioning. > These items are tightly coupled and it makes sense to implement them in one > change-set. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Updated] (IMPALA-7368) Add initial support for DATE type
[ https://issues.apache.org/jira/browse/IMPALA-7368?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Attila Jeges updated IMPALA-7368: - Description: DATE values describe a particular year/month/day, in the form -MM-DD. For example, DATE '2013-01-01'. Date types do not have a time of day component. The range of values supported for the Date type is -01-01 to -12-31. The initial DATE type support should incluide the following changes: - new internal type - text scanner/writer - casting between DATE and other types - codegen infrastructure for expression evaluation - "IS [NOT] NULL" and "[NOT] IN" predicates - common comparison operators - conditional functions - infrastructure changes for builtin scalar functions. These items are tightly coupled and it makes sense to implement them in one change-set. was: DATE values describe a particular year/month/day, in the form -MM-DD. For example, DATE '2013-01-01'. Date types do not have a time of day component. The range of values supported for the Date type is -01-01 to -12-31. The initial DATE type support should incluide the following changes: - new internal type - text scanner/writer - casting between DATE and other types - expression evaluation - "IS [NOT] NULL" and "[NOT] IN" predicates - common comparison operators - conditional functions - infrastructure changes for builtin scalar functions. These items are tightly coupled and it makes sense to implement them in one change-set. > Add initial support for DATE type > - > > Key: IMPALA-7368 > URL: https://issues.apache.org/jira/browse/IMPALA-7368 > Project: IMPALA > Issue Type: Sub-task >Reporter: Attila Jeges >Assignee: Attila Jeges >Priority: Major > > DATE values describe a particular year/month/day, in the form -MM-DD. > For example, DATE '2013-01-01'. Date types do not have a time of day > component. The range of values supported for the Date type is -01-01 to > -12-31. > The initial DATE type support should incluide the following changes: > - new internal type > - text scanner/writer > - casting between DATE and other types > - codegen infrastructure for expression evaluation > - "IS [NOT] NULL" and "[NOT] IN" predicates > - common comparison operators > - conditional functions > - infrastructure changes for builtin scalar functions. > These items are tightly coupled and it makes sense to implement them in one > change-set. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Commented] (IMPALA-7364) Upgrade RapidJson to the latest version
[ https://issues.apache.org/jira/browse/IMPALA-7364?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16563493#comment-16563493 ] Quanlong Huang commented on IMPALA-7364: [~tarmstr...@cloudera.com] Are there any docs about how to contribute to the native-toolchain project? Looks like it depends on the s3 account that's only managed by Cloudera. > Upgrade RapidJson to the latest version > --- > > Key: IMPALA-7364 > URL: https://issues.apache.org/jira/browse/IMPALA-7364 > Project: IMPALA > Issue Type: Improvement >Reporter: Quanlong Huang >Priority: Major > > Currently, we're using rapidjson-0.11. It's old enough and can't show > detailed parse errors. In the implementation of built-in JSON function in > IMPALA-376, we can't show warnings on how we fail to parse a json string and > where is the error location like this: > http://rapidjson.org/group___r_a_p_i_d_j_s_o_n___e_r_r_o_r_s.html#structrapidjson_1_1_parse_result > This JIRA aim to upgrade the version of RapidJson to v1.1. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Work started] (IMPALA-7368) Add initial support for DATE type
[ https://issues.apache.org/jira/browse/IMPALA-7368?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on IMPALA-7368 started by Attila Jeges. > Add initial support for DATE type > - > > Key: IMPALA-7368 > URL: https://issues.apache.org/jira/browse/IMPALA-7368 > Project: IMPALA > Issue Type: Sub-task >Reporter: Attila Jeges >Assignee: Attila Jeges >Priority: Major > > DATE values describe a particular year/month/day, in the form -MM-DD. > For example, DATE '2013-01-01'. Date types do not have a time of day > component. The range of values supported for the Date type is -01-01 to > -12-31. > The initial DATE type support should incluide the following changes: > - new internal type > - text scanner/writer > - casting between DATE and other types > - expression evaluation > - "IS [NOT] NULL" and "[NOT] IN" predicates > - common comparison operators > - conditional functions > - infrastructure changes for builtin scalar functions. > These items are tightly coupled and it makes sense to implement them in one > change-set. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Work started] (IMPALA-6169) Implement DATE type
[ https://issues.apache.org/jira/browse/IMPALA-6169?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on IMPALA-6169 started by Attila Jeges. > Implement DATE type > --- > > Key: IMPALA-6169 > URL: https://issues.apache.org/jira/browse/IMPALA-6169 > Project: IMPALA > Issue Type: New Feature > Components: Backend >Reporter: Quanlong Huang >Assignee: Attila Jeges >Priority: Major > > In Hive, the Date type describes a particular year/month/day, in the form > -MM-DD. > Hive has supported Date type in Parquet two years ago in Hive-1.2.0. (See > https://issues.apache.org/jira/browse/HIVE-8119 and > https://cwiki.apache.org/confluence/display/Hive/Parquet#Parquet-VersionsandLimitations.) > We should add support for Date type too. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Created] (IMPALA-7370) Add DATE type support to Parquet scanner/writer
Attila Jeges created IMPALA-7370: Summary: Add DATE type support to Parquet scanner/writer Key: IMPALA-7370 URL: https://issues.apache.org/jira/browse/IMPALA-7370 Project: IMPALA Issue Type: Sub-task Reporter: Attila Jeges Assignee: Attila Jeges Implement Parquet DATE type support. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Created] (IMPALA-7369) Implement DATE builtin functions
Attila Jeges created IMPALA-7369: Summary: Implement DATE builtin functions Key: IMPALA-7369 URL: https://issues.apache.org/jira/browse/IMPALA-7369 Project: IMPALA Issue Type: Sub-task Reporter: Attila Jeges Assignee: Attila Jeges - Built-in functions supported in Hive should be implemented in Impala es well. - Already implemented TIMESTAMP built-in functions that work on the date part of timestamps should be implemented for DATE types too. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Updated] (IMPALA-7368) Add initial support for DATE type
[ https://issues.apache.org/jira/browse/IMPALA-7368?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Attila Jeges updated IMPALA-7368: - Description: DATE values describe a particular year/month/day, in the form -MM-DD. For example, DATE '2013-01-01'. Date types do not have a time of day component. The range of values supported for the Date type is -01-01 to -12-31. The initial DATE type support should incluide the following changes: - new internal type - text scanner/writer - casting between DATE and other types - expression evaluation - "IS [NOT] NULL" and "[NOT] IN" predicates - common comparison operators - conditional functions - infrastructure changes for builtin scalar functions. These items are tightly coupled and it makes sense to implement them in one change-set. was: DATE values describe a particular year/month/day, in the form -MM-DD. For example, DATE '2013-01-01'. Date types do not have a time of day component. The range of values supported for the Date type is -01-01 to -12-31. The initial DATE type support should incluide the following changes: - new internal type - text scanner/writer - casting between DATE and other types - expression evaluation - "IS [NOT] NULL" and "[NOT] IN" predicates - common comparison operators - conditiioal functions - infrastructure changes for builtin scalar functions. These items are tightly coupled and it makes sense to implement them in one change-set. > Add initial support for DATE type > - > > Key: IMPALA-7368 > URL: https://issues.apache.org/jira/browse/IMPALA-7368 > Project: IMPALA > Issue Type: Sub-task >Reporter: Attila Jeges >Assignee: Attila Jeges >Priority: Major > > DATE values describe a particular year/month/day, in the form -MM-DD. > For example, DATE '2013-01-01'. Date types do not have a time of day > component. The range of values supported for the Date type is -01-01 to > -12-31. > The initial DATE type support should incluide the following changes: > - new internal type > - text scanner/writer > - casting between DATE and other types > - expression evaluation > - "IS [NOT] NULL" and "[NOT] IN" predicates > - common comparison operators > - conditional functions > - infrastructure changes for builtin scalar functions. > These items are tightly coupled and it makes sense to implement them in one > change-set. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Updated] (IMPALA-7368) Add initial support for DATE type
[ https://issues.apache.org/jira/browse/IMPALA-7368?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Attila Jeges updated IMPALA-7368: - Description: DATE values describe a particular year/month/day, in the form -MM-DD. For example, DATE '2013-01-01'. Date types do not have a time of day component. The range of values supported for the Date type is -01-01 to -12-31. The initial DATE type support should incluide the following changes: - new internal type - text scanner/writer - casting between DATE and other types - expression evaluation - "IS [NOT] NULL" and "[NOT] IN" predicates - common comparison operators - conditiioal functions - infrastructure changes for builtin scalar functions. These items are tightly coupled and it makes sense to implement them in one change-set. was: DATE values describe a particular year/month/day, in the form -MM-DD. For example, DATE '2013-01-01'. Date types do not have a time of day component. The range of values supported for the Date type is -01-01 to -12-31. The initial DATE type support should incluide the following changes: - text scanner/writer - casting between DATE and other types - expression evaluation - "IS [NOT] NULL" and "[NOT] IN" predicates - common comparison operators - conditiioal functions - infrastructure changes for builtin scalar functions. These items are tightly coupled and it makes sense to implement them in one change-set. > Add initial support for DATE type > - > > Key: IMPALA-7368 > URL: https://issues.apache.org/jira/browse/IMPALA-7368 > Project: IMPALA > Issue Type: Sub-task >Reporter: Attila Jeges >Assignee: Attila Jeges >Priority: Major > > DATE values describe a particular year/month/day, in the form -MM-DD. > For example, DATE '2013-01-01'. Date types do not have a time of day > component. The range of values supported for the Date type is -01-01 to > -12-31. > The initial DATE type support should incluide the following changes: > - new internal type > - text scanner/writer > - casting between DATE and other types > - expression evaluation > - "IS [NOT] NULL" and "[NOT] IN" predicates > - common comparison operators > - conditiioal functions > - infrastructure changes for builtin scalar functions. > These items are tightly coupled and it makes sense to implement them in one > change-set. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Created] (IMPALA-7368) Add initial support for DATE type
Attila Jeges created IMPALA-7368: Summary: Add initial support for DATE type Key: IMPALA-7368 URL: https://issues.apache.org/jira/browse/IMPALA-7368 Project: IMPALA Issue Type: Sub-task Reporter: Attila Jeges Assignee: Attila Jeges DATE values describe a particular year/month/day, in the form -MM-DD. For example, DATE '2013-01-01'. Date types do not have a time of day component. The range of values supported for the Date type is -01-01 to -12-31. The initial DATE type support should incluide the following changes: - text scanner/writer - casting between DATE and other types - expression evaluation - "IS [NOT] NULL" and "[NOT] IN" predicates - common comparison operators - conditiioal functions - infrastructure changes for builtin scalar functions. These items are tightly coupled and it makes sense to implement them in one change-set. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Resolved] (IMPALA-7333) Remove MarkNeedsDeepCopy from Aggregation and Hash Join Nodes
[ https://issues.apache.org/jira/browse/IMPALA-7333?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tim Armstrong resolved IMPALA-7333. --- Resolution: Fixed Fix Version/s: Impala 3.1.0 > Remove MarkNeedsDeepCopy from Aggregation and Hash Join Nodes > - > > Key: IMPALA-7333 > URL: https://issues.apache.org/jira/browse/IMPALA-7333 > Project: IMPALA > Issue Type: Sub-task > Components: Backend >Reporter: Tim Armstrong >Assignee: Tim Armstrong >Priority: Major > Labels: resource-management > Fix For: Impala 3.1.0 > > > The main part of this is fixing BufferedTupleStream. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Resolved] (IMPALA-7296) Soft limit for memory queue in scan node row batch queue
[ https://issues.apache.org/jira/browse/IMPALA-7296?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tim Armstrong resolved IMPALA-7296. --- Resolution: Fixed Fix Version/s: Impala 3.1.0 > Soft limit for memory queue in scan node row batch queue > > > Key: IMPALA-7296 > URL: https://issues.apache.org/jira/browse/IMPALA-7296 > Project: IMPALA > Issue Type: Improvement > Components: Backend >Reporter: Tim Armstrong >Assignee: Tim Armstrong >Priority: Major > Labels: resource-management > Fix For: Impala 3.1.0 > > > I'm splitting this out from IMPALA-7096. > It would be good to have some kind of soft limit for the amount of memory > that can be queued in the scan node's row batch queue. This would make it > easier to reason about the expected memory consumption of a scan. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Commented] (IMPALA-5844) Fix management of FunctionContext "local" allocations.
[ https://issues.apache.org/jira/browse/IMPALA-5844?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16563203#comment-16563203 ] ASF subversion and git services commented on IMPALA-5844: - Commit 240fde62d532c7166fc613a97b38c199cec09f1f in impala's branch refs/heads/master from [~tarmstr...@cloudera.com] [ https://git-wip-us.apache.org/repos/asf?p=impala.git;h=240fde6 ] IMPALA-7333: remove MarkNeedsDeepCopy() in agg and BTS This takes advantage of work (e.g. IMPALA-3200, IMPALA-5844) to remove a couple of uses of the API. Testing: Ran core, ASAN and exhaustive builds. Added unit tests to directly test the attaching behaviour. Change-Id: I91ac53bacc00df4726c015a30ba5a2026aa4b5f5 Reviewed-on: http://gerrit.cloudera.org:8080/11007 Reviewed-by: Tim Armstrong Tested-by: Impala Public Jenkins > Fix management of FunctionContext "local" allocations. > -- > > Key: IMPALA-5844 > URL: https://issues.apache.org/jira/browse/IMPALA-5844 > Project: IMPALA > Issue Type: Improvement > Components: Backend >Reporter: Tim Armstrong >Assignee: Tim Armstrong >Priority: Major > Fix For: Impala 2.11.0 > > > FunctionContexts associated with expressions make two kind of allocations > with very different lifecycles. One type of allocation is owned and managed > by the expression, while the other "local" allocation is implicitly > transferred to the Impala daemon after control flow returns from the > expression. Both are currently allocated from the same pool. > RowBatches returned from plan nodes may reference variable-length data stored > in local allocations so this memory should be attached to the RowBatches. > One approach here is: > * Separate local and other allocations to allocate from different MemPools. > * Manage local allocations in bulk by clearing, freeing, or transferring data > from that MemPool, similar to other memory that would be allocated from a > MemPool. > I think there are some potential wrinkles here and work to set up correct > MemPools for all places that use expressions but I think the high-level > approach should work. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Commented] (IMPALA-7209) Disallow self referencing ALTER VIEW statments
[
https://issues.apache.org/jira/browse/IMPALA-7209?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16563200#comment-16563200
]
ASF subversion and git services commented on IMPALA-7209:
-
Commit 9146f73a58c645cd861006cd6c90d9cbf08feeec in impala's branch
refs/heads/master from poojanilangekar
[ https://git-wip-us.apache.org/repos/asf?p=impala.git;h=9146f73 ]
IMPALA-7209: Disallow self referencing in ALTER VIEW statements
Previously, ALTER VIEW queries did not carry out reference checks
in the analysis phase. This allowed the DDL operation to succeed
but subsequent queries to the view threw StackOverflowError
because the catalog was unable to resolve the reference. With this
change, the AlterViewStmt checks for direct and in-direct self
references before altering a view.
Testing: Added tests to AnalyzeDDLTest to verify it.
Change-Id: I17c231c9d74d9d411463a408b086eb874090b9b7
Reviewed-on: http://gerrit.cloudera.org:8080/10908
Reviewed-by: Impala Public Jenkins
Tested-by: Impala Public Jenkins
> Disallow self referencing ALTER VIEW statments
> --
>
> Key: IMPALA-7209
> URL: https://issues.apache.org/jira/browse/IMPALA-7209
> Project: IMPALA
> Issue Type: Bug
> Components: Frontend
>Reporter: Pooja Nilangekar
>Assignee: Pooja Nilangekar
>Priority: Major
>
> Currently, an ALTER VIEW statement accepts self referencing definitions.
> However, upon querying the altered view, the analyzer is unable to find the
> reference and hence throws a StackOverflowError: null error.
> The expected behavior would be to throw an AnalysisException while executing
> the alter view statement.
>
> Example:
>
> {code:java}
> [localhost:21000] default> create view foo as select * from
> functional.alltypes;
> Query: create view foo as select * from functional.alltypes
> Query submitted at: 2018-07-03 11:36:48 (Coordinator:
> http://pooja-OptiPlex-7040:25000)
> Query progress can be monitored at:
> http://pooja-OptiPlex-7040:25000/query_plan?query_id=614e03efbcb4d8b1:586a4bad
> ++
> | summary |
> ++
> | View has been created. |
> ++
> Fetched 1 row(s) in 0.26s
> [localhost:21000] default> alter view foo as select * from foo;
> Query: alter view foo as select * from foo
> ++
> | summary |
> ++
> | View has been altered. |
> ++
> Fetched 1 row(s) in 5.65s
> [localhost:21000] default> select * from foo;
> Query: select * from foo
> Query submitted at: 2018-07-03 11:37:12 (Coordinator:
> http://pooja-OptiPlex-7040:25000)
> ERROR: StackOverflowError: null
> {code}
>
> The select statement on the view fails because the analyzer can't resolve its
> reference. Other databases return failure during the alter view statement
> because stating.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Commented] (IMPALA-3200) Replace BufferedBlockMgr with new buffer pool
[ https://issues.apache.org/jira/browse/IMPALA-3200?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16563202#comment-16563202 ] ASF subversion and git services commented on IMPALA-3200: - Commit 240fde62d532c7166fc613a97b38c199cec09f1f in impala's branch refs/heads/master from [~tarmstr...@cloudera.com] [ https://git-wip-us.apache.org/repos/asf?p=impala.git;h=240fde6 ] IMPALA-7333: remove MarkNeedsDeepCopy() in agg and BTS This takes advantage of work (e.g. IMPALA-3200, IMPALA-5844) to remove a couple of uses of the API. Testing: Ran core, ASAN and exhaustive builds. Added unit tests to directly test the attaching behaviour. Change-Id: I91ac53bacc00df4726c015a30ba5a2026aa4b5f5 Reviewed-on: http://gerrit.cloudera.org:8080/11007 Reviewed-by: Tim Armstrong Tested-by: Impala Public Jenkins > Replace BufferedBlockMgr with new buffer pool > - > > Key: IMPALA-3200 > URL: https://issues.apache.org/jira/browse/IMPALA-3200 > Project: IMPALA > Issue Type: Improvement > Components: Backend >Affects Versions: Impala 2.6.0 >Reporter: Tim Armstrong >Assignee: Tim Armstrong >Priority: Blocker > Labels: resource-management > Fix For: Impala 2.10.0 > > > We want to replace BufferedBlockMgr, a query-wide buffer pool, with a new > BufferPool that is shared between all queries. The goals are: > * Support for guaranteed reservations: i.e. if a reservation is granted, the > buffer pool will fulfil it (unless the OS is unable to fulfill the buffer > pool's memory requirements). > * Simplified interaction between reservations and pins (if you have a > reservation, you can pin, if you don't, you can't) > * Support for increasing reservations (up to a planner-specified limit). > * Support for smaller buffer sizes with similar performance (so we can reduce > minimum memory requirement to execute spill-to-disk algorithm) > * Support for larger buffers to support wide rows > * Reduced reliance on TCMalloc, which isn't suited to management of large > buffers (e.g. see IMPALA-2800) > * Better transfer model for buffer pool pages, so we can implement > transfer-of-ownership consistently for row batches (instead of mixing > transfer with MarkNeedsToReturn()). -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Commented] (IMPALA-7317) Instant validation job that runs on Gerrit upload (clang tidy, other simple checks)
[ https://issues.apache.org/jira/browse/IMPALA-7317?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16563204#comment-16563204 ] ASF subversion and git services commented on IMPALA-7317: - Commit 47b4606742287361b8c3751b4db77ab5b3508783 in impala's branch refs/heads/master from [~tarmstr...@cloudera.com] [ https://git-wip-us.apache.org/repos/asf?p=impala.git;h=47b4606 ] IMPALA-7317: add scripts to post flake8 comments The script is used by a new jenkins job gerrit-auto-critic to post comments on code reviews. Testing: This patch deliberately contains some flake8 violations so that gerrit-auto-critic will flag them. Change-Id: I7d348ea4944f829a407bd7b2939654f272736170 Reviewed-on: http://gerrit.cloudera.org:8080/11054 Reviewed-by: Impala Public Jenkins Reviewed-by: Michael Brown Tested-by: Impala Public Jenkins > Instant validation job that runs on Gerrit upload (clang tidy, other simple > checks) > --- > > Key: IMPALA-7317 > URL: https://issues.apache.org/jira/browse/IMPALA-7317 > Project: IMPALA > Issue Type: Task >Reporter: Tim Armstrong >Assignee: Tim Armstrong >Priority: Major > > Add a new job to jenkins.impala.io that runs clang tidy upon upload to > Gerrit. The goal is to prefetch any obvious failures that will prevent the > code being merged. > Existing checks that are reasonably cheap: > * clang-tidy-ub1604 > * ubuntu-16.04-build-only > * rat-check-ub1604 > Look into other checks to add (flake8, lines too long, etc) -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Commented] (IMPALA-7316) Fix broken build due to Hadoop JAR mismatch
[
https://issues.apache.org/jira/browse/IMPALA-7316?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16563208#comment-16563208
]
ASF subversion and git services commented on IMPALA-7316:
-
Commit 316b17ac55adb3d1deeb1289b4045688269b201d in impala's branch
refs/heads/master from [~fredyw]
[ https://git-wip-us.apache.org/repos/asf?p=impala.git;h=316b17a ]
IMPALA-7329: Blacklist CDH Maven snapshots repository
The Impala development bootstrapping depends on CDH Maven snapshots
which transitively pull dependencies from other repositories which
can cause the build to be non-reproducible, e.g. IMPALA-7316. This
patch makes the build to be reproducible by blacklisting
cdh.snapshots.repo so that Maven does not accidentally downloads the
latest CDH snapshots when running a build, which can cause
incompatibility issues.
Testing:
- Ran core tests with CDH_BUILD_NUMBER=422770 and did not hit the
issue described in IMPALA-7316
Change-Id: Id945bc2769f92f3df3bb4f617b00db77a6502ff3
Reviewed-on: http://gerrit.cloudera.org:8080/10999
Reviewed-by: Impala Public Jenkins
Tested-by: Impala Public Jenkins
> Fix broken build due to Hadoop JAR mismatch
> ---
>
> Key: IMPALA-7316
> URL: https://issues.apache.org/jira/browse/IMPALA-7316
> Project: IMPALA
> Issue Type: Bug
> Components: Infrastructure
>Affects Versions: Impala 3.0
>Reporter: Fredy Wijaya
>Assignee: Fredy Wijaya
>Priority: Blocker
> Labels: broken-build
> Fix For: Impala 3.1.0
>
>
> {noformat}
> I0718 10:03:13.658380 23734 jni-util.cc:230] java.lang.NoClassDefFoundError:
> org/apache/hadoop/fs/Options$ChecksumCombineMode
> at
> org.apache.hadoop.hdfs.client.impl.DfsClientConf.getChecksumCombineModeFromConf(DfsClientConf.java:314)
> at
> org.apache.hadoop.hdfs.client.impl.DfsClientConf.(DfsClientConf.java:184)
> at org.apache.hadoop.hdfs.DFSClient.(DFSClient.java:302)
> at org.apache.hadoop.hdfs.DFSClient.(DFSClient.java:286)
> at
> org.apache.hadoop.hdfs.DistributedFileSystem.initialize(DistributedFileSystem.java:167)
> at org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:3288)
> at org.apache.hadoop.fs.FileSystem.access$200(FileSystem.java:123)
> at org.apache.hadoop.fs.FileSystem$Cache.getInternal(FileSystem.java:3337)
> at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:3305)
> at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:476)
> at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:225)
> at
> org.apache.impala.service.JniFrontend.checkFileSystem(JniFrontend.java:778)
> at
> org.apache.impala.service.JniFrontend.checkConfiguration(JniFrontend.java:690)
> I0718 10:03:13.659090 23734 status.cc:125] NoClassDefFoundError:
> org/apache/hadoop/fs/Options$ChecksumCombineMode
> @ 0x1951b3a
> @ 0x1ff307f
> @ 0x1e9ec51
> @ 0x1eba8bb
> @ 0x1eb6df2
> @ 0x190d4bf
> @ 0x7fe6e95bf82f
> @ 0x190d338
> E0718 10:03:13.659101 23734 impala-server.cc:289] NoClassDefFoundError:
> org/apache/hadoop/fs/Options$ChecksumCombineMode
> E0718 10:03:13.659122 23734 impala-server.cc:292] Aborting Impala Server
> startup due to improper configuration. Impalad exiting.
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
