[jira] [Updated] (IMPALA-7027) Multiple Cast to Varchar with different limit fails with "AnalysisException: null CAUSED BY: IllegalArgumentException: "
[ https://issues.apache.org/jira/browse/IMPALA-7027?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alexander Behm updated IMPALA-7027: --- Description: If we have multiple cast of '' to varchar statements in a impala query which has a distinct like below, the query breaks for scenario when the cast to varchar limit in the SQL is lower than the previous cast. Query 1> Fails with " AnalysisException: null CAUSED BY: IllegalArgumentException: targetType=VARCHAR(100) type=VARCHAR(101)" SELECT DISTINCT CAST('' as VARCHAR(101)) as CL_COMMENTS,CAST('' as VARCHAR(100)) as CL_USER_ID FROM tablename limit 1 Where as the below query succeeds Query 2> Success SELECT DISTINCT CAST('' as VARCHAR(100)) as CL_COMMENTS,CAST('' as VARCHAR(101)) as CL_USER_ID FROM tablename limit 1 *Workaround* SET ENABLE_EXPR_REWRITES=false; was: If we have multiple cast of '' to varchar statements in a impala query which has a distinct like below, the query breaks for scenario when the cast to varchar limit in the SQL is lower than the previous cast. Query 1> Fails with " AnalysisException: null CAUSED BY: IllegalArgumentException: targetType=VARCHAR(100) type=VARCHAR(101)" SELECT DISTINCT CAST('' as VARCHAR(101)) as CL_COMMENTS,CAST('' as VARCHAR(100)) as CL_USER_ID FROM tablename limit 1 Where as the below query succeeds Query 2> Success SELECT DISTINCT CAST('' as VARCHAR(100)) as CL_COMMENTS,CAST('' as VARCHAR(101)) as CL_USER_ID FROM tablename limit 1 The Query succeeds is we remove the distinct clause or add set enable_expr_rewrites=false; > Multiple Cast to Varchar with different limit fails with "AnalysisException: > null CAUSED BY: IllegalArgumentException: " > > > Key: IMPALA-7027 > URL: https://issues.apache.org/jira/browse/IMPALA-7027 > Project: IMPALA > Issue Type: Bug > Components: Frontend >Affects Versions: Impala 3.0, Impala 2.12.0 >Reporter: Meenakshi >Priority: Critical > Labels: planner, regression > > If we have multiple cast of '' to varchar statements in a impala query which > has a distinct like below, the query breaks for scenario when the cast to > varchar limit in the SQL is lower than the previous cast. > > Query 1> Fails with " AnalysisException: null CAUSED BY: > IllegalArgumentException: targetType=VARCHAR(100) type=VARCHAR(101)" > SELECT DISTINCT CAST('' as VARCHAR(101)) as CL_COMMENTS,CAST('' as > VARCHAR(100)) as CL_USER_ID FROM tablename limit 1 > Where as the below query succeeds > Query 2> Success > SELECT DISTINCT CAST('' as VARCHAR(100)) as CL_COMMENTS,CAST('' as > VARCHAR(101)) as CL_USER_ID FROM tablename limit 1 > *Workaround* > SET ENABLE_EXPR_REWRITES=false; -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Updated] (IMPALA-7027) Multiple Cast to Varchar with different limit fails with "AnalysisException: null CAUSED BY: IllegalArgumentException: "
[ https://issues.apache.org/jira/browse/IMPALA-7027?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alexander Behm updated IMPALA-7027: --- Affects Version/s: (was: Impala 2.9.0) Impala 3.0 Impala 2.12.0 Labels: planner regression (was: ) > Multiple Cast to Varchar with different limit fails with "AnalysisException: > null CAUSED BY: IllegalArgumentException: " > > > Key: IMPALA-7027 > URL: https://issues.apache.org/jira/browse/IMPALA-7027 > Project: IMPALA > Issue Type: Bug > Components: Frontend >Affects Versions: Impala 3.0, Impala 2.12.0 >Reporter: Meenakshi >Priority: Critical > Labels: planner, regression > > If we have multiple cast of '' to varchar statements in a impala query which > has a distinct like below, the query breaks for scenario when the cast to > varchar limit in the SQL is lower than the previous cast. > > Query 1> Fails with " AnalysisException: null CAUSED BY: > IllegalArgumentException: targetType=VARCHAR(100) type=VARCHAR(101)" > SELECT DISTINCT CAST('' as VARCHAR(101)) as CL_COMMENTS,CAST('' as > VARCHAR(100)) as CL_USER_ID FROM tablename limit 1 > Where as the below query succeeds > Query 2> Success > SELECT DISTINCT CAST('' as VARCHAR(100)) as CL_COMMENTS,CAST('' as > VARCHAR(101)) as CL_USER_ID FROM tablename limit 1 > The Query succeeds is we remove the distinct clause or add set > enable_expr_rewrites=false; -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Closed] (IMPALA-3813) How to handle replication factor while creating KUDU table through impala
[ https://issues.apache.org/jira/browse/IMPALA-3813?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alex Rodoni closed IMPALA-3813. --- Resolution: Fixed Fix Version/s: Impala 3.1.0 Impala 2.13.0 > How to handle replication factor while creating KUDU table through impala > - > > Key: IMPALA-3813 > URL: https://issues.apache.org/jira/browse/IMPALA-3813 > Project: IMPALA > Issue Type: Bug > Components: Docs, Frontend >Affects Versions: Kudu_Impala > Environment: CDH 5.4.7 >Reporter: Ravi sharma >Assignee: Alex Rodoni >Priority: Major > Labels: performance > Fix For: Impala 2.13.0, Impala 3.1.0 > > > while to create kudu table from impala shell > facing below error > ERROR: > ImpalaRuntimeException: Error creating Kudu table > CAUSED BY: MasterErrorException: Server[Kudu Master - > cdhnode1.infocepts.com:7051] INVALID_ARGUMENT[code 4]: Not enough live tablet > servers to create a table with the requested replication factor 3. 2 tablet > servers are alive. > [cdhnode1.infocepts.com:21000] > > Please tell me how can i limit the replication factor manually, since i have > only 2 tablet servers. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Commented] (IMPALA-7025) PlannerTest.testTableSample has wrong mem-reservation
[ https://issues.apache.org/jira/browse/IMPALA-7025?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16475099#comment-16475099 ] Tim Armstrong commented on IMPALA-7025: --- It could be because a different set of files was selected or because the files were slightly different sizes. Need to investigate to determine which, although maybe the ultimate fix is to allow ignoring the resource estimates in these tests > PlannerTest.testTableSample has wrong mem-reservation > - > > Key: IMPALA-7025 > URL: https://issues.apache.org/jira/browse/IMPALA-7025 > Project: IMPALA > Issue Type: Bug > Components: Frontend >Affects Versions: Impala 3.1.0 >Reporter: Joe McDonnell >Assignee: Tim Armstrong >Priority: Critical > Labels: broken-build, flaky > > Seen once on master exhaustive: > {noformat} > Error Message > Section PLAN of query: > select id from functional_parquet.alltypes tablesample system(10) > repeatable(1234) > Actual does not match expected result: > F00:PLAN FRAGMENT [UNPARTITIONED] hosts=1 instances=1 > | Per-Host Resources: mem-estimate=16.00MB mem-reservation=8.00KB > thread-reservation=2 > ^^^ > PLAN-ROOT SINK > | mem-estimate=0B mem-reservation=0B thread-reservation=0 > | > 00:SCAN HDFS [functional_parquet.alltypes] >partitions=3/24 files=3 size=23.70KB >stored statistics: > table: rows=unavailable size=unavailable > partitions: 0/24 rows=unavailable > columns: unavailable >extrapolated-rows=disabled max-scan-range-rows=unavailable >mem-estimate=16.00MB mem-reservation=8.00KB thread-reservation=1 >tuple-ids=0 row-size=4B cardinality=unavailable > Expected: > F00:PLAN FRAGMENT [UNPARTITIONED] hosts=1 instances=1 > | Per-Host Resources: mem-estimate=16.00MB mem-reservation=16.00KB > thread-reservation=2 > PLAN-ROOT SINK > | mem-estimate=0B mem-reservation=0B thread-reservation=0 > | > 00:SCAN HDFS [functional_parquet.alltypes] >partitions=3/24 files=3 size=24.23KB >stored statistics: > table: rows=unavailable size=unavailable > partitions: 0/24 rows=unavailable > columns: unavailable >extrapolated-rows=disabled max-scan-range-rows=unavailable >mem-estimate=16.00MB mem-reservation=16.00KB thread-reservation=1 >tuple-ids=0 row-size=4B cardinality=unavailable{noformat} > This succeeded on the next build, so it is flaky and might not recur. > -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Resolved] (IMPALA-4464) Remove remote_data_load.py
[ https://issues.apache.org/jira/browse/IMPALA-4464?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Knupp resolved IMPALA-4464. - Resolution: Fixed > Remove remote_data_load.py > -- > > Key: IMPALA-4464 > URL: https://issues.apache.org/jira/browse/IMPALA-4464 > Project: IMPALA > Issue Type: Improvement > Components: Infrastructure >Affects Versions: Impala 2.8.0 >Reporter: David Knupp >Assignee: David Knupp >Priority: Major > Labels: remote_cluster_test > > A patch was recently submitted that allows data load and end-end tests to run > against a remote cluster. At its core was this file: > https://github.com/apache/incubator-impala/blob/master/bin/remote_data_load.py > However, while this script relies on several changes to existing build and > test scripts, nothing else in turns relies on it. In retrospect, it does not > make sense to have this script in the Impala repo if nothing can use it > externally. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Comment Edited] (IMPALA-7015) Insert into Kudu table returns with Status OK even if there are Kudu errors
[ https://issues.apache.org/jira/browse/IMPALA-7015?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16474673#comment-16474673 ] Thomas Tauber-Marshall edited comment on IMPALA-7015 at 5/14/18 7:42 PM: - So I agree with the point from IMPALA-3710 that we don't want these types of errors to prematurely terminate the query, which would happen if we just returned them immediately. -I think that it would be reasonable to just save the errors and return an error status once all of the rows have been sent to Kudu, eg in FlushFinal().- edit: realized the above is wrong - even if we return the error from FlushFinal(), it could still cause other fragment instances to be cancelled. In theory, it would be possible to achieve the behavior of "run query to completion but then return an error", but it would require more significant changes. Agreed that this whole error propagation issue is one of the biggest problems with Impala/Kudu integration at the moment, but not sure anyone has the time at the moment to fix it the right way. was (Author: twmarshall): So I agree with the point from IMPALA-3710 that we don't want these types of errors to prematurely terminate the query, which would happen if we just returned them immediately. I think that it would be reasonable to just save the errors and return an error status once all of the rows have been sent to Kudu, eg in FlushFinal(). We could then return an error message with the counts of rows that hit various errors. That wouldn't be as good as the more structured approach suggested in IMPALA-4416 and IMPALA-1789, since clients would have to parse the message, but it would at least be something, rather than just returning no info about what happened. My main hesitation is that this would be a somewhat breaking change, and if we're going to do that it might be better to wait until everything is in place to do it the right way, rather than doing breaking changes around this twice. > Insert into Kudu table returns with Status OK even if there are Kudu errors > --- > > Key: IMPALA-7015 > URL: https://issues.apache.org/jira/browse/IMPALA-7015 > Project: IMPALA > Issue Type: Bug > Components: Backend >Affects Versions: Impala 2.12.0 >Reporter: Mostafa Mokhtar >Priority: Major > Attachments: Insert into kudu profile with errors.txt > > > DML statements against Kudu tables return status OK even if there are Kudu > errors. > This behavior is misleading. > {code} > Summary: > Session ID: 18430b000e5dd8dc:e3e5dadb4a15d4b4 > Session Type: BEESWAX > Start Time: 2018-05-11 10:10:07.314218000 > End Time: 2018-05-11 10:10:07.434017000 > Query Type: DML > Query State: FINISHED > Query Status: OK > Impala Version: impalad version 2.12.0-cdh5.15.0 RELEASE (build > 2f9498d5c2f980aa7ff9505c56654c8e59e026ca) > User: mmokhtar > Connected User: mmokhtar > Delegated User: > Network Address: :::10.17.234.27:60760 > Default Db: tpcds_1000_kudu > Sql Statement: insert into store_2 select * from store > Coordinator: vd1317.foo:22000 > Query Options (set by configuration): > Query Options (set by configuration and planner): MT_DOP=0 > Plan: > {code} > {code} > Operator #Hosts Avg Time Max Time #Rows Est. #Rows Peak Mem > Est. Peak Mem Detail > - > 02:PARTIAL SORT5 909.030us 1.025ms 1.00K 1.00K 6.14 MB > 4.00 MB > 01:EXCHANGE56.262ms 7.232ms 1.00K 1.00K 75.50 KB > 0 KUDU(KuduPartition(tpcds_1000_kudu.store.s_store_sk)) > 00:SCAN KUDU 53.694ms 4.137ms 1.00K 1.00K 4.34 MB > 0 tpcds_1000_kudu.store > Errors: Key already present in Kudu table > 'impala::tpcds_1000_kudu.store_2'. (1 of 1002 similar) > {code} -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Commented] (IMPALA-7005) The 3.0 changelog page should mention the previous version
[ https://issues.apache.org/jira/browse/IMPALA-7005?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16474698#comment-16474698 ] Sailesh Mukil commented on IMPALA-7005: --- [~arodoni_cloudera] https://cwiki.apache.org/confluence/display/IMPALA/How+to+Release Step 15 in the above link should have what you need. > The 3.0 changelog page should mention the previous version > -- > > Key: IMPALA-7005 > URL: https://issues.apache.org/jira/browse/IMPALA-7005 > Project: IMPALA > Issue Type: Bug > Components: Docs >Affects Versions: Impala 3.0 >Reporter: Lars Volker >Assignee: Alex Rodoni >Priority: Major > Labels: website > > It's not clear to me from looking at the [3.0 > changlog|http://impala.apache.org/docs/changelog-3.0.html] what base version > it compares to. Can we add a sentence to point out that these changes are in > comparison to 2.11 (are they?)? -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Commented] (IMPALA-7028) TestKuduOperations.test_kudu_update fails due to wrong NumModifiedRows
[ https://issues.apache.org/jira/browse/IMPALA-7028?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16474677#comment-16474677 ] Joe McDonnell commented on IMPALA-7028: --- {noformat} query_test/test_kudu.py:92: in test_kudu_update self.run_test_case('QueryTest/kudu_update', vector, use_db=unique_database) common/impala_test_suite.py:451: in run_test_case verify_runtime_profile(test_section['RUNTIME_PROFILE'], result.runtime_profile) common/test_result_verifier.py:590: in verify_runtime_profile actual)) E AssertionError: Did not find matches for lines in runtime profile: E EXPECTED LINES: E NumModifiedRows: 7300 E E ACTUAL PROFILE: E Query (id=de4b7a0e99753cd5:6ffa44fd): E DEBUG MODE WARNING: Query profile created while running a DEBUG build of Impala. Use RELEASE builds to measure query performance. E Summary: E Session ID: 7f4a67a73c9ecf4e:5d51ba77392831b4 E Session Type: BEESWAX E Start Time: 2018-05-12 15:08:10.372271000 E End Time: E Query Type: DML E Query State: FINISHED E Query Status: OK E Impala Version: impalad version 3.1.0-SNAPSHOT DEBUG (build e12ee485cf4c77203b144c053ee167509cc39374) E User: jenkins E Connected User: jenkins E Delegated User: E Network Address: 127.0.0.1:55173 E Default Db: test_kudu_update_628738b5 E Sql Statement: update tdata set vali = -1 E Coordinator: ec2-m2-4xlarge-centos-6-4-092a.vpc.cloudera.com:22000 E Query Options (set by configuration): ABORT_ON_ERROR=1,DISABLE_CODEGEN=1,EXEC_SINGLE_NODE_ROWS_THRESHOLD=0,DISABLE_CODEGEN_ROWS_THRESHOLD=0 E Query Options (set by configuration and planner): ABORT_ON_ERROR=1,DISABLE_CODEGEN=1,EXEC_SINGLE_NODE_ROWS_THRESHOLD=0,MT_DOP=0,DISABLE_CODEGEN_ROWS_THRESHOLD=0 E Plan: E E Max Per-Host Resource Reservation: Memory=0B Threads=2 E Per-Host Resource Estimates: Memory=10.00MB E WARNING: The following tables are missing relevant table and/or column statistics. E test_kudu_update_628738b5.tdata E E F00:PLAN FRAGMENT [RANDOM] hosts=3 instances=3 E | Per-Host Resources: mem-estimate=0B mem-reservation=0B thread-reservation=2 E UPDATE KUDU [test_kudu_update_628738b5.tdata] E | mem-estimate=0B mem-reservation=0B thread-reservation=0 E | E 00:SCAN KUDU [test_kudu_update_628738b5.tdata] E mem-estimate=0B mem-reservation=0B thread-reservation=1 E tuple-ids=0 row-size=4B cardinality=unavailable E E Estimated Per-Host Mem: 10485760 E Tables Missing Stats: test_kudu_update_628738b5.tdata E Per Host Min Memory Reservation: ec2-m2-4xlarge-centos-6-4-092a.vpc.cloudera.com:22000(0) ec2-m2-4xlarge-centos-6-4-092a.vpc.cloudera.com:22001(0) ec2-m2-4xlarge-centos-6-4-092a.vpc.cloudera.com:22002(0) E Request Pool: default-pool E Admission result: Admitted immediately E Query Compilation: 6.606ms E - Metadata of all 1 tables cached: 1.087ms (1.087ms) E - Analysis finished: 1.457ms (369.234us) E - Value transfer graph computed: 1.508ms (51.577us) E - Single node plan created: 4.839ms (3.330ms) E - Runtime filters computed: 4.876ms (37.680us) E - Distributed plan created: 4.934ms (57.370us) E - Planning finished: 6.606ms (1.672ms) E Query Timeline: 84.999ms E - Query submitted: 999.994us (999.994us) E - Planning finished: 8.999ms (7.999ms) E - Submit for admission: 11.999ms (2.999ms) E - Completed admission: 11.999ms (0.000ns) E - Ready to start on 3 backends: 12.999ms (999.994us) E - All 3 execution backends (3 fragment instances) started: 24.999ms (11.999ms) E - Last row fetched: 39.999ms (14.999ms) E - Released admission control resources: 39.999ms (0.000ns) E - DML data written: 40.999ms (999.993us) E - DML Metastore update finished: 40.999ms (0.000ns) E - Request finished: 40.999ms (0.000ns) E - First row fetched: 82.999ms (41.999ms) E- ComputeScanRangeAssignmentTimer: 0.000ns E ImpalaServer: E- ClientFetchWaitTimer: 0.000ns E- MetastoreUpdateTimer: 0.000ns E- RowMaterializationTimer: 0.000ns E Execution Profile de4b7a0e99753cd5:6ffa44fd:(Total: 27.999ms, non-child: 0.000ns, % non-child: 0.00%) E Number of filters: 0 E Filter routing table: EID Src. Node Tgt. Node(s) Target type Partition filter Pending (Expected) First arrived Completed Enabled E --- E E Backend startup latencies: Count: 3, min / max: 1ms / 11ms, 25th %-ile: 1ms, 50th %-ile: 2ms, 75th %-ile: 2ms, 90th %-ile: 11ms, 95th %-ile: 11ms, 99.9th %-ile: 11ms E Per Node Peak
[jira] [Created] (IMPALA-7028) TestKuduOperations.test_kudu_update fails due to wrong NumModifiedRows
Joe McDonnell created IMPALA-7028: - Summary: TestKuduOperations.test_kudu_update fails due to wrong NumModifiedRows Key: IMPALA-7028 URL: https://issues.apache.org/jira/browse/IMPALA-7028 Project: IMPALA Issue Type: Bug Components: Backend Affects Versions: Impala 3.1.0 Reporter: Joe McDonnell Seen once on master exhaustive: {noformat} query_test/test_kudu.py:92: in test_kudu_update self.run_test_case('QueryTest/kudu_update', vector, use_db=unique_database) common/impala_test_suite.py:451: in run_test_case verify_runtime_profile(test_section['RUNTIME_PROFILE'], result.runtime_profile) common/test_result_verifier.py:590: in verify_runtime_profile actual)) E AssertionError: Did not find matches for lines in runtime profile: E EXPECTED LINES: E NumModifiedRows: 7300 ... # From the actual profile: E Partition: Default E NumModifiedRows: 32 E NumRowErrors: 0{noformat} Will add a comment with the profile. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Commented] (IMPALA-7025) PlannerTest.testTableSample has wrong mem-reservation
[ https://issues.apache.org/jira/browse/IMPALA-7025?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16474672#comment-16474672 ] Joe McDonnell commented on IMPALA-7025: --- [~tarmstrong] This is on the 3.x branch > PlannerTest.testTableSample has wrong mem-reservation > - > > Key: IMPALA-7025 > URL: https://issues.apache.org/jira/browse/IMPALA-7025 > Project: IMPALA > Issue Type: Bug > Components: Frontend >Affects Versions: Impala 3.1.0 >Reporter: Joe McDonnell >Assignee: Tim Armstrong >Priority: Critical > Labels: broken-build, flaky > > Seen once on master exhaustive: > {noformat} > Error Message > Section PLAN of query: > select id from functional_parquet.alltypes tablesample system(10) > repeatable(1234) > Actual does not match expected result: > F00:PLAN FRAGMENT [UNPARTITIONED] hosts=1 instances=1 > | Per-Host Resources: mem-estimate=16.00MB mem-reservation=8.00KB > thread-reservation=2 > ^^^ > PLAN-ROOT SINK > | mem-estimate=0B mem-reservation=0B thread-reservation=0 > | > 00:SCAN HDFS [functional_parquet.alltypes] >partitions=3/24 files=3 size=23.70KB >stored statistics: > table: rows=unavailable size=unavailable > partitions: 0/24 rows=unavailable > columns: unavailable >extrapolated-rows=disabled max-scan-range-rows=unavailable >mem-estimate=16.00MB mem-reservation=8.00KB thread-reservation=1 >tuple-ids=0 row-size=4B cardinality=unavailable > Expected: > F00:PLAN FRAGMENT [UNPARTITIONED] hosts=1 instances=1 > | Per-Host Resources: mem-estimate=16.00MB mem-reservation=16.00KB > thread-reservation=2 > PLAN-ROOT SINK > | mem-estimate=0B mem-reservation=0B thread-reservation=0 > | > 00:SCAN HDFS [functional_parquet.alltypes] >partitions=3/24 files=3 size=24.23KB >stored statistics: > table: rows=unavailable size=unavailable > partitions: 0/24 rows=unavailable > columns: unavailable >extrapolated-rows=disabled max-scan-range-rows=unavailable >mem-estimate=16.00MB mem-reservation=16.00KB thread-reservation=1 >tuple-ids=0 row-size=4B cardinality=unavailable{noformat} > This succeeded on the next build, so it is flaky and might not recur. > -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Updated] (IMPALA-7027) Multiple Cast to Varchar with different limit fails with "AnalysisException: null CAUSED BY: IllegalArgumentException: "
[ https://issues.apache.org/jira/browse/IMPALA-7027?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Meenakshi updated IMPALA-7027: -- Description: If we have multiple cast of '' to varchar statements in a impala query which has a distinct like below, the query breaks for scenario when the cast to varchar limit in the SQL is lower than the previous cast. Query 1> Fails with " AnalysisException: null CAUSED BY: IllegalArgumentException: targetType=VARCHAR(100) type=VARCHAR(101)" SELECT DISTINCT CAST('' as VARCHAR(101)) as CLAIM_COMMENTS,CAST('' as VARCHAR(100)) as CA_USER_ID FROM tablename limit 1 Where as the below query succeeds Query 2> Success SELECT DISTINCT CAST('' as VARCHAR(100)) as CLAIM_COMMENTS,CAST('' as VARCHAR(101)) as CA_USER_ID FROM tablename limit 1 The Query succeeds is we remove the distinct clause or add set enable_expr_rewrites=false; was: If we have multiple cast of '' to varchar statements in a impala query which has a distinct like below, the query breaks for scenario when the cast to varchar limit in the SQL is lower than the previous cast. Query 1> Fails with " AnalysisException: null CAUSED BY: IllegalArgumentException: targetType=VARCHAR(100) type=VARCHAR(101)" SELECT DISTINCT CAST('' as VARCHAR(101)) as CLAIM_COMMENTS,CAST('' as VARCHAR(100)) as CA_USER_ID FROM pinnga1ph_ds_prd_wh_gbd.concept_flat_dtl limit 1 Where as the below query succeeds Query 2> Success SELECT DISTINCT CAST('' as VARCHAR(100)) as CLAIM_COMMENTS,CAST('' as VARCHAR(101)) as CA_USER_ID FROM pinnga1ph_ds_prd_wh_gbd.concept_flat_dtl limit 1 The Query succeeds is we remove the distinct clause or add set enable_expr_rewrites=false; > Multiple Cast to Varchar with different limit fails with "AnalysisException: > null CAUSED BY: IllegalArgumentException: " > > > Key: IMPALA-7027 > URL: https://issues.apache.org/jira/browse/IMPALA-7027 > Project: IMPALA > Issue Type: Bug > Components: Frontend >Affects Versions: Impala 2.9.0 >Reporter: Meenakshi >Priority: Critical > > If we have multiple cast of '' to varchar statements in a impala query which > has a distinct like below, the query breaks for scenario when the cast to > varchar limit in the SQL is lower than the previous cast. > > Query 1> Fails with " AnalysisException: null CAUSED BY: > IllegalArgumentException: targetType=VARCHAR(100) type=VARCHAR(101)" > SELECT DISTINCT CAST('' as VARCHAR(101)) as CLAIM_COMMENTS,CAST('' as > VARCHAR(100)) as CA_USER_ID FROM tablename limit 1 > Where as the below query succeeds > Query 2> Success > SELECT DISTINCT CAST('' as VARCHAR(100)) as CLAIM_COMMENTS,CAST('' as > VARCHAR(101)) as CA_USER_ID FROM tablename limit 1 > The Query succeeds is we remove the distinct clause or add set > enable_expr_rewrites=false; -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Updated] (IMPALA-7027) Multiple Cast to Varchar with different limit fails with "AnalysisException: null CAUSED BY: IllegalArgumentException: "
[ https://issues.apache.org/jira/browse/IMPALA-7027?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Meenakshi updated IMPALA-7027: -- Description: If we have multiple cast of '' to varchar statements in a impala query which has a distinct like below, the query breaks for scenario when the cast to varchar limit in the SQL is lower than the previous cast. Query 1> Fails with " AnalysisException: null CAUSED BY: IllegalArgumentException: targetType=VARCHAR(100) type=VARCHAR(101)" SELECT DISTINCT CAST('' as VARCHAR(101)) as CL_COMMENTS,CAST('' as VARCHAR(100)) as CL_USER_ID FROM tablename limit 1 Where as the below query succeeds Query 2> Success SELECT DISTINCT CAST('' as VARCHAR(100)) as CL_COMMENTS,CAST('' as VARCHAR(101)) as CL_USER_ID FROM tablename limit 1 The Query succeeds is we remove the distinct clause or add set enable_expr_rewrites=false; was: If we have multiple cast of '' to varchar statements in a impala query which has a distinct like below, the query breaks for scenario when the cast to varchar limit in the SQL is lower than the previous cast. Query 1> Fails with " AnalysisException: null CAUSED BY: IllegalArgumentException: targetType=VARCHAR(100) type=VARCHAR(101)" SELECT DISTINCT CAST('' as VARCHAR(101)) as CLAIM_COMMENTS,CAST('' as VARCHAR(100)) as CA_USER_ID FROM tablename limit 1 Where as the below query succeeds Query 2> Success SELECT DISTINCT CAST('' as VARCHAR(100)) as CLAIM_COMMENTS,CAST('' as VARCHAR(101)) as CA_USER_ID FROM tablename limit 1 The Query succeeds is we remove the distinct clause or add set enable_expr_rewrites=false; > Multiple Cast to Varchar with different limit fails with "AnalysisException: > null CAUSED BY: IllegalArgumentException: " > > > Key: IMPALA-7027 > URL: https://issues.apache.org/jira/browse/IMPALA-7027 > Project: IMPALA > Issue Type: Bug > Components: Frontend >Affects Versions: Impala 2.9.0 >Reporter: Meenakshi >Priority: Critical > > If we have multiple cast of '' to varchar statements in a impala query which > has a distinct like below, the query breaks for scenario when the cast to > varchar limit in the SQL is lower than the previous cast. > > Query 1> Fails with " AnalysisException: null CAUSED BY: > IllegalArgumentException: targetType=VARCHAR(100) type=VARCHAR(101)" > SELECT DISTINCT CAST('' as VARCHAR(101)) as CL_COMMENTS,CAST('' as > VARCHAR(100)) as CL_USER_ID FROM tablename limit 1 > Where as the below query succeeds > Query 2> Success > SELECT DISTINCT CAST('' as VARCHAR(100)) as CL_COMMENTS,CAST('' as > VARCHAR(101)) as CL_USER_ID FROM tablename limit 1 > The Query succeeds is we remove the distinct clause or add set > enable_expr_rewrites=false; -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Commented] (IMPALA-4268) Allow PlanRootSink to buffer more than a batch of rows
[ https://issues.apache.org/jira/browse/IMPALA-4268?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16474669#comment-16474669 ] Tim Armstrong commented on IMPALA-4268: --- I think it would solve that as a side-effect. > Allow PlanRootSink to buffer more than a batch of rows > -- > > Key: IMPALA-4268 > URL: https://issues.apache.org/jira/browse/IMPALA-4268 > Project: IMPALA > Issue Type: Improvement > Components: Backend >Affects Versions: Impala 2.8.0 >Reporter: Henry Robinson >Priority: Major > Labels: resource-management > > In IMPALA-2905, we are introducing a {{PlanRootSink}} that handles the > production of output rows at the root of a plan. > The implementation in IMPALA-2905 has the plan execute in a separate thread > to the consumer, which calls {{GetNext()}} to retrieve the rows. However, the > sender thread will block until {{GetNext()}} is called, so that there are no > complications about memory usage and ownership due to having several batches > in flight at one time. > However, this also leads to many context switches, as each {{GetNext()}} call > yields to the sender to produce the rows. If the sender was to fill a buffer > asynchronously, the consumer could pull out of that buffer without taking a > context switch in many cases (and the extra buffering might smooth out any > performance spikes due to client delays, which currently directly affect plan > execution). > The tricky part is managing the mismatch between the size of the row batches > processed in {{Send()}} and the size of the fetch result asked for by the > client. The sender materializes output rows in a {{QueryResultSet}} that is > owned by the coordinator. That is not, currently, a splittable object - > instead it contains the actual RPC response struct that will hit the wire > when the RPC completes. As asynchronous sender cannot know the batch size, > which may change on every fetch call. So the {{GetNext()}} implementation > would need to be able to split out the {{QueryResultSet}} to match the > correct fetch size, and handle stitching together other {{QueryResultSets}} - > without doing extra copies. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Commented] (IMPALA-7025) PlannerTest.testTableSample has wrong mem-reservation
[ https://issues.apache.org/jira/browse/IMPALA-7025?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16474666#comment-16474666 ] Tim Armstrong commented on IMPALA-7025: --- Is this on a 2.x or 3.x branch? > PlannerTest.testTableSample has wrong mem-reservation > - > > Key: IMPALA-7025 > URL: https://issues.apache.org/jira/browse/IMPALA-7025 > Project: IMPALA > Issue Type: Bug > Components: Frontend >Affects Versions: Impala 3.1.0 >Reporter: Joe McDonnell >Assignee: Tim Armstrong >Priority: Critical > Labels: broken-build, flaky > > Seen once on master exhaustive: > {noformat} > Error Message > Section PLAN of query: > select id from functional_parquet.alltypes tablesample system(10) > repeatable(1234) > Actual does not match expected result: > F00:PLAN FRAGMENT [UNPARTITIONED] hosts=1 instances=1 > | Per-Host Resources: mem-estimate=16.00MB mem-reservation=8.00KB > thread-reservation=2 > ^^^ > PLAN-ROOT SINK > | mem-estimate=0B mem-reservation=0B thread-reservation=0 > | > 00:SCAN HDFS [functional_parquet.alltypes] >partitions=3/24 files=3 size=23.70KB >stored statistics: > table: rows=unavailable size=unavailable > partitions: 0/24 rows=unavailable > columns: unavailable >extrapolated-rows=disabled max-scan-range-rows=unavailable >mem-estimate=16.00MB mem-reservation=8.00KB thread-reservation=1 >tuple-ids=0 row-size=4B cardinality=unavailable > Expected: > F00:PLAN FRAGMENT [UNPARTITIONED] hosts=1 instances=1 > | Per-Host Resources: mem-estimate=16.00MB mem-reservation=16.00KB > thread-reservation=2 > PLAN-ROOT SINK > | mem-estimate=0B mem-reservation=0B thread-reservation=0 > | > 00:SCAN HDFS [functional_parquet.alltypes] >partitions=3/24 files=3 size=24.23KB >stored statistics: > table: rows=unavailable size=unavailable > partitions: 0/24 rows=unavailable > columns: unavailable >extrapolated-rows=disabled max-scan-range-rows=unavailable >mem-estimate=16.00MB mem-reservation=16.00KB thread-reservation=1 >tuple-ids=0 row-size=4B cardinality=unavailable{noformat} > This succeeded on the next build, so it is flaky and might not recur. > -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Commented] (IMPALA-7026) buildall.sh make shell fails due to sqlparse problem
[ https://issues.apache.org/jira/browse/IMPALA-7026?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16474667#comment-16474667 ] Fredy Wijaya commented on IMPALA-7026: -- Can you rebase the CR against master? > buildall.sh make shell fails due to sqlparse problem > > > Key: IMPALA-7026 > URL: https://issues.apache.org/jira/browse/IMPALA-7026 > Project: IMPALA > Issue Type: Bug > Components: Infrastructure >Affects Versions: Impala 3.1.0 >Reporter: Michael Brown >Assignee: Fredy Wijaya >Priority: Blocker > Labels: broken-build > Fix For: Impala 3.1.0 > > > https://jenkins.impala.io/job/ubuntu-16.04-from-scratch/2159/ > {noformat} > 22:43:05 Creating an egg for /home/ubuntu/Impala/shell/ext-py/sqlparse-0.1.14 > 22:43:05 [ 47%] Building CXX object > be/src/udf_samples/CMakeFiles/udf-sample-test.dir/udf-sample-test.cc.o > 22:43:05 [ 47%] Building CXX object > be/src/util/CMakeFiles/Util.dir/hdr-histogram.cc.o > 22:43:05 [ 47%] Built target uda-sample-test > 22:43:05 [ 47%] Building CXX object > be/src/util/CMakeFiles/Util.dir/impalad-metrics.cc.o > 22:43:05 [ 47%] Building CXX object > be/src/exprs/CMakeFiles/Exprs.dir/math-functions-ir.cc.o > 22:43:05 [ 47%] Building CXX object > be/src/testutil/CMakeFiles/TestUtil.dir/desc-tbl-builder.cc.o > 22:43:05 [ 47%] Building CXX object > be/src/testutil/CMakeFiles/TestUtil.dir/fault-injection-util.cc.o > 22:43:05 [ 47%] Linking CXX executable > ../../build/debug/udf_samples/udf-sample-test > 22:43:05 [ 47%] Built target udf-sample-test > 22:43:05 Scanning dependencies of target krpc > 22:43:05 [ 47%] Building CXX object > be/src/exprs/CMakeFiles/Exprs.dir/null-literal.cc.o > 22:43:05 [ 47%] Building CXX object > be/src/util/CMakeFiles/Util.dir/jni-util.cc.o > 22:43:05 [ 48%] Building CXX object > be/src/testutil/CMakeFiles/TestUtil.dir/impalad-query-executor.cc.o > 22:43:05 warning: no files found matching 'docs/Makefile' > 22:43:05 warning: no files found matching 'Makefile' > 22:43:05 [ 48%] Building CXX object > be/src/runtime/CMakeFiles/Runtime.dir/hbase-table.cc.o > 22:43:05 [ 50%] Building CXX object > be/src/kudu/rpc/CMakeFiles/krpc.dir/acceptor_pool.cc.o > 22:43:05 [ 50%] Building CXX object > be/src/testutil/CMakeFiles/TestUtil.dir/in-process-servers.cc.o > 22:43:05 [ 50%] Building CXX object > be/src/kudu/rpc/CMakeFiles/krpc.dir/blocking_ops.cc.o > 22:43:05 [ 50%] Building CXX object > be/src/kudu/rpc/CMakeFiles/krpc.dir/client_negotiation.cc.o > 22:43:05 [ 50%] Building CXX object > be/src/runtime/CMakeFiles/Runtime.dir/hbase-table-factory.cc.o > 22:43:05 [ 50%] Building CXX object > be/src/util/CMakeFiles/Util.dir/logging-support.cc.o > 22:43:05 [ 50%] Building CXX object > be/src/util/CMakeFiles/Util.dir/mem-info.cc.o > 22:43:05 [ 50%] Building CXX object > be/src/util/CMakeFiles/Util.dir/memory-metrics.cc.o > 22:43:05 zip_safe flag not set; analyzing archive contents... > 22:43:05 Cleaning up old build artifacts. > 22:43:06 Creating an egg for /home/ubuntu/Impala/shell/ext-py/sqlparse-0.1.19 > 22:43:06 [ 51%] Building CXX object > be/src/runtime/CMakeFiles/Runtime.dir/hdfs-fs-cache.cc.o > 22:43:06 [ 51%] Building CXX object > be/src/runtime/CMakeFiles/Runtime.dir/initial-reservations.cc.o > 22:43:06 python: can't open file 'setup.py': [Errno 2] No such file or > directory > 22:43:06 Error in shell/make_shell_tarball.sh at line 96: python setup.py -q > bdist_egg clean > 22:43:06 CMakeFiles/shell_tarball.dir/build.make:57: recipe for target > 'CMakeFiles/shell_tarball' failed > 22:43:06 make[2]: *** [CMakeFiles/shell_tarball] Error 2 > 22:43:06 CMakeFiles/Makefile2:141: recipe for target > 'CMakeFiles/shell_tarball.dir/all' failed > 22:43:06 make[1]: *** [CMakeFiles/shell_tarball.dir/all] Error 2 > 22:43:06 make[1]: *** Waiting for unfinished jobs > {noformat} > I just reproduced this locally using {{./buildall.sh -notests > -start_minicluster -start_impala_cluster}}. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Assigned] (IMPALA-7025) PlannerTest.testTableSample has wrong mem-reservation
[ https://issues.apache.org/jira/browse/IMPALA-7025?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tim Armstrong reassigned IMPALA-7025: - Assignee: Tim Armstrong > PlannerTest.testTableSample has wrong mem-reservation > - > > Key: IMPALA-7025 > URL: https://issues.apache.org/jira/browse/IMPALA-7025 > Project: IMPALA > Issue Type: Bug > Components: Frontend >Affects Versions: Impala 3.1.0 >Reporter: Joe McDonnell >Assignee: Tim Armstrong >Priority: Critical > Labels: broken-build, flaky > > Seen once on master exhaustive: > {noformat} > Error Message > Section PLAN of query: > select id from functional_parquet.alltypes tablesample system(10) > repeatable(1234) > Actual does not match expected result: > F00:PLAN FRAGMENT [UNPARTITIONED] hosts=1 instances=1 > | Per-Host Resources: mem-estimate=16.00MB mem-reservation=8.00KB > thread-reservation=2 > ^^^ > PLAN-ROOT SINK > | mem-estimate=0B mem-reservation=0B thread-reservation=0 > | > 00:SCAN HDFS [functional_parquet.alltypes] >partitions=3/24 files=3 size=23.70KB >stored statistics: > table: rows=unavailable size=unavailable > partitions: 0/24 rows=unavailable > columns: unavailable >extrapolated-rows=disabled max-scan-range-rows=unavailable >mem-estimate=16.00MB mem-reservation=8.00KB thread-reservation=1 >tuple-ids=0 row-size=4B cardinality=unavailable > Expected: > F00:PLAN FRAGMENT [UNPARTITIONED] hosts=1 instances=1 > | Per-Host Resources: mem-estimate=16.00MB mem-reservation=16.00KB > thread-reservation=2 > PLAN-ROOT SINK > | mem-estimate=0B mem-reservation=0B thread-reservation=0 > | > 00:SCAN HDFS [functional_parquet.alltypes] >partitions=3/24 files=3 size=24.23KB >stored statistics: > table: rows=unavailable size=unavailable > partitions: 0/24 rows=unavailable > columns: unavailable >extrapolated-rows=disabled max-scan-range-rows=unavailable >mem-estimate=16.00MB mem-reservation=16.00KB thread-reservation=1 >tuple-ids=0 row-size=4B cardinality=unavailable{noformat} > This succeeded on the next build, so it is flaky and might not recur. > -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Updated] (IMPALA-7027) Multiple Cast to Varchar with different limit fails with "AnalysisException: null CAUSED BY: IllegalArgumentException: "
[ https://issues.apache.org/jira/browse/IMPALA-7027?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tim Armstrong updated IMPALA-7027: -- Target Version: Impala 2.13.0, Impala 3.1.0 > Multiple Cast to Varchar with different limit fails with "AnalysisException: > null CAUSED BY: IllegalArgumentException: " > > > Key: IMPALA-7027 > URL: https://issues.apache.org/jira/browse/IMPALA-7027 > Project: IMPALA > Issue Type: Bug > Components: Frontend >Affects Versions: Impala 2.9.0 >Reporter: Meenakshi >Priority: Critical > > If we have multiple cast of '' to varchar statements in a impala query which > has a distinct like below, the query breaks for scenario when the cast to > varchar limit in the SQL is lower than the previous cast. > > Query 1> Fails with " AnalysisException: null CAUSED BY: > IllegalArgumentException: targetType=VARCHAR(100) type=VARCHAR(101)" > SELECT DISTINCT CAST('' as VARCHAR(101)) as CLAIM_COMMENTS,CAST('' as > VARCHAR(100)) as CA_USER_ID FROM pinnga1ph_ds_prd_wh_gbd.concept_flat_dtl > limit 1 > Where as the below query succeeds > Query 2> Success > SELECT DISTINCT CAST('' as VARCHAR(100)) as CLAIM_COMMENTS,CAST('' as > VARCHAR(101)) as CA_USER_ID FROM pinnga1ph_ds_prd_wh_gbd.concept_flat_dtl > limit 1 > The Query succeeds is we remove the distinct clause or add set > enable_expr_rewrites=false; -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Updated] (IMPALA-7027) Multiple Cast to Varchar with different limit fails with "AnalysisException: null CAUSED BY: IllegalArgumentException: "
[ https://issues.apache.org/jira/browse/IMPALA-7027?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tim Armstrong updated IMPALA-7027: -- Priority: Critical (was: Major) > Multiple Cast to Varchar with different limit fails with "AnalysisException: > null CAUSED BY: IllegalArgumentException: " > > > Key: IMPALA-7027 > URL: https://issues.apache.org/jira/browse/IMPALA-7027 > Project: IMPALA > Issue Type: Bug > Components: Frontend >Affects Versions: Impala 2.9.0 >Reporter: Meenakshi >Priority: Critical > > If we have multiple cast of '' to varchar statements in a impala query which > has a distinct like below, the query breaks for scenario when the cast to > varchar limit in the SQL is lower than the previous cast. > > Query 1> Fails with " AnalysisException: null CAUSED BY: > IllegalArgumentException: targetType=VARCHAR(100) type=VARCHAR(101)" > SELECT DISTINCT CAST('' as VARCHAR(101)) as CLAIM_COMMENTS,CAST('' as > VARCHAR(100)) as CA_USER_ID FROM pinnga1ph_ds_prd_wh_gbd.concept_flat_dtl > limit 1 > Where as the below query succeeds > Query 2> Success > SELECT DISTINCT CAST('' as VARCHAR(100)) as CLAIM_COMMENTS,CAST('' as > VARCHAR(101)) as CA_USER_ID FROM pinnga1ph_ds_prd_wh_gbd.concept_flat_dtl > limit 1 > The Query succeeds is we remove the distinct clause or add set > enable_expr_rewrites=false; -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Updated] (IMPALA-7027) Multiple Cast to Varchar with different limit fails with "AnalysisException: null CAUSED BY: IllegalArgumentException: "
[ https://issues.apache.org/jira/browse/IMPALA-7027?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tim Armstrong updated IMPALA-7027: -- Component/s: (was: Clients) Frontend > Multiple Cast to Varchar with different limit fails with "AnalysisException: > null CAUSED BY: IllegalArgumentException: " > > > Key: IMPALA-7027 > URL: https://issues.apache.org/jira/browse/IMPALA-7027 > Project: IMPALA > Issue Type: Bug > Components: Frontend >Affects Versions: Impala 2.9.0 >Reporter: Meenakshi >Priority: Critical > > If we have multiple cast of '' to varchar statements in a impala query which > has a distinct like below, the query breaks for scenario when the cast to > varchar limit in the SQL is lower than the previous cast. > > Query 1> Fails with " AnalysisException: null CAUSED BY: > IllegalArgumentException: targetType=VARCHAR(100) type=VARCHAR(101)" > SELECT DISTINCT CAST('' as VARCHAR(101)) as CLAIM_COMMENTS,CAST('' as > VARCHAR(100)) as CA_USER_ID FROM pinnga1ph_ds_prd_wh_gbd.concept_flat_dtl > limit 1 > Where as the below query succeeds > Query 2> Success > SELECT DISTINCT CAST('' as VARCHAR(100)) as CLAIM_COMMENTS,CAST('' as > VARCHAR(101)) as CA_USER_ID FROM pinnga1ph_ds_prd_wh_gbd.concept_flat_dtl > limit 1 > The Query succeeds is we remove the distinct clause or add set > enable_expr_rewrites=false; -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Updated] (IMPALA-7025) PlannerTest.testTableSample has wrong mem-reservation
[ https://issues.apache.org/jira/browse/IMPALA-7025?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joe McDonnell updated IMPALA-7025: -- Summary: PlannerTest.testTableSample has wrong mem-reservation (was: PlannerTest.testTableSample) > PlannerTest.testTableSample has wrong mem-reservation > - > > Key: IMPALA-7025 > URL: https://issues.apache.org/jira/browse/IMPALA-7025 > Project: IMPALA > Issue Type: Bug > Components: Frontend >Affects Versions: Impala 3.1.0 >Reporter: Joe McDonnell >Priority: Critical > Labels: broken-build, flaky > > Seen once on master exhaustive: > {noformat} > Error Message > Section PLAN of query: > select id from functional_parquet.alltypes tablesample system(10) > repeatable(1234) > Actual does not match expected result: > F00:PLAN FRAGMENT [UNPARTITIONED] hosts=1 instances=1 > | Per-Host Resources: mem-estimate=16.00MB mem-reservation=8.00KB > thread-reservation=2 > ^^^ > PLAN-ROOT SINK > | mem-estimate=0B mem-reservation=0B thread-reservation=0 > | > 00:SCAN HDFS [functional_parquet.alltypes] >partitions=3/24 files=3 size=23.70KB >stored statistics: > table: rows=unavailable size=unavailable > partitions: 0/24 rows=unavailable > columns: unavailable >extrapolated-rows=disabled max-scan-range-rows=unavailable >mem-estimate=16.00MB mem-reservation=8.00KB thread-reservation=1 >tuple-ids=0 row-size=4B cardinality=unavailable > Expected: > F00:PLAN FRAGMENT [UNPARTITIONED] hosts=1 instances=1 > | Per-Host Resources: mem-estimate=16.00MB mem-reservation=16.00KB > thread-reservation=2 > PLAN-ROOT SINK > | mem-estimate=0B mem-reservation=0B thread-reservation=0 > | > 00:SCAN HDFS [functional_parquet.alltypes] >partitions=3/24 files=3 size=24.23KB >stored statistics: > table: rows=unavailable size=unavailable > partitions: 0/24 rows=unavailable > columns: unavailable >extrapolated-rows=disabled max-scan-range-rows=unavailable >mem-estimate=16.00MB mem-reservation=16.00KB thread-reservation=1 >tuple-ids=0 row-size=4B cardinality=unavailable{noformat} > This succeeded on the next build, so it is flaky and might not recur. > -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Created] (IMPALA-7027) Multiple Cast to Varchar with different limit fails with "AnalysisException: null CAUSED BY: IllegalArgumentException: "
Meenakshi created IMPALA-7027: - Summary: Multiple Cast to Varchar with different limit fails with "AnalysisException: null CAUSED BY: IllegalArgumentException: " Key: IMPALA-7027 URL: https://issues.apache.org/jira/browse/IMPALA-7027 Project: IMPALA Issue Type: Bug Components: Clients Affects Versions: Impala 2.9.0 Reporter: Meenakshi If we have multiple cast of '' to varchar statements in a impala query which has a distinct like below, the query breaks for scenario when the cast to varchar limit in the SQL is lower than the previous cast. Query 1> Fails with " AnalysisException: null CAUSED BY: IllegalArgumentException: targetType=VARCHAR(100) type=VARCHAR(101)" SELECT DISTINCT CAST('' as VARCHAR(101)) as CLAIM_COMMENTS,CAST('' as VARCHAR(100)) as CA_USER_ID FROM pinnga1ph_ds_prd_wh_gbd.concept_flat_dtl limit 1 Where as the below query succeeds Query 2> Success SELECT DISTINCT CAST('' as VARCHAR(100)) as CLAIM_COMMENTS,CAST('' as VARCHAR(101)) as CA_USER_ID FROM pinnga1ph_ds_prd_wh_gbd.concept_flat_dtl limit 1 The Query succeeds is we remove the distinct clause or add set enable_expr_rewrites=false; -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Created] (IMPALA-7026) buildall.sh make shell fails due to sqlparse problem
Michael Brown created IMPALA-7026: - Summary: buildall.sh make shell fails due to sqlparse problem Key: IMPALA-7026 URL: https://issues.apache.org/jira/browse/IMPALA-7026 Project: IMPALA Issue Type: Bug Components: Infrastructure Affects Versions: Impala 3.1.0 Reporter: Michael Brown Assignee: Fredy Wijaya Fix For: Impala 3.1.0 https://jenkins.impala.io/job/ubuntu-16.04-from-scratch/2159/ {noformat} 22:43:05 Creating an egg for /home/ubuntu/Impala/shell/ext-py/sqlparse-0.1.14 22:43:05 [ 47%] Building CXX object be/src/udf_samples/CMakeFiles/udf-sample-test.dir/udf-sample-test.cc.o 22:43:05 [ 47%] Building CXX object be/src/util/CMakeFiles/Util.dir/hdr-histogram.cc.o 22:43:05 [ 47%] Built target uda-sample-test 22:43:05 [ 47%] Building CXX object be/src/util/CMakeFiles/Util.dir/impalad-metrics.cc.o 22:43:05 [ 47%] Building CXX object be/src/exprs/CMakeFiles/Exprs.dir/math-functions-ir.cc.o 22:43:05 [ 47%] Building CXX object be/src/testutil/CMakeFiles/TestUtil.dir/desc-tbl-builder.cc.o 22:43:05 [ 47%] Building CXX object be/src/testutil/CMakeFiles/TestUtil.dir/fault-injection-util.cc.o 22:43:05 [ 47%] Linking CXX executable ../../build/debug/udf_samples/udf-sample-test 22:43:05 [ 47%] Built target udf-sample-test 22:43:05 Scanning dependencies of target krpc 22:43:05 [ 47%] Building CXX object be/src/exprs/CMakeFiles/Exprs.dir/null-literal.cc.o 22:43:05 [ 47%] Building CXX object be/src/util/CMakeFiles/Util.dir/jni-util.cc.o 22:43:05 [ 48%] Building CXX object be/src/testutil/CMakeFiles/TestUtil.dir/impalad-query-executor.cc.o 22:43:05 warning: no files found matching 'docs/Makefile' 22:43:05 warning: no files found matching 'Makefile' 22:43:05 [ 48%] Building CXX object be/src/runtime/CMakeFiles/Runtime.dir/hbase-table.cc.o 22:43:05 [ 50%] Building CXX object be/src/kudu/rpc/CMakeFiles/krpc.dir/acceptor_pool.cc.o 22:43:05 [ 50%] Building CXX object be/src/testutil/CMakeFiles/TestUtil.dir/in-process-servers.cc.o 22:43:05 [ 50%] Building CXX object be/src/kudu/rpc/CMakeFiles/krpc.dir/blocking_ops.cc.o 22:43:05 [ 50%] Building CXX object be/src/kudu/rpc/CMakeFiles/krpc.dir/client_negotiation.cc.o 22:43:05 [ 50%] Building CXX object be/src/runtime/CMakeFiles/Runtime.dir/hbase-table-factory.cc.o 22:43:05 [ 50%] Building CXX object be/src/util/CMakeFiles/Util.dir/logging-support.cc.o 22:43:05 [ 50%] Building CXX object be/src/util/CMakeFiles/Util.dir/mem-info.cc.o 22:43:05 [ 50%] Building CXX object be/src/util/CMakeFiles/Util.dir/memory-metrics.cc.o 22:43:05 zip_safe flag not set; analyzing archive contents... 22:43:05 Cleaning up old build artifacts. 22:43:06 Creating an egg for /home/ubuntu/Impala/shell/ext-py/sqlparse-0.1.19 22:43:06 [ 51%] Building CXX object be/src/runtime/CMakeFiles/Runtime.dir/hdfs-fs-cache.cc.o 22:43:06 [ 51%] Building CXX object be/src/runtime/CMakeFiles/Runtime.dir/initial-reservations.cc.o 22:43:06 python: can't open file 'setup.py': [Errno 2] No such file or directory 22:43:06 Error in shell/make_shell_tarball.sh at line 96: python setup.py -q bdist_egg clean 22:43:06 CMakeFiles/shell_tarball.dir/build.make:57: recipe for target 'CMakeFiles/shell_tarball' failed 22:43:06 make[2]: *** [CMakeFiles/shell_tarball] Error 2 22:43:06 CMakeFiles/Makefile2:141: recipe for target 'CMakeFiles/shell_tarball.dir/all' failed 22:43:06 make[1]: *** [CMakeFiles/shell_tarball.dir/all] Error 2 22:43:06 make[1]: *** Waiting for unfinished jobs {noformat} I just reproduced this locally using {{./buildall.sh -notests -start_minicluster -start_impala_cluster}}. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Created] (IMPALA-7025) PlannerTest.testTableSample
Joe McDonnell created IMPALA-7025: - Summary: PlannerTest.testTableSample Key: IMPALA-7025 URL: https://issues.apache.org/jira/browse/IMPALA-7025 Project: IMPALA Issue Type: Bug Components: Frontend Affects Versions: Impala 3.1.0 Reporter: Joe McDonnell Seen once on master exhaustive: {noformat} Error Message Section PLAN of query: select id from functional_parquet.alltypes tablesample system(10) repeatable(1234) Actual does not match expected result: F00:PLAN FRAGMENT [UNPARTITIONED] hosts=1 instances=1 | Per-Host Resources: mem-estimate=16.00MB mem-reservation=8.00KB thread-reservation=2 ^^^ PLAN-ROOT SINK | mem-estimate=0B mem-reservation=0B thread-reservation=0 | 00:SCAN HDFS [functional_parquet.alltypes] partitions=3/24 files=3 size=23.70KB stored statistics: table: rows=unavailable size=unavailable partitions: 0/24 rows=unavailable columns: unavailable extrapolated-rows=disabled max-scan-range-rows=unavailable mem-estimate=16.00MB mem-reservation=8.00KB thread-reservation=1 tuple-ids=0 row-size=4B cardinality=unavailable Expected: F00:PLAN FRAGMENT [UNPARTITIONED] hosts=1 instances=1 | Per-Host Resources: mem-estimate=16.00MB mem-reservation=16.00KB thread-reservation=2 PLAN-ROOT SINK | mem-estimate=0B mem-reservation=0B thread-reservation=0 | 00:SCAN HDFS [functional_parquet.alltypes] partitions=3/24 files=3 size=24.23KB stored statistics: table: rows=unavailable size=unavailable partitions: 0/24 rows=unavailable columns: unavailable extrapolated-rows=disabled max-scan-range-rows=unavailable mem-estimate=16.00MB mem-reservation=16.00KB thread-reservation=1 tuple-ids=0 row-size=4B cardinality=unavailable{noformat} This succeeded on the next build, so it is flaky and might not recur. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Commented] (IMPALA-6987) Impala Doc: Clear up Impala' invalidate metadata page
[ https://issues.apache.org/jira/browse/IMPALA-6987?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16474638#comment-16474638 ] Alex Rodoni commented on IMPALA-6987: - [~balazsj_impala_220b] > Impala Doc: Clear up Impala' invalidate metadata page > - > > Key: IMPALA-6987 > URL: https://issues.apache.org/jira/browse/IMPALA-6987 > Project: IMPALA > Issue Type: Bug > Components: Docs >Affects Versions: Impala 2.12.0 >Reporter: Alex Rodoni >Assignee: Alex Rodoni >Priority: Major > > The usage notes section on this page has information that is from pre-catalog > service days: > {code:java} > A metadata update for an impalad instance is required if: > A metadata change occurs > and the change is made from another impalad instance in your cluster, or > through Hive. > and the change is made to a metastore database to which clients such as the > Impala shell or ODBC directly connect. > A metadata update for an Impala node is not required when you issue queries > from the same Impala node where you ran ALTER TABLE, INSERT, or other > table-modifying statement. > {code} > This should be cleared up. > The first point is incorrect, catalogd distributes metadata from other > impalads. > The last point is unnecessary and should be removed. > -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Commented] (IMPALA-4268) Allow PlanRootSink to buffer more than a batch of rows
[ https://issues.apache.org/jira/browse/IMPALA-4268?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16474634#comment-16474634 ] Dan Hecht commented on IMPALA-4268: --- [~tarmstrong], do you also expect to solve IMPALA-1618 with that? > Allow PlanRootSink to buffer more than a batch of rows > -- > > Key: IMPALA-4268 > URL: https://issues.apache.org/jira/browse/IMPALA-4268 > Project: IMPALA > Issue Type: Improvement > Components: Backend >Affects Versions: Impala 2.8.0 >Reporter: Henry Robinson >Priority: Major > Labels: resource-management > > In IMPALA-2905, we are introducing a {{PlanRootSink}} that handles the > production of output rows at the root of a plan. > The implementation in IMPALA-2905 has the plan execute in a separate thread > to the consumer, which calls {{GetNext()}} to retrieve the rows. However, the > sender thread will block until {{GetNext()}} is called, so that there are no > complications about memory usage and ownership due to having several batches > in flight at one time. > However, this also leads to many context switches, as each {{GetNext()}} call > yields to the sender to produce the rows. If the sender was to fill a buffer > asynchronously, the consumer could pull out of that buffer without taking a > context switch in many cases (and the extra buffering might smooth out any > performance spikes due to client delays, which currently directly affect plan > execution). > The tricky part is managing the mismatch between the size of the row batches > processed in {{Send()}} and the size of the fetch result asked for by the > client. The sender materializes output rows in a {{QueryResultSet}} that is > owned by the coordinator. That is not, currently, a splittable object - > instead it contains the actual RPC response struct that will hit the wire > when the RPC completes. As asynchronous sender cannot know the batch size, > which may change on every fetch call. So the {{GetNext()}} implementation > would need to be able to split out the {{QueryResultSet}} to match the > correct fetch size, and handle stitching together other {{QueryResultSets}} - > without doing extra copies. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Commented] (IMPALA-7000) Wrong info about Impala dedicated executore
[ https://issues.apache.org/jira/browse/IMPALA-7000?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16474632#comment-16474632 ] Alex Rodoni commented on IMPALA-7000: - https://gerrit.cloudera.org/#/c/10357/ > Wrong info about Impala dedicated executore > --- > > Key: IMPALA-7000 > URL: https://issues.apache.org/jira/browse/IMPALA-7000 > Project: IMPALA > Issue Type: Bug > Components: Docs >Affects Versions: Impala 2.12.0 >Reporter: Alex Rodoni >Assignee: Alex Rodoni >Priority: Major > > The following is not correct. > "Then, you specify that the other hosts act as executors but not > coordinators. These hosts do not communicate with the statestored daemon or > process the final result sets from queries. You cannot connect to these hosts > through clients such as impala-shell or business intelligence tools." > executor still communicates with statestore for other topics (membership, > admission control, etc.) The only part it doesn't get from statestore is the > metadata topic. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Commented] (IMPALA-7005) The 3.0 changelog page should mention the previous version
[ https://issues.apache.org/jira/browse/IMPALA-7005?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16474630#comment-16474630 ] Alex Rodoni commented on IMPALA-7005: - [~lv] [~sailesh] Could you tell me where the script to generate change logs? I would like to take a look to see if this is something I can take care of. Thanks! > The 3.0 changelog page should mention the previous version > -- > > Key: IMPALA-7005 > URL: https://issues.apache.org/jira/browse/IMPALA-7005 > Project: IMPALA > Issue Type: Bug > Components: Docs >Affects Versions: Impala 3.0 >Reporter: Lars Volker >Assignee: Alex Rodoni >Priority: Major > Labels: website > > It's not clear to me from looking at the [3.0 > changlog|http://impala.apache.org/docs/changelog-3.0.html] what base version > it compares to. Can we add a sentence to point out that these changes are in > comparison to 2.11 (are they?)? -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Resolved] (IMPALA-1790) FetchResults() sometimes returns very few resuts
[ https://issues.apache.org/jira/browse/IMPALA-1790?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dan Hecht resolved IMPALA-1790. --- Resolution: Duplicate > FetchResults() sometimes returns very few resuts > > > Key: IMPALA-1790 > URL: https://issues.apache.org/jira/browse/IMPALA-1790 > Project: IMPALA > Issue Type: Bug > Components: Backend >Affects Versions: Impala 2.1.1 >Reporter: Henry Robinson >Assignee: Dan Hecht >Priority: Major > > Hue has found that the {{FetchResults()}} RPC sometimes returns as few as 2 > results, when it really should be waiting for a full batch if possible. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Created] (IMPALA-7024) Convert Coordinator::wait_lock_ from boost::mutex to SpinLock
Dan Hecht created IMPALA-7024: - Summary: Convert Coordinator::wait_lock_ from boost::mutex to SpinLock Key: IMPALA-7024 URL: https://issues.apache.org/jira/browse/IMPALA-7024 Project: IMPALA Issue Type: Improvement Affects Versions: Impala 3.1.0 Reporter: Dan Hecht Assignee: Dan Hecht For consistency with other locks in this class, convert Coordinator::wait_lock_. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Commented] (IMPALA-7015) Insert into Kudu table returns with Status OK even if there are Kudu errors
[ https://issues.apache.org/jira/browse/IMPALA-7015?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16474595#comment-16474595 ] Lars Volker commented on IMPALA-7015: - [~mmokhtar] - I suspect you meant [~twmarshall] ? > Insert into Kudu table returns with Status OK even if there are Kudu errors > --- > > Key: IMPALA-7015 > URL: https://issues.apache.org/jira/browse/IMPALA-7015 > Project: IMPALA > Issue Type: Bug > Components: Backend >Affects Versions: Impala 2.12.0 >Reporter: Mostafa Mokhtar >Priority: Major > Attachments: Insert into kudu profile with errors.txt > > > DML statements against Kudu tables return status OK even if there are Kudu > errors. > This behavior is misleading. > {code} > Summary: > Session ID: 18430b000e5dd8dc:e3e5dadb4a15d4b4 > Session Type: BEESWAX > Start Time: 2018-05-11 10:10:07.314218000 > End Time: 2018-05-11 10:10:07.434017000 > Query Type: DML > Query State: FINISHED > Query Status: OK > Impala Version: impalad version 2.12.0-cdh5.15.0 RELEASE (build > 2f9498d5c2f980aa7ff9505c56654c8e59e026ca) > User: mmokhtar > Connected User: mmokhtar > Delegated User: > Network Address: :::10.17.234.27:60760 > Default Db: tpcds_1000_kudu > Sql Statement: insert into store_2 select * from store > Coordinator: vd1317.foo:22000 > Query Options (set by configuration): > Query Options (set by configuration and planner): MT_DOP=0 > Plan: > {code} > {code} > Operator #Hosts Avg Time Max Time #Rows Est. #Rows Peak Mem > Est. Peak Mem Detail > - > 02:PARTIAL SORT5 909.030us 1.025ms 1.00K 1.00K 6.14 MB > 4.00 MB > 01:EXCHANGE56.262ms 7.232ms 1.00K 1.00K 75.50 KB > 0 KUDU(KuduPartition(tpcds_1000_kudu.store.s_store_sk)) > 00:SCAN KUDU 53.694ms 4.137ms 1.00K 1.00K 4.34 MB > 0 tpcds_1000_kudu.store > Errors: Key already present in Kudu table > 'impala::tpcds_1000_kudu.store_2'. (1 of 1002 similar) > {code} -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Assigned] (IMPALA-7022) TestQueries.test_subquery: Subquery must not return more than one row
[ https://issues.apache.org/jira/browse/IMPALA-7022?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tim Armstrong reassigned IMPALA-7022: - Assignee: Zoltán Borók-Nagy > TestQueries.test_subquery: Subquery must not return more than one row > - > > Key: IMPALA-7022 > URL: https://issues.apache.org/jira/browse/IMPALA-7022 > Project: IMPALA > Issue Type: Bug > Components: Frontend >Affects Versions: Impala 2.13.0, Impala 3.1.0 >Reporter: Joe McDonnell >Assignee: Zoltán Borók-Nagy >Priority: Blocker > Labels: broken-build > > Seen on master and on 2.x based branches on exhaustive tests: > {noformat} > Error Message > query_test/test_queries.py:102: in test_subquery > self.run_test_case('QueryTest/subquery', vector) > common/impala_test_suite.py:408: in run_test_case > self.__verify_exceptions(test_section['CATCH'], str(e), use_db) > common/impala_test_suite.py:286: in __verify_exceptions (expected_str, > actual_str) E AssertionError: Unexpected exception string. Expected: Query > aborted:Subquery must not return more than one row: SELECT bigint_col FROM > functional.alltypes_view E Not found in actual: ImpalaBeeswaxException: > INNER EXCEPTION: MESSAGE: Subquery > must not return more than one row: SELECT bigint_col FROM > functional.alltypes_view > Stacktrace > query_test/test_queries.py:102: in test_subquery > self.run_test_case('QueryTest/subquery', vector) > common/impala_test_suite.py:408: in run_test_case > self.__verify_exceptions(test_section['CATCH'], str(e), use_db) > common/impala_test_suite.py:286: in __verify_exceptions > (expected_str, actual_str) > E AssertionError: Unexpected exception string. Expected: Query > aborted:Subquery must not return more than one row: SELECT bigint_col FROM > functional.alltypes_view > E Not found in actual: ImpalaBeeswaxException: INNER EXCEPTION: 'beeswaxd.ttypes.BeeswaxException'> MESSAGE: Subquery must not return more > than one row: SELECT bigint_col FROM functional.alltypes_view{noformat} -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Created] (IMPALA-7023) TestInsertQueries.test_insert_overwrite fails by hitting memory limit
Joe McDonnell created IMPALA-7023: - Summary: TestInsertQueries.test_insert_overwrite fails by hitting memory limit Key: IMPALA-7023 URL: https://issues.apache.org/jira/browse/IMPALA-7023 Project: IMPALA Issue Type: Bug Components: Backend Affects Versions: Impala 2.13.0, Impala 3.1.0 Reporter: Joe McDonnell This failure is seen on exhaustive builds on both master and 2.x: {noformat} Error Message ImpalaBeeswaxException: ImpalaBeeswaxException: INNER EXCEPTION: MESSAGE: AnalysisException: Failed to evaluate expr: 20 CAUSED BY: InternalException: Memory limit exceeded: Error occurred on backend impala-boost-static-burst-slave-el7-03d4.vpc.cloudera.com:22000 by fragment 0:0 Memory left in process limit: -4.29 GB Query(d24ea53242b4cedc:fc8e0885): Total=0 Peak=0 : Total=0 Peak=0Process: memory limit exceeded. Limit=12.00 GB Total=16.29 GB Peak=16.29 GB Buffer Pool: Free Buffers: Total=160.00 KB Buffer Pool: Clean Pages: Total=0 Buffer Pool: Unused Reservation: Total=-328.00 KB Data Stream Service Queue: Limit=614.40 MB Total=0 Peak=116.12 KB Data Stream Manager Early RPCs: Total=0 Peak=6.76 KB TCMalloc Overhead: Total=103.56 MB RequestPool=fe-eval-exprs: Total=0 Peak=52.83 KB Query(d24ea53242b4cedc:fc8e0885): Total=0 Peak=0 RequestPool=default-pool: Total=5.20 GB Peak=5.20 GB Query(f4014f7bb49ea78:6b926b19): memory limit exceeded. Limit=64.00 MB Reservation=0 ReservationLimit=32.00 MB OtherMemory=1.76 GB Total=1.76 GB Peak=1.96 GB Query(2c44b65fbcb4e1ce:3d73badf): memory limit exceeded. Limit=64.00 MB Reservation=0 ReservationLimit=32.00 MB OtherMemory=1.04 GB Total=1.04 GB Peak=1.61 GB Query(214cc23c1376176f:7844977b): memory limit exceeded. Limit=64.00 MB Reservation=0 ReservationLimit=32.00 MB OtherMemory=1.23 GB Total=1.23 GB Peak=1.23 GB Query(8949bdf792a32ad2:33a36c03): memory limit exceeded. Limit=64.00 MB Reservation=0 ReservationLimit=32.00 MB OtherMemory=642.20 MB Total=642.20 MB Peak=642.20 MB Query(5412ff4e6065721:519d3e61): memory limit exceeded. Limit=64.00 MB Reservation=0 ReservationLimit=32.00 MB OtherMemory=556.49 MB Total=556.49 MB Peak=556.49 MB Untracked Memory: Total=10.98 GB Stacktrace query_test/test_insert.py:132: in test_insert_overwrite multiple_impalad=vector.get_value('exec_option')['sync_ddl'] == 1) common/impala_test_suite.py:405: in run_test_case result = self.__execute_query(target_impalad_client, query, user=user) common/impala_test_suite.py:620: in __execute_query return impalad_client.execute(query, user=user) common/impala_connection.py:160: in execute return self.__beeswax_client.execute(sql_stmt, user=user) beeswax/impala_beeswax.py:173: in execute handle = self.__execute_query(query_string.strip(), user=user) beeswax/impala_beeswax.py:339: in __execute_query handle = self.execute_query_async(query_string, user=user) beeswax/impala_beeswax.py:335: in execute_query_async return self.__do_rpc(lambda: self.imp_service.query(query,)) beeswax/impala_beeswax.py:460: in __do_rpc raise ImpalaBeeswaxException(self.__build_error_message(b), b) E ImpalaBeeswaxException: ImpalaBeeswaxException: EINNER EXCEPTION: EMESSAGE: AnalysisException: Failed to evaluate expr: 20 E CAUSED BY: InternalException: Memory limit exceeded: Error occurred on backend impala-boost-static-burst-slave-el7-03d4.vpc.cloudera.com:22000 by fragment 0:0 E Memory left in process limit: -4.29 GB E Query(d24ea53242b4cedc:fc8e0885): Total=0 Peak=0 E : Total=0 Peak=0Process: memory limit exceeded. Limit=12.00 GB Total=16.29 GB Peak=16.29 GB E Buffer Pool: Free Buffers: Total=160.00 KB E Buffer Pool: Clean Pages: Total=0 E Buffer Pool: Unused Reservation: Total=-328.00 KB E Data Stream Service Queue: Limit=614.40 MB Total=0 Peak=116.12 KB E Data Stream Manager Early RPCs: Total=0 Peak=6.76 KB E TCMalloc Overhead: Total=103.56 MB E RequestPool=fe-eval-exprs: Total=0 Peak=52.83 KB E Query(d24ea53242b4cedc:fc8e0885): Total=0 Peak=0 E RequestPool=default-pool: Total=5.20 GB Peak=5.20 GB E Query(f4014f7bb49ea78:6b926b19): memory limit exceeded. Limit=64.00 MB Reservation=0 ReservationLimit=32.00 MB OtherMemory=1.76 GB Total=1.76 GB Peak=1.96 GB E Query(2c44b65fbcb4e1ce:3d73badf): memory limit exceeded. Limit=64.00 MB Reservation=0 ReservationLimit=32.00 MB OtherMemory=1.04 GB Total=1.04 GB Peak=1.61 GB E Query(214cc23c1376176f:7844977b): memory limit exceeded. Limit=64.00 MB Reservation=0 ReservationLimit=32.00 MB OtherMemory=1.23 GB Total=1.23 GB Peak=1.23 GB E Query(8949bdf792a32ad2:33a36c03): memory limit exceeded. Limit=64.00 MB Reservation=0 ReservationLimit=32.00 MB OtherMemory=642.20
[jira] [Commented] (IMPALA-6819) Add new performance test workloads
[ https://issues.apache.org/jira/browse/IMPALA-6819?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16474582#comment-16474582 ] ASF subversion and git services commented on IMPALA-6819: - Commit bb9237d942abc69efcf1b26bfd10f149dd781948 in impala's branch refs/heads/master from [~janarthanan] [ https://git-wip-us.apache.org/repos/asf?p=impala.git;h=bb9237d ] IMPALA-6819: Add new performance test workload - tpcds-unmodified used by Impala Performance Tests Description: Impala versions prior to 2.5 didn't have Runtimefilters, which made TPC-DS queries run very slow, so queries under tpcds have explicit partition filters to workaround the limitation. Post Impala 2.5 adding tpcds-unmodified which has the unmodified version of the workload to provide more coverage Testing: Ran the performance tests using the new workload and all tests passed Change-Id: I3957621d88b80fffc8fc89fd8104a58137b86e92 Reviewed-on: http://gerrit.cloudera.org:8080/9973 Reviewed-by: David KnuppTested-by: Impala Public Jenkins > Add new performance test workloads > --- > > Key: IMPALA-6819 > URL: https://issues.apache.org/jira/browse/IMPALA-6819 > Project: IMPALA > Issue Type: Task > Components: Infrastructure >Reporter: nithya >Assignee: nithya >Priority: Major > > Add additional workloads to impala-asf rep > Workloads that will be added > {code:java} > [targeted-perf] > [tpcds-unmodified] > {code} -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Commented] (IMPALA-6221) Kudu partition clause doc is wrong
[ https://issues.apache.org/jira/browse/IMPALA-6221?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16474583#comment-16474583 ] ASF subversion and git services commented on IMPALA-6221: - Commit 19bcc3099ef5e244fe74a0466b0b0eeb673acc8e in impala's branch refs/heads/master from [~arodoni_cloudera] [ https://git-wip-us.apache.org/repos/asf?p=impala.git;h=19bcc30 ] IMPALA-6221: [DOCS] Correct syntax for Kudu Partition clause Change-Id: I8d8ccdcc8b28fcd44511040f1e05969af83b3d1e Reviewed-on: http://gerrit.cloudera.org:8080/10353 Reviewed-by: Thomas Tauber-MarshallTested-by: Impala Public Jenkins > Kudu partition clause doc is wrong > -- > > Key: IMPALA-6221 > URL: https://issues.apache.org/jira/browse/IMPALA-6221 > Project: IMPALA > Issue Type: Bug > Components: Docs >Affects Versions: Impala 2.11.0 >Reporter: Thomas Tauber-Marshall >Assignee: Alex Rodoni >Priority: Major > Labels: newbie > Fix For: Impala 2.13.0, Impala 3.1.0 > > > The following line in the docs: > {noformat} > kudu_partition_clause ::= [hash_clause] [, range_clause [ , range_clause ] ] > {noformat} > which can be found here: > https://impala.apache.org/docs/build/html/topics/impala_create_table.html > is incorrect. We support an unlimited number of hash clauses followed by > either 0 or 1 range clauses, whereas the doc shows the opposite. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Closed] (IMPALA-6221) Kudu partition clause doc is wrong
[ https://issues.apache.org/jira/browse/IMPALA-6221?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alex Rodoni closed IMPALA-6221. --- Resolution: Fixed Fix Version/s: Impala 3.1.0 Impala 2.13.0 > Kudu partition clause doc is wrong > -- > > Key: IMPALA-6221 > URL: https://issues.apache.org/jira/browse/IMPALA-6221 > Project: IMPALA > Issue Type: Bug > Components: Docs >Affects Versions: Impala 2.11.0 >Reporter: Thomas Tauber-Marshall >Assignee: Alex Rodoni >Priority: Major > Labels: newbie > Fix For: Impala 2.13.0, Impala 3.1.0 > > > The following line in the docs: > {noformat} > kudu_partition_clause ::= [hash_clause] [, range_clause [ , range_clause ] ] > {noformat} > which can be found here: > https://impala.apache.org/docs/build/html/topics/impala_create_table.html > is incorrect. We support an unlimited number of hash clauses followed by > either 0 or 1 range clauses, whereas the doc shows the opposite. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Created] (IMPALA-7022) TestQueries.test_subquery: Subquery must not return more than one row
Joe McDonnell created IMPALA-7022: - Summary: TestQueries.test_subquery: Subquery must not return more than one row Key: IMPALA-7022 URL: https://issues.apache.org/jira/browse/IMPALA-7022 Project: IMPALA Issue Type: Bug Components: Frontend Affects Versions: Impala 2.13.0, Impala 3.1.0 Reporter: Joe McDonnell Seen on master and on 2.x based branches on exhaustive tests: {noformat} Error Message query_test/test_queries.py:102: in test_subquery self.run_test_case('QueryTest/subquery', vector) common/impala_test_suite.py:408: in run_test_case self.__verify_exceptions(test_section['CATCH'], str(e), use_db) common/impala_test_suite.py:286: in __verify_exceptions (expected_str, actual_str) E AssertionError: Unexpected exception string. Expected: Query aborted:Subquery must not return more than one row: SELECT bigint_col FROM functional.alltypes_view E Not found in actual: ImpalaBeeswaxException: INNER EXCEPTION: MESSAGE: Subquery must not return more than one row: SELECT bigint_col FROM functional.alltypes_view Stacktrace query_test/test_queries.py:102: in test_subquery self.run_test_case('QueryTest/subquery', vector) common/impala_test_suite.py:408: in run_test_case self.__verify_exceptions(test_section['CATCH'], str(e), use_db) common/impala_test_suite.py:286: in __verify_exceptions (expected_str, actual_str) E AssertionError: Unexpected exception string. Expected: Query aborted:Subquery must not return more than one row: SELECT bigint_col FROM functional.alltypes_view E Not found in actual: ImpalaBeeswaxException: INNER EXCEPTION: MESSAGE: Subquery must not return more than one row: SELECT bigint_col FROM functional.alltypes_view{noformat} -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Commented] (IMPALA-7008) TestSpillingDebugActionDimensions.test_spilling test setup fails
[ https://issues.apache.org/jira/browse/IMPALA-7008?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16474539#comment-16474539 ] Joe McDonnell commented on IMPALA-7008: --- OOM killer killed python: Out of memory: Kill process 14714 (python) score 780 or sacrifice child Killed process 14714, UID 1106, (python) total-vm:54718348kB, anon-rss:54115332kB, file-rss:276kB > TestSpillingDebugActionDimensions.test_spilling test setup fails > > > Key: IMPALA-7008 > URL: https://issues.apache.org/jira/browse/IMPALA-7008 > Project: IMPALA > Issue Type: Bug > Components: Infrastructure >Affects Versions: Impala 3.0, Impala 2.13.0 >Reporter: Sailesh Mukil >Priority: Blocker > Labels: broken-build, flaky > > We've seen multiple instances of this test failing with the following error: > {code:java} > Error Message > test setup failure > Stacktrace > Slave 'gw0' crashed while running > "query_test/test_spilling.py::TestSpillingDebugActionDimensions::()::test_spilling[exec_option: > {'debug_action': None, 'default_spillable_buffer_size': '256k'} | > table_format: parquet/none]" > {code} > We need to investigate why this is happening. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Commented] (IMPALA-6981) Set pthread names
[ https://issues.apache.org/jira/browse/IMPALA-6981?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16474426#comment-16474426 ] Lars Volker commented on IMPALA-6981: - Assuming we need to keep those printable, I think it would even make sense to shorten the prefix, eg to LaThNa (either automatically or manually) to make sure we capture as much of the ID as possible. If we drop the IDs, then it seems reasonable to just set the thread name to something short. I don't think we have that many different thread names and it could be a good ramp-up task to show folks around the codebase. > Set pthread names > - > > Key: IMPALA-6981 > URL: https://issues.apache.org/jira/browse/IMPALA-6981 > Project: IMPALA > Issue Type: Improvement > Components: Backend >Affects Versions: Impala 2.13.0, Impala 3.1.0 >Reporter: Tianyi Wang >Priority: Minor > Labels: newbie, ramp-up, supportability > > Currently impala has names for some threads but pthread names are not set. > pthread names can be use useful for debugging/logging, especially for some > process hang cases, so we should set it. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Assigned] (IMPALA-7016) Statement to allow setting ownership for database
[ https://issues.apache.org/jira/browse/IMPALA-7016?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Fredy Wijaya reassigned IMPALA-7016: Assignee: Fredy Wijaya > Statement to allow setting ownership for database > - > > Key: IMPALA-7016 > URL: https://issues.apache.org/jira/browse/IMPALA-7016 > Project: IMPALA > Issue Type: Sub-task > Components: Frontend >Affects Versions: Impala 3.0, Impala 2.13.0 >Reporter: Adam Holley >Assignee: Fredy Wijaya >Priority: Major > > Create statement to allow setting owner on database > {{ALTER DATABASE database_name SET OWNER [USER|ROLE] user_or_role;}} > examples: > ALTER DATABASE SET OWNER USER > ALTER DATABASE SET OWNER ROLE -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org