[jira] [Commented] (IMPALA-7028) TestKuduOperations.test_kudu_update fails due to wrong NumModifiedRows
[
https://issues.apache.org/jira/browse/IMPALA-7028?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16487846#comment-16487846
]
Tim Armstrong commented on IMPALA-7028:
---
According to the profile, all of the Kudu scan nodes returned only a fraction
of the expected rows. I don't think the profile bug applies since it's a
successful DML, which waits for all the final status reports to be sent in.
{noformat}
18:26:14 E KUDU_SCAN_NODE (id=0):(Total: 4.999ms, non-child: 4.999ms,
% non-child: 100.00%)
18:26:14 E - BytesRead: 0
18:26:14 E - CollectionItemsRead: 0 (0)
18:26:14 E - KuduRemoteScanTokens: 0 (0)
18:26:14 E - NumScannerThreadsStarted: 1 (1)
18:26:14 E - PeakMemoryUsage: 12.00 KB (12288)
18:26:14 E - RowsRead: 10 (10)
18:26:14 E - RowsReturned: 10 (10)
18:26:14 E - RowsReturnedRate: 2.00 K/sec
18:26:14 E - ScanRangesComplete: 1 (1)
18:26:14 E - ScannerThreadsInvoluntaryContextSwitches: 4 (4)
18:26:14 E - ScannerThreadsTotalWallClockTime: 3.999ms
18:26:14 E- MaterializeTupleTime(*): 1.999ms
18:26:14 E- ScannerThreadsSysTime: 0.000ns
18:26:14 E- ScannerThreadsUserTime: 999.000us
18:26:14 E - ScannerThreadsVoluntaryContextSwitches: 3 (3)
18:26:14 E - TotalKuduScanRoundTrips: 1 (1)
18:26:14 E - TotalReadThroughput: 0.00 /sec
{noformat}
[~twmarshall] this looks like another case of Kudu scans mysteriously not
returning the expected number of rows. Are we tracking those with a single
JIRA. I guess the rows were inserted by the previous statement, run against the
same coordinator.
{noformat}
18:26:14 -- executing against localhost:21000
18:26:14 insert into tdata
18:26:14 select id, string_col, float_col, bigint_col, string_col, bool_col,
tinyint_col,
18:26:14 smallint_col, double_col, NULL, NULL, NULL from
functional_kudu.alltypes;
18:26:14
18:26:14 -- executing against localhost:21000
18:26:14 update tdata set vali = -1;
{noformat}
> TestKuduOperations.test_kudu_update fails due to wrong NumModifiedRows
> --
>
> Key: IMPALA-7028
> URL: https://issues.apache.org/jira/browse/IMPALA-7028
> Project: IMPALA
> Issue Type: Bug
> Components: Backend
>Affects Versions: Impala 3.1.0
>Reporter: Joe McDonnell
>Assignee: Tim Armstrong
>Priority: Critical
> Labels: broken-build, flaky
>
> Seen once on master exhaustive:
> {noformat}
> query_test/test_kudu.py:92: in test_kudu_update
> self.run_test_case('QueryTest/kudu_update', vector,
> use_db=unique_database)
> common/impala_test_suite.py:451: in run_test_case
> verify_runtime_profile(test_section['RUNTIME_PROFILE'],
> result.runtime_profile)
> common/test_result_verifier.py:590: in verify_runtime_profile
> actual))
> E AssertionError: Did not find matches for lines in runtime profile:
> E EXPECTED LINES:
> E NumModifiedRows: 7300
> ...
> # From the actual profile:
> E Partition: Default
> E NumModifiedRows: 32
> E NumRowErrors: 0{noformat}
> Will add a comment with the profile.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Commented] (IMPALA-7028) TestKuduOperations.test_kudu_update fails due to wrong NumModifiedRows
[
https://issues.apache.org/jira/browse/IMPALA-7028?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16474677#comment-16474677
]
Joe McDonnell commented on IMPALA-7028:
---
{noformat}
query_test/test_kudu.py:92: in test_kudu_update
self.run_test_case('QueryTest/kudu_update', vector, use_db=unique_database)
common/impala_test_suite.py:451: in run_test_case
verify_runtime_profile(test_section['RUNTIME_PROFILE'],
result.runtime_profile)
common/test_result_verifier.py:590: in verify_runtime_profile
actual))
E AssertionError: Did not find matches for lines in runtime profile:
E EXPECTED LINES:
E NumModifiedRows: 7300
E
E ACTUAL PROFILE:
E Query (id=de4b7a0e99753cd5:6ffa44fd):
E DEBUG MODE WARNING: Query profile created while running a DEBUG build of
Impala. Use RELEASE builds to measure query performance.
E Summary:
E Session ID: 7f4a67a73c9ecf4e:5d51ba77392831b4
E Session Type: BEESWAX
E Start Time: 2018-05-12 15:08:10.372271000
E End Time:
E Query Type: DML
E Query State: FINISHED
E Query Status: OK
E Impala Version: impalad version 3.1.0-SNAPSHOT DEBUG (build
e12ee485cf4c77203b144c053ee167509cc39374)
E User: jenkins
E Connected User: jenkins
E Delegated User:
E Network Address: 127.0.0.1:55173
E Default Db: test_kudu_update_628738b5
E Sql Statement: update tdata set vali = -1
E Coordinator: ec2-m2-4xlarge-centos-6-4-092a.vpc.cloudera.com:22000
E Query Options (set by configuration):
ABORT_ON_ERROR=1,DISABLE_CODEGEN=1,EXEC_SINGLE_NODE_ROWS_THRESHOLD=0,DISABLE_CODEGEN_ROWS_THRESHOLD=0
E Query Options (set by configuration and planner):
ABORT_ON_ERROR=1,DISABLE_CODEGEN=1,EXEC_SINGLE_NODE_ROWS_THRESHOLD=0,MT_DOP=0,DISABLE_CODEGEN_ROWS_THRESHOLD=0
E Plan:
E
E Max Per-Host Resource Reservation: Memory=0B Threads=2
E Per-Host Resource Estimates: Memory=10.00MB
E WARNING: The following tables are missing relevant table and/or column
statistics.
E test_kudu_update_628738b5.tdata
E
E F00:PLAN FRAGMENT [RANDOM] hosts=3 instances=3
E | Per-Host Resources: mem-estimate=0B mem-reservation=0B
thread-reservation=2
E UPDATE KUDU [test_kudu_update_628738b5.tdata]
E | mem-estimate=0B mem-reservation=0B thread-reservation=0
E |
E 00:SCAN KUDU [test_kudu_update_628738b5.tdata]
E mem-estimate=0B mem-reservation=0B thread-reservation=1
E tuple-ids=0 row-size=4B cardinality=unavailable
E
E Estimated Per-Host Mem: 10485760
E Tables Missing Stats: test_kudu_update_628738b5.tdata
E Per Host Min Memory Reservation:
ec2-m2-4xlarge-centos-6-4-092a.vpc.cloudera.com:22000(0)
ec2-m2-4xlarge-centos-6-4-092a.vpc.cloudera.com:22001(0)
ec2-m2-4xlarge-centos-6-4-092a.vpc.cloudera.com:22002(0)
E Request Pool: default-pool
E Admission result: Admitted immediately
E Query Compilation: 6.606ms
E - Metadata of all 1 tables cached: 1.087ms (1.087ms)
E - Analysis finished: 1.457ms (369.234us)
E - Value transfer graph computed: 1.508ms (51.577us)
E - Single node plan created: 4.839ms (3.330ms)
E - Runtime filters computed: 4.876ms (37.680us)
E - Distributed plan created: 4.934ms (57.370us)
E - Planning finished: 6.606ms (1.672ms)
E Query Timeline: 84.999ms
E - Query submitted: 999.994us (999.994us)
E - Planning finished: 8.999ms (7.999ms)
E - Submit for admission: 11.999ms (2.999ms)
E - Completed admission: 11.999ms (0.000ns)
E - Ready to start on 3 backends: 12.999ms (999.994us)
E - All 3 execution backends (3 fragment instances) started: 24.999ms
(11.999ms)
E - Last row fetched: 39.999ms (14.999ms)
E - Released admission control resources: 39.999ms (0.000ns)
E - DML data written: 40.999ms (999.993us)
E - DML Metastore update finished: 40.999ms (0.000ns)
E - Request finished: 40.999ms (0.000ns)
E - First row fetched: 82.999ms (41.999ms)
E- ComputeScanRangeAssignmentTimer: 0.000ns
E ImpalaServer:
E- ClientFetchWaitTimer: 0.000ns
E- MetastoreUpdateTimer: 0.000ns
E- RowMaterializationTimer: 0.000ns
E Execution Profile de4b7a0e99753cd5:6ffa44fd:(Total: 27.999ms,
non-child: 0.000ns, % non-child: 0.00%)
E Number of filters: 0
E Filter routing table:
EID Src. Node Tgt. Node(s) Target type Partition filter Pending
(Expected) First arrived Completed Enabled
E
---
E
E Backend startup latencies: Count: 3, min / max: 1ms / 11ms, 25th %-ile:
1ms, 50th %-ile: 2ms, 75th %-ile: 2ms, 90th %-ile: 11ms, 95th %-ile: 11ms,
99.9th %-ile: 11ms
E Per Node Peak
