[
https://issues.apache.org/jira/browse/TRAFODION-231?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14706309#comment-14706309
]
Atanu Mishra commented on TRAFODION-231:
----------------------------------------
Weishiun Tsai (wei-shiun-tsai) wrote on 2014-04-30: #4
Khaled did some experiment by turning off the CQD:
cqd HBASE_COPROCESSORS 'off';
select count(*) took much longer when this CQD is turned off, but it did seem
to return the correct count. This may give us some hints on where the problem
is.
khaled Bouaziz (khaled-bouaziz) wrote on 2014-04-30: #5
Worked with Wei-shiun and suggested we test with cqd HBASE_COPROCESSORS 'off';
on 2 different clusters
and in both cases we got the the correct number of rows this time. The only
issue is that it took longer to return the number of rows than the case when
the cqd is to ON
this cane be considered as workaround for and we will more inverstigation
around the area of co-processors ad why the count is not correct when we
coprocessor and region split happens
>>upsert using load into fourmillion
+> select t1.a + t2.a +t3.a, 0 from hundred t1, tenK t2, ten t3
+> where t3.a in (3,5,7,9);
--- 4000000 row(s) inserted.
>>cqd HBASE_COPROCESSORS 'off';
--- SQL operation complete.
>>select count(*) from fourmillion;
(EXPR)
--------------------
4000000
--- 1 row(s) selected.
>>select count(*) from fourmillion;
(EXPR)
--------------------
4000000
--- 1 row(s) selected.
>>
khaled Bouaziz (khaled-bouaziz) wrote on 2014-05-01: #6
I did some debugging and it the java side stack trace looks like below
Thread [main] (Suspended (breakpoint at line 1287 in HTable)) (out of synch)
TransactionalTable(HTable).coprocessorExec(Class<T>, byte[], byte[],
Call<T,R>, Callback<R>) line: 1287 (out of synch)
TransactionalAggregationClient.rowCount(long, byte[], ColumnInterpreter<R,S>,
Scan) line: 56
HTableClient.coProcAggr(long, int, byte[], byte[], byte[], byte[], boolean,
int) line: 625 (out of synch)
public <T extends CoprocessorProtocol, R> void coprocessorExec(
Class<T> protocol, byte[] startKey, byte[] endKey,
Batch.Call<T,R> callable, Batch.Callback<R> callback)
throws IOException, Throwable {
// get regions covered by the row range
List<byte[]> keys = getStartKeysInRange(startKey, endKey);
connection.processExecs(protocol, keys, tableName, pool, callable,
callback);
}
I also did a search on the web and came across an HBASE -case (HBASE-6877
Coprocessor exec result is incorrect when region is in splitting)
https://issues.apache.org/jira/browse/HBASE-6877
This HABSE case seems still open and not fixed yet.
Description (HBASE-6877)
--------------------------
When we execute the coprocessor, we will called HTable#getStartKeysInRange
first and get the Keys to exec coprocessor,
if then some regions are split before execCoprocessor RPC, the Keys are
something wrong now, and the result we get is not integrated,
for example:
parent region is split into daughter region A and daughter region B,
we executed coprocessor on the parent region, but the result data is only
daughter region A or daughter region B
--------------------------
this issue seems related to the issue HBASE-6877
may need to re-assigne to the seatrans team since the code used is in that area
khaled Bouaziz (khaled-bouaziz) on 2014-05-06
Changed in trafodion:
assignee: khaled Bouaziz (khaled-bouaziz) → Kevin Xu (kai-hua-xu)
Stacey Johnson (sjohnson-w) on 2014-06-10
information type: Proprietary → Public
Pavani Puppala (pavani-puppala) on 2014-06-11
tags: added: dtm
removed: sql-exe
Narendra Goyal (narendra-goyal) on 2014-06-11
Changed in trafodion:
status: In Progress → Fix Committed
Weishiun Tsai (wei-shiun-tsai) wrote on 2014-06-11: #7
Verified on 2 clusters (centos-mapr1 and rhel-mapr1) using the GIT 0606_0930
build. This problem has been fixed:
>>set schema trafodion.sq_fixes_m6;
--- SQL operation complete.
>>
>>create table ten (a int not null) store by (a);
--- SQL operation complete.
>>
>>create table hundred (a int not null) store by (a);
--- SQL operation complete.
>>
>>create table tenK (a int not null) store by (a);
--- SQL operation complete.
>>
>>create table fourmillion (a largeint not null, b largeint default 0 )
+>no partition
+>--store by (a)
+>;
--- SQL operation complete.
>>
>>insert into ten values (1),(2),(3),(4),(5),(6),(7),(8),(9),(10);
--- 10 row(s) inserted.
>>
>>upsert using load into hundred
+>select t1.a + t2.a from ten t1, ten t2;
--- 100 row(s) inserted.
>>
>>select count(*) from hundred;
(EXPR)
--------------------
100
--- 1 row(s) selected.
>>
>>upsert using load into tenK
+>select t1.a + t2.a from hundred t1, hundred t2;
--- 10000 row(s) inserted.
>>
>>select count(*) from tenK;
(EXPR)
--------------------
10000
--- 1 row(s) selected.
>>
>>upsert using load into fourmillion
+>select t1.a + t2.a +t3.a, 0 from hundred t1, tenK t2, ten t3
+>where t3.a in (3,5,7,9);
--- 4000000 row(s) inserted.
>>
>>select count(*) from fourmillion;
(EXPR)
--------------------
4000000
--- 1 row(s) selected.
>>
Changed in trafodion:
status: Fix Committed → Fix Released
> LP Bug: 1309121 - select count(*) does not reflect upsert result immediately
> ----------------------------------------------------------------------------
>
> Key: TRAFODION-231
> URL: https://issues.apache.org/jira/browse/TRAFODION-231
> Project: Apache Trafodion
> Issue Type: Bug
> Components: dtm
> Reporter: Weishiun Tsai
> Assignee: kai-hua.xu
> Priority: Critical
>
> We often see this inconsistent row count returned by select count(*) right
> after upsert. The select statement would show a lower row count. The
> correct value is not returned until the user waits for a while. As shown
> here, the table fourmillion should really have 4000000 rows, but the select
> count(*) issued immediately after the upsert shows only 1842367 rows. The
> user has to wait for a while and reissue the select statement again to see
> the correct 4000000 rows returned:
> SQL>set schema trafodion.seaquest_fixes_m6;
> --- SQL operation complete.
> SQL>create table ten (a int not null) store by (a);
> --- SQL operation complete.
> SQL>create table hundred (a int not null) store by (a);
> --- SQL operation complete.
> SQL>create table tenK (a int not null) store by (a);
> --- SQL operation complete.
> SQL>create table fourmillion (a largeint not null, b largeint default 0 )
> no partition
> --store by (a)
> ;
> --- SQL operation complete.
> -------------------------------------------------------------------------------
> == TEST: tcase.test001
> -------------------------------------------------------------------------------
> SQL>insert into ten values (1),(2),(3),(4),(5),(6),(7),(8),(9),(10);
> --- 10 row(s) inserted.
> SQL>upsert using load into hundred
> select t1.a + t2.a from ten t1, ten t2;
> --- SQL operation complete.
> SQL>select count(*) from hundred;
> (EXPR)
> --------------------
> 100
> --- 1 row(s) selected.
> SQL>upsert using load into tenK
> select t1.a + t2.a from hundred t1, hundred t2;
> --- SQL operation complete.
> SQL>select count(*) from tenK;
> (EXPR)
> --------------------
> 10000
> --- 1 row(s) selected.
> SQL>upsert using load into fourmillion
> select t1.a + t2.a +t3.a, 0 from hundred t1, tenK t2, ten t3
> where t3.a in (3,5,7,9);
> --- SQL operation complete.
> SQL>select count(*) from fourmillion;
> (EXPR)
> --------------------
> 1842367
> --- 1 row(s) selected.
> =================================================
> After waiting for a while and reissue the same select statement again:
> -bash-4.1$ trafci.sh -h centos-mapr1.hpl.hp.com:37800 -u dontcare -p dontcare
>
> Welcome to Trafodion Command Interface
> Copyright(C) 2013-2014 Hewlett-Packard Development Company, L.P.
> Host Name/IP Address: centos-mapr1.hpl.hp.com:37800
> User Name: dontcare
> Connected to Trafodion
> SQL>select count(*) from trafodion.seaquest_fixes_m6.fourmillion;
> (EXPR)
> --------------------
> 4000000
> --- 1 row(s) selected.
> SQL>
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
