[
https://issues.apache.org/jira/browse/OMID-240?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17788611#comment-17788611
]
Rajeshbabu Chintaguntla edited comment on OMID-240 at 11/22/23 4:16 AM:
------------------------------------------------------------------------
Currently with default configurations client TSO server initialising with
inmemory storage modules.
- With InMemoryCommitTableStorageModule, NullCommitTable is getting used which
doesn't maintain any commit timestamps information.
{noformat}
public class InMemoryCommitTableStorageModule extends AbstractModule {
@Override
public void configure() {
bind(CommitTable.class).to(NullCommitTable.class).in(Singleton.class);
}
}
{noformat}
{noformat}
public static class Writer implements CommitTable.Writer {
@Override
public void addCommittedTransaction(long startTimestamp, long
commitTimestamp) {
// noop
}
{noformat}
- What ever the commit timestamp generated by Transaction Oracle persisted in
shadow cells only.
- At the same time in the server side HBase Commit Table is getting used to
fetch the commit timestamp of the transactions which will not have any
information because TSO not writing to Hbase commit table.
{noformat}
connection = RegionConnectionFactory
.getConnection(RegionConnectionFactory.ConnectionType.READ_CONNECTION,
(RegionCoprocessorEnvironment) env);
commitTableClient = new HBaseCommitTable(connection,
commitTableConf).getClient();
LOG.info("Snapshot filter started");
{noformat}
- During the commit the tso client updates the shadow cells with the commit
timestamp.
{noformat}
private void commitRegularTransaction(AbstractTransaction<? extends CellId>
tx)
throws RollbackException, TransactionException
{
try {
long commitTs = tsoClient.commit(tx.getStartTimestamp(),
tx.getWriteSet(), tx.getConflictFreeWriteSet()).get();
certifyCommitForTx(tx, commitTs);
updateShadowCellsAndRemoveCommitTableEntry(tx, postCommitter);
{noformat}
- Omid Filters at the server checks for commit timestamp from shadow cells or
commit table, while filtering chances of the cells will be skipped, if commit
timestamp available neither in shadow cells not commit table. This is what
happening when both commit and scan running at the same time and mismatch is
number of records.
{noformat}
if (CellUtils.isShadowCell(v)) {
Long commitTs = Bytes.toLong(CellUtil.cloneValue(v));
commitCache.put(v.getTimestamp(), commitTs);
// Continue getting shadow cells until one of them fits this
transaction
if (hbaseTransaction.getStartTimestamp() >= commitTs) {
return ReturnCode.NEXT_COL;
} else {
return ReturnCode.SKIP;
}
}
Optional<Long> commitTS = getCommitIfInSnapshot(v,
CellUtils.isFamilyDeleteCell(v));
if (commitTS.isPresent()) {
....
}
return ReturnCode.SKIP;
{noformat}
This issue is not happening when HBase Storage Modules is configured for TSO
server where the omid filters can find the commit timestamp in case of shadow
cells doens't have.
So it can be avoided with proper usage the HBase storage modules in TSO server.
[~stoty] I think this can be closed as not an issue. WDYT?
was (Author: rajeshbabu):
Currently with default configurations client TSO server initialising with
inmemory storage modules.
- With InMemoryCommitTableStorageModule, NullCommitTable is getting used which
doesn't maintain any commit timestamps information.
- What ever the commit timestamp generated by Transaction Oracle persisted in
shadow cells only.
- At the same time in the server side HBase Commit Table is getting used to
fetch the commit timestamp of the transactions which will not have any
information because TSO not writing to Hbase commit table.
- During the commit the tso client updates the shadow cells with the commit
timestamp.
- Omid Filters at the server checks for commit timestamp from shadow cells or
commit table, while filtering chances of the cells will be skipped, if commit
timestamp available neither in shadow cells not commit table. This is what
happening when both commit and scan running at the same time and mismatch is
number of records.
This issue is not happening when HBase Storage Modules is configured for TSO
server where the omid filters can find the commit timestamp in case of shadow
cells doens't have.
So it can be avoided with proper usage the HBase storage modules in TSO server.
[~stoty] I think this can be closed as not an issue. WDYT?
> Transactional visibility is broken
> ----------------------------------
>
> Key: OMID-240
> URL: https://issues.apache.org/jira/browse/OMID-240
> Project: Phoenix Omid
> Issue Type: Bug
> Affects Versions: 1.1.0
> Reporter: Lars Hofhansl
> Assignee: Rajeshbabu Chintaguntla
> Priority: Critical
> Attachments: hbase-omid-client-config.yml,
> omid-server-configuration.yml
>
>
> Client I:
> {code:java}
> > create table test(x float primary key, y float) DISABLE_WAL=true,
> TRANSACTIONAL=true;
> No rows affected (1.872 seconds)
> > !autocommit off
> Autocommit status: false
> > upsert into test values(rand(), rand());
> 1 row affected (0.018 seconds)
> > upsert into test select rand(), rand() from test;
> -- 18-20x
> > !commit{code}
>
> Client II:
> {code:java}
> -- repeat quickly after the commit on client I
> > select count(*) from test;
> +----------+
> | COUNT(1) |
> +----------+
> | 0 |
> +----------+
> 1 row selected (1.408 seconds)
> > select count(*) from test;
> +----------+
> | COUNT(1) |
> +----------+
> | 259884 |
> +----------+
> 1 row selected (2.959 seconds)
> > select count(*) from test;
> +----------+
> | COUNT(1) |
> +----------+
> | 260145 |
> +----------+
> 1 row selected (4.274 seconds)
> > select count(*) from test;
> +----------+
> | COUNT(1) |
> +----------+
> | 260148 |
> +----------+
> 1 row selected (5.563 seconds)
> > select count(*) from test;
> +----------+
> | COUNT(1) |
> +----------+
> | 260148 |
> +----------+
> 1 row selected (5.573 seconds){code}
> The second client should either show 0 or 260148. But no other value!
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
