[
https://issues.apache.org/jira/browse/TRAFODION-2775?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Suresh Subbiah reopened TRAFODION-2775:
---------------------------------------
Jenkins regressions failed once after this PR was merged.During "initialize
trafodion;" we get this exception stack from Java.
org.apache.hadoop.hbase.client.transactional.TransactionalTable.checkAndPut(TransactionalTable.java:795)
org.apache.hadoop.hbase.client.transactional.RMInterface.checkAndPut(RMInterface.java:635)
org.trafodion.sql.HTableClient.putRow(HTableClient.java:1594)
org.trafodion.sql.HBaseClient.insertRow(HBaseClient.java:1931).
2017-10-24 09:27:48,863, ERROR, SQL.EXE, Node Number: 0, CPU: 0, PIN: 15679,
Process Name: $Z000CSZ, SQLCODE: 8448, QID:
MXID11000015679212375597091582815000000000206U3333300_36_36, *** ERROR[8448]
Unable to access Hbase interface. Call to ExpHbaseInterface::checkAndInsertRow
returned error HBASE_ACCESS_ERROR(-706). Cause: java.io.IOException:
java.lang.NullPointerException
org.apache.hadoop.hbase.client.transactional.TransactionalTable.checkAndPut(TransactionalTable.java:795)
org.apache.hadoop.hbase.client.transactional.RMInterface.checkAndPut(RMInterface.java:635)
org.trafodion.sql.HTableClient.putRow(HTableClient.java:1594)
org.trafodion.sql.HBaseClient.insertRow(HBaseClient.java:1931).
This snippet from HMaster log indicates that all MD tables were created in
HBase.
2017-10-24 09:25:22,256 INFO org.apache.hadoop.hbase.master.HMaster:
Client=trafodion//172.31.3.24 create 'TRAFODION._MD_.VIEWS', {TABLE_ATTRIBUTES
=> {coprocessor$1 =>
'|org.apache.hadoop.hbase.coprocessor.transactional.TrxRegionObserver|1073741823|',
coprocessor$2 =>
'|org.apache.hadoop.hbase.coprocessor.transactional.TrxRegionEndpoint|1073741823|',
coprocessor$3 =>
'|org.apache.hadoop.hbase.coprocessor.AggregateImplementation|1073741823|'},
{NAME => '#1', DATA_BLOCK_ENCODING => 'NONE', BLOOMFILTER => 'ROW',
REPLICATION_SCOPE => '0', COMPRESSION => 'NONE', VERSIONS => '1', TTL =>
'FOREVER', MIN_VERSIONS => '0', KEEP_DELETED_CELLS => 'FALSE', BLOCKSIZE =>
'65536', IN_MEMORY => 'false', BLOCKCACHE => 'true'}, {NAME => 'mt_',
DATA_BLOCK_ENCODING => 'NONE', BLOOMFILTER => 'ROW', REPLICATION_SCOPE => '0',
COMPRESSION => 'NONE', VERSIONS => '2', TTL => 'FOREVER', MIN_VERSIONS => '0',
KEEP_DELETED_CELLS => 'FALSE', BLOCKSIZE => '65536', IN_MEMORY => 'true',
BLOCKCACHE => 'true'}
2017-10-24 09:25:22,272 INFO
org.apache.hadoop.hbase.master.handler.CreateTableHandler: Create table
TRAFODION._MD_.VIEWS
2017-10-24 09:25:22,304 INFO org.apache.hadoop.hbase.regionserver.HRegion:
creating HRegion TRAFODION._MD_.VIEWS HTD == 'TRAFODION._MD_.VIEWS',
{TABLE_ATTRIBUTES => {coprocessor$1 =>
'|org.apache.hadoop.hbase.coprocessor.transactional.TrxRegionObserver|1073741823|',
coprocessor$2 =>
'|org.apache.hadoop.hbase.coprocessor.transactional.TrxRegionEndpoint|1073741823|',
coprocessor$3 =>
'|org.apache.hadoop.hbase.coprocessor.AggregateImplementation|1073741823|'},
{NAME => '#1', DATA_BLOCK_ENCODING => 'NONE', BLOOMFILTER => 'ROW',
REPLICATION_SCOPE => '0', COMPRESSION => 'NONE', VERSIONS => '1', TTL =>
'FOREVER', MIN_VERSIONS => '0', KEEP_DELETED_CELLS => 'FALSE', BLOCKSIZE =>
'65536', IN_MEMORY => 'false', BLOCKCACHE => 'true'}, {NAME => 'mt_',
DATA_BLOCK_ENCODING => 'NONE', BLOOMFILTER => 'ROW', REPLICATION_SCOPE => '0',
COMPRESSION => 'NONE', VERSIONS => '2', TTL => 'FOREVER', MIN_VERSIONS => '0',
KEEP_DELETED_CELLS => 'FALSE', BLOCKSIZE => '65536', IN_MEMORY => 'true',
BLOCKCACHE => 'true'} RootDir = hdfs://slave-cm54.trafodion.org:8020/hbase/.tmp
Table name == TRAFODION._MD_.VIEWS
2017-10-24 09:25:22,315 INFO org.apache.hadoop.hbase.regionserver.HRegion:
Closed TRAFODION._MD_.VIEWS,,1508837122254.fc024211ee0b77221c0b9baa891a9128.
2017-10-24 09:25:22,322 INFO org.apache.hadoop.hbase.MetaTableAccessor: Added 2
2017-10-24 09:25:22,322 INFO org.apache.hadoop.hbase.master.AssignmentManager:
Assigning 1 region(s) to slave-cm54.trafodion.org,60020,1508836904162
2017-10-24 09:25:22,328 INFO org.apache.hadoop.hbase.master.RegionStates:
Transition {fc024211ee0b77221c0b9baa891a9128 state=OFFLINE, ts=1508837122322,
server=null} to {fc024211ee0b77221c0b9baa891a9128 state=PENDING_OPEN,
ts=1508837122328, server=slave-cm54.trafodion.org,60020,1508836904162}
2017-10-24 09:25:22,334 WARN
org.apache.hadoop.hbase.zookeeper.ZKTableStateManager: Moving table
TRAFODION._MD_.VIEWS state from ENABLING to ENABLED
2017-10-24 09:25:22,338 INFO org.apache.hadoop.hbase.master.RegionStates:
Transition {fc024211ee0b77221c0b9baa891a9128 state=PENDING_OPEN,
ts=1508837122328, server=slave-cm54.trafodion.org,60020,1508836904162} to
{fc024211ee0b77221c0b9baa891a9128 state=OPENING, ts=1508837122338,
server=slave-cm54.trafodion.org,60020,1508836904162}
2017-10-24 09:25:22,345 INFO
org.apache.hadoop.hbase.master.handler.CreateTableHandler: Table,
TRAFODION._MD_.VIEWS, creation successful
2017-10-24 09:25:22,374 INFO org.apache.hadoop.hbase.master.RegionStates:
Transition {fc024211ee0b77221c0b9baa891a9128 state=OPENING, ts=1508837122338,
server=slave-cm54.trafodion.org,60020,1508836904162} to
{fc024211ee0b77221c0b9baa891a9128 state=OPEN, ts=1508837122374,
server=slave-cm54.trafodion.org,60020,1508836904162}
2017-10-24 09:25:23,442 INFO org.apache.hadoop.hbase.master.HMaster:
Client=trafodion//172.31.3.24 create 'TRAFODION._MD_.VIEWS_USAGE',
{TABLE_ATTRIBUTES => {coprocessor$1 =>
'|org.apache.hadoop.hbase.coprocessor.transactional.TrxRegionObserver|1073741823|',
coprocessor$2 =>
'|org.apache.hadoop.hbase.coprocessor.transactional.TrxRegionEndpoint|1073741823|',
coprocessor$3 =>
'|org.apache.hadoop.hbase.coprocessor.AggregateImplementation|1073741823|'},
{NAME => '#1', DATA_BLOCK_ENCODING => 'NONE', BLOOMFILTER => 'ROW',
REPLICATION_SCOPE => '0', COMPRESSION => 'NONE', VERSIONS => '1', TTL =>
'FOREVER', MIN_VERSIONS => '0', KEEP_DELETED_CELLS => 'FALSE', BLOCKSIZE =>
'65536', IN_MEMORY => 'false', BLOCKCACHE => 'true'}, {NAME => 'mt_',
DATA_BLOCK_ENCODING => 'NONE', BLOOMFILTER => 'ROW', REPLICATION_SCOPE => '0',
COMPRESSION => 'NONE', VERSIONS => '2', TTL => 'FOREVER', MIN_VERSIONS => '0',
KEEP_DELETED_CELLS => 'FALSE', BLOCKSIZE => '65536', IN_MEMORY => 'true',
BLOCKCACHE => 'true'}
2017-10-24 09:25:23,457 INFO
org.apache.hadoop.hbase.master.handler.CreateTableHandler: Create table
TRAFODION._MD_.VIEWS_USAGE
2017-10-24 09:25:23,475 INFO org.apache.hadoop.hbase.regionserver.HRegion:
creating HRegion TRAFODION._MD_.VIEWS_USAGE HTD ==
'TRAFODION._MD_.VIEWS_USAGE', {TABLE_ATTRIBUTES => {coprocessor$1 =>
'|org.apache.hadoop.hbase.coprocessor.transactional.TrxRegionObserver|1073741823|',
coprocessor$2 =>
'|org.apache.hadoop.hbase.coprocessor.transactional.TrxRegionEndpoint|1073741823|',
coprocessor$3 =>
'|org.apache.hadoop.hbase.coprocessor.AggregateImplementation|1073741823|'},
{NAME => '#1', DATA_BLOCK_ENCODING => 'NONE', BLOOMFILTER => 'ROW',
REPLICATION_SCOPE => '0', COMPRESSION => 'NONE', VERSIONS => '1', TTL =>
'FOREVER', MIN_VERSIONS => '0', KEEP_DELETED_CELLS => 'FALSE', BLOCKSIZE =>
'65536', IN_MEMORY => 'false', BLOCKCACHE => 'true'}, {NAME => 'mt_',
DATA_BLOCK_ENCODING => 'NONE', BLOOMFILTER => 'ROW', REPLICATION_SCOPE => '0',
COMPRESSION => 'NONE', VERSIONS => '2', TTL => 'FOREVER', MIN_VERSIONS => '0',
KEEP_DELETED_CELLS => 'FALSE', BLOCKSIZE => '65536', IN_MEMORY => 'true',
BLOCKCACHE => 'true'} RootDir = hdfs://slave-cm54.trafodion.org:8020/hbase/.tmp
Table name == TRAFODION._MD_.VIEWS_USAGE
2017-10-24 09:25:23,486 INFO org.apache.hadoop.hbase.regionserver.HRegion:
Closed
TRAFODION._MD_.VIEWS_USAGE,,1508837123440.12418588039df9e0e710ed90f71c3327.
2017-10-24 09:25:23,492 INFO org.apache.hadoop.hbase.MetaTableAccessor: Added 2
2017-10-24 09:25:23,492 INFO org.apache.hadoop.hbase.master.AssignmentManager:
Assigning 1 region(s) to slave-cm54.trafodion.org,60020,1508836904162
2017-10-24 09:25:23,498 INFO org.apache.hadoop.hbase.master.RegionStates:
Transition {12418588039df9e0e710ed90f71c3327 state=OFFLINE, ts=1508837123492,
server=null} to {12418588039df9e0e710ed90f71c3327 state=PENDING_OPEN,
ts=1508837123498, server=slave-cm54.trafodion.org,60020,1508836904162}
2017-10-24 09:25:23,504 WARN
org.apache.hadoop.hbase.zookeeper.ZKTableStateManager: Moving table
TRAFODION._MD_.VIEWS_USAGE state from ENABLING to ENABLED
2017-10-24 09:25:23,524 INFO org.apache.hadoop.hbase.master.RegionStates:
Transition {12418588039df9e0e710ed90f71c3327 state=PENDING_OPEN,
ts=1508837123498, server=slave-cm54.trafodion.org,60020,1508836904162} to
{12418588039df9e0e710ed90f71c3327 state=OPENING, ts=1508837123524,
server=slave-cm54.trafodion.org,60020,1508836904162}
2017-10-24 09:25:23,530 INFO
org.apache.hadoop.hbase.master.handler.CreateTableHandler: Table,
TRAFODION._MD_.VIEWS_USAGE, creation successful
2017-10-24 09:25:23,555 INFO org.apache.hadoop.hbase.master.RegionStates:
Transition {12418588039df9e0e710ed90f71c3327 state=OPENING, ts=1508837123524,
server=slave-cm54.trafodion.org,60020,1508836904162} to
{12418588039df9e0e710ed90f71c3327 state=OPEN, ts=1508837123555,
server=slave-cm54.trafodion.org,60020,1508836904162}
2017-10-24 09:26:44,694 INFO org.apache.hadoop.hbase.io.hfile.LruBlockCache:
totalSize=209.16 KB, freeSize=197.77 MB, max=197.98 MB, blockCount=0,
accesses=0, hits=0, hitRatio=0, cachingAccesses=0, cachingHits=0,
cachingHitsRatio=0,evictions=29, evicted=0, evictedPerRun=0.0
2017-10-24 09:27:22,626 INFO org.apache.hadoop.hbase.master.HMaster:
Client=trafodion//172.31.3.24 disable TRAFODION._MD_.AUTHS
2017-10-24 09:27:22,637 INFO
org.apache.hadoop.hbase.master.handler.DisableTableHandler: Attempting to
disable table TRAFODION._MD_.AUTHS
2017-10-24 09:27:22,637 WARN
org.apache.hadoop.hbase.zookeeper.ZKTableStateManager: Moving table
TRAFODION._MD_.AUTHS state from DISABLING to DISABLING
2017-10-24 09:27:22,639 INFO
org.apache.hadoop.hbase.master.handler.DisableTableHandler: Offlining 1 regions.
2017-10-24 09:27:22,641 INFO org.apache.hadoop.hbase.master.RegionStates:
Transition {5c7b2edc80d207443f7c80976b951ea8 state=OPEN, ts=1508837099144,
server=slave-cm54.trafodion.org,60020,1508836904162} to
{5c7b2edc80d207443f7c80976b951ea8 state=PENDING_CLOSE, ts=1508837242641,
server=slave-cm54.trafodion.org,60020,1508836904162}
2017-10-24 09:27:22,665 INFO org.apache.hadoop.hbase.master.RegionStates:
Transition {5c7b2edc80d207443f7c80976b951ea8 state=PENDING_CLOSE,
ts=1508837242641, server=slave-cm54.trafodion.org,60020,1508836904162} to
{5c7b2edc80d207443f7c80976b951ea8 state=OFFLINE, ts=1508837242665,
server=slave-cm54.trafodion.org,60020,1508836904162}
2017-10-24 09:27:22,665 INFO org.apache.hadoop.hbase.master.RegionStates:
Offlined 5c7b2edc80d207443f7c80976b951ea8 from
slave-cm54.trafodion.org,60020,1508836904162
2017-10-24 09:27:23,639 WARN
org.apache.hadoop.hbase.zookeeper.ZKTableStateManager: Moving table
TRAFODION._MD_.AUTHS state from DISABLING to DISABLED
2017-10-24 09:27:23,642 INFO
org.apache.hadoop.hbase.master.handler.DisableTableHandler: Disabled table,
TRAFODION._MD_.AUTHS, is done=true
2017-10-24 09:27:23,794 INFO org.apache.hadoop.hbase.master.HMaster:
Client=trafodion//172.31.3.24 delete TRAFODION._MD_.AUTHS
2017-10-24 09:27:23,806 INFO
org.apache.hadoop.hbase.master.handler.TableEventHandler: Handling table
operation C_M_DELETE_TABLE on table TRAFODION._MD_.AUTHS
2017-10-24 09:27:23,819 INFO org.apache.hadoop.hbase.MetaTableAccessor: Deleted
[{ENCODED => 5c7b2edc80d207443f7c80976b951ea8, NAME =>
'TRAFODION._MD_.AUTHS,,1508837098871.5c7b2edc80d207443f7c80976b951ea8.',
STARTKEY => '', ENDKEY => ''}]
> Insert does not raise duplicate row error for hbase format table with
> defaulted first column
> ---------------------------------------------------------------------------------------------
>
> Key: TRAFODION-2775
> URL: https://issues.apache.org/jira/browse/TRAFODION-2775
> Project: Apache Trafodion
> Issue Type: Bug
> Components: sql-exe
> Affects Versions: any
> Reporter: Suresh Subbiah
> Assignee: Suresh Subbiah
> Fix For: 2.3-incubating
>
>
> This issue was found by Gunnar Tapper and Carol Pearson.
> With HBase format Trafodion tables (each column in a row is a separate Cell),
> if the first column in the table can be defaulted, then uniqueness violations
> are not raised as expected for INSERT statements. Here is an example
> create table def1 (c1 int, c2 int not null, c3 int, primary key (c2))
> attributes hbase format;
> insert into def1 (c2) values (1);
> -- raises unique constraint error as expected
> insert into def1 (c2, c3) values (1,3);
> -- does not raise constraint error
> insert into def1 (c1, c2) values (1,1);
> The problem is that during the checkAndPut for INSERTcall we are specifying
> the column to be checked as the one that has index 0 in the row being
> inserted. This would the first column being inserted into for the row, as
> specified by DDL, once omitted columns are removed. Columns with default
> value could be omitted in a given INSERT, if they are not part of the
> clustering key.
> The fix utilizes that fact that clustering key columns are always present in
> the row being inserted, even if they can be defaulted and not explicitly in
> the INSERT statement. We now pass in the index of the first clustering key
> column, in the row being inserted, to the java layer. The java layer will get
> the column name/qualifier from the java byte buffer version of row being
> inserted and use it in the CheckAndPut call. Not that the index of first
> clustering key column will depend on both which default columns are being
> skipped and order of columns in DDL. This index does not depend only on DDL.
> With the change we get the expected error
> >>insert into def1 (c1, c2) values (1,1);
> *** ERROR[8102] The operation is prevented by a unique constraint.
> --- 0 row(s) inserted.
> >>insert into def1 (c2, c1) values (51,1);
> --- 1 row(s) inserted.
> >>insert into def1 (c2, c1) values (1,51);
> *** ERROR[8102] The operation is prevented by a unique constraint.
> --- 0 row(s) inserted.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
