[
https://issues.apache.org/jira/browse/HBASE-3197?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12930561#action_12930561
]
Francesco Angi commented on HBASE-3197:
---------------------------------------
Hello and sorry for the late reply. I did some tests trying to reproduce a
"small unit test" but that wasn't that simple. I'll try to explain. The DAO I
developed is part of a server accepting telnet connections and storing the data
it computes to HBase. The server is built upon Mina. I wasn't able to produce a
local program that shows the same erroneous behaviour as the server. I tried
running a simple for loop, calling the DAO's saving method and every data were
saved correctly. So I tried running multiple threads and everything went ok too.
Here's a snippet of the DAO involved:
{code:java}
public class HBaseDataStorageDAO implements DataStorageDAO {
private TSerializer serializer;
private TDeserializer deserializer;
private HBaseConfiguration config;
private String zookeperQuorumList;
public HBaseDataStorageDAO(String zookeperQuorumList) throws
DataStorageException {
serializer = new TSerializer(new TBinaryProtocol.Factory());
deserializer = new TDeserializer(new TBinaryProtocol.Factory());
this.zookeperQuorumList = zookeperQuorumList;
config = new HBaseConfiguration();
config.set("hbase.zookeeper.quorum", zookeeperQuorumList);
}
public void saveDataInfo(String dataId, DataInfoWrapper dataInfoWrapper)
throws DataStorageException {
HTable tableData = null;
try {
tableData = new HTable(config, "dataTable");
Put p = new Put(Bytes.toBytes(dataId));
p.add(Bytes.toBytes("dataInfo"), // column family
Bytes.toBytes(""), // no column id
serializer.serialize(dataInfoWrapper.getDataInfo())); //
value
tableDatas.put(p);
} catch (IOException e) {
handleException(e);
} catch (TException e) {
handleException(e);
} finally {
if (tableData != null)
try {
tableData.close();
} catch (IOException e) {
handleException(e);
}
}
}
}
{code}
When the server starts it instantiates a HBaseDataStorageDAO, so as you can see
every saveDataInfo call shares the same config, serializer and deserializer
objects. I suspected that the problem could be the single serializer and
deserializer, so I tried instantiating a new TSerializer in every saveDataInfo
invocation, not even this solved the problem. As I said in my first post the
only solution I found was creating a new HBaseConfiguration for each
saveDataInfo invocation, and this is a bit weird, at least in my opinion.
In conclusion I'm not sure if I posted a "false alarm", indeed the problem
could be related to the underlying architecture of my server.
Sorry if I wasted your time,
best regards,
f.
> Using the same HBaseConfiguration for multiple puts makes some data not to be
> written
> -------------------------------------------------------------------------------------
>
> Key: HBASE-3197
> URL: https://issues.apache.org/jira/browse/HBASE-3197
> Project: HBase
> Issue Type: Bug
> Affects Versions: 0.20.6
> Environment: HBase cluster running on 6 machines (Centos 5.5, Intel
> Dual Core/core i3, ram 4/8GB)
> Reporter: Francesco Angi
>
> I created a DAO object for loading and storing data into HBase. The DAO has a
> HBaseConfiguration field, created inside the DAO constructor. Each DAO's
> method creates a new HTable using the class's HBaseConfiguration. The problem
> shows up when subsequent writings (using a put) are invoked, since not all
> the data is written. Moreover this behaviour is not deterministic: invoking
> the same writings never writes all the data, but the missing ones change
> every time.
> I solved this situation removing the class's HBaseConfiguration and creating
> a new HBaseConfiguration inside each method.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
