200 MB is below the region split point of 256 MB so your table will only have
one region, therefore hosted on one region server. You can split the table
manually by any of the following options:
1) Adjust hbase.hregion.max.filesize in hbase-site.xml (global setting)
2) Set the MAX_FILE_SIZE attribute on the table when creating it:
HTableDescriptor htd = new HTableDescriptor("foo");
htd.setMaxFileSize(67108864); // 64 MB
3) Use HBaseAdmin.split()
4) Use 'split' command in the HBase shell.
> We'll enlarge our cluster to 12PCs with 8G RAM.
I presume a lot more data will be stored into that cluster, then.
- Andy
________________________________
From: "[email protected]" <[email protected]>
To: [email protected]
Sent: Tuesday, September 1, 2009 7:15:03 PM
Subject: Re: Save data to HBase
Hi,
I want to put only 200MB data to a HBase table.
Just use a single thread to put 1MB data in a time.
So, there are 200 job in turn.
It occured WARN zookeeper.ZooKeeperWrapper: Failed to create /hbase,
while it run to around the 20th job.
I found that table has just one regin.
I wonder whether these tight job will make that region server choke.
About the region split, I want to know when the table's region will be
splitted.
Is its threshold setting in hbase-default.xml?
Following are a snippet of my code about putting my data to hbase, anything
can improve it?
table1 = new HTable("MyTable");
table1.setAutoFlush(false);
table1.setWriteBufferSize(1024*1024*12);
Put p = new Put(Bytes.toBytes(rs.getString(KeyColumn)));
p.add(Bytes.toBytes(FamColumn),
Bytes.toBytes("content"),Bytes.toBytes(json));
table1.put(p);
table1.flushCommits();
table1.close();
My cluster is only 3 PC with (1G,1G,2G RAM ,2 core CPU )
3 region server
3 ZK quorum server
We'll enlarge our cluster to 12PCs with 8G RAM.
Thanks
Fleming
Amandeep Khurana
<[email protected] To:
[email protected]
> cc: (bcc: Y_823910/TSMC)
Subject: Re: Save data to HBase
2009/09/02 03:07
AM
Please respond to
hbase-user
What do you mean by quick way? You can use the api to put data into it
through a standalone java program or you can use a MR job to do it..
On Tue, Sep 1, 2009 at 2:48 AM, <[email protected]> wrote:
> Hi there,
>
> Any quick way that I can save my data(12G) to HBase?
> Thanks
>
> Fleming
>
>
>
>
>
---------------------------------------------------------------------------
TSMC PROPERTY
This email communication (and any attachments) is proprietary information
for the sole use of its
intended recipient. Any unauthorized review, use or distribution by anyone
other than the intended
recipient is strictly prohibited. If you are not the intended recipient,
please notify the sender by
replying to this email, and then delete this email and any copies of it
immediately. Thank you.
---------------------------------------------------------------------------
