By multiple JVMs I mean multiple HBase clients, it's hard to get more basic than that. Were you planning on doing that 100TB upload from a single client? If so, you should revise your plans. Do split the input between many inserting processes, use MapReduce if you can as it will do that for you, and you will leverage the parallelism offered by HBase/Hadoop.
For the rest, remember that google is your friend. J-D On Tue, Aug 10, 2010 at 10:42 PM, Han Liu <ha...@andrew.cmu.edu> wrote: > Hi J-D, > > Can you explain a bit more about multiple JVMs？ For example how to use it in > the case of HBase clients? Or maybe point me to a reference on such topics > since I am not really an expert on Java. :p > > Thanks again for your reply. > > Han > > > On Aug 9, 2010, at 4:13 PM, Jean-Daniel Cryans wrote: > >>> I see. 0.89 is the still a developer release and I hear that it is not >>> stable. But it sounds really tempting because it boosts performance by a >>> lot. Can I trust it if my final goal is to insert about 100TB of data? What >>> could be the possible issues? Also when shall I expect to see a stable >>> release? >> >> The problem will be your 6 machines, not the software. And if you need >> to insert that much data, please use the bulk uploader as it will be >> much faster: http://hbase.apache.org/docs/r0.89.20100621/bulk-loads.html >> >>> >>> So in order to do multiple-client insertion I basically just need to create >>> multiple HTable objects to handle the insertions? Do I need to do >>> multi-threading manually? >> >> By multiple clients I mean multiple JVMs, you can also do >> multi-threading inside each client with their own HTable (since it's >> not thread-safe). But please use the bulk loader. >> >>> >>> And finally a possibly stupid question: how do I check what is the total >>> number of regions? >> >> Check the "status" command in the shell, or checkout the master web UI >> on port 60030 like it says here >> http://hbase.apache.org/docs/r0.20.6/api/overview-summary.html#runandconfirm >> >> J-D >> > > -- > Han Liu > SCS & HCI Institute > Undergrad. Class of 2012 > Carnegie Mellon University > > > > >
