Re: Why is scaling HBase much simpler then scaling a relational db?
I've read some papers and tutorials this week and now got some conrete questions: (1) Sharding is also available in common relational systems. Often it is discribed that you need an application layer for the (shards) federation. I unterstand HBase like this layer, which implements the whole sharding thing. HBase distributes the shards (regions) over the region servers if a certain size increases. Wouldn't it be more practicabel to distribute the regions by load and not by size? So if a region gets more requests (load), it would be splittet and ditributed? (2) What happens if the ressources of one machine are not able to handle the load of the most used region (which is still splitted to the maximum) - then you need replication, so that one region is served by multiple region servers. Is this feature planed? (3) I don't understand at the moment, how restoring after crash works. On each commit, a snippet of data is written into the commit log which is distributed/replicated over all region servers? So how many servers may die whithout loosing any data? (4) How is backup of the data realised? In a perhaps MySQL environment you run the Admin console and perform a backup. Is this also available in an HBase enviroment? Thanks a lot for answering these questions :) Steve Loughran schrieb: Mork0075 wrote: Hello, can someone please explain oder point me to some documentation or papers, where i can read well proven facts, why scaling a relational db is so hard and scaling a document oriented db isnt? http://labs.google.com/papers/bigtable.html relational dbs are great for having lots of structured data, where you can run SELECT operations, do O/R mapping to make them look like objects, etc. Its one thing to back up, and you get transactions. They're bad places to store binary data, or, say, billions and billions of rows of web server log data by relaxing some of the expectations of a relational db, things like bigtable, hbase and others can scale well, but as they have relaxed the rules, may not do everything you want. So perhaps if i got lots of requests to my relational db, i would duplicate it to several servers and partition the requests. So why this doenst scale and why HBase for instance could manage this? That's called sharding/horizontal partitioning. It works well if you can partition all your data so that different users can go on different places. though once you've done that. you cant think about JOIN-ing stuff from multiple machines. The alternative option (which is apparently common in places like myspace and imdb) is to or have one r/w master and a number of read only slaves. All changes go into the master, the slaves pick the changes later I'am really new to this topic and would like to dive in deeper. check out the articles in http://highscalability.com/ -steve
Re: Distributed Lucene - from hadoop contrib
On 8/12/08, Deepika Khera [EMAIL PROTECTED] wrote: I was imagining the 2 concepts of i) using hadoop.contrib.index to index documents ii) providing search in a distributed fashion, to be all in one box. Ideally, yes. However, while it's good to use map/reduce when batch-building index, there is no consensus whether it'll be a good idea to serve index on HDFS. This is because of the poor performance of random reads in HDFS. On 8/14/08, Anoop Bhatti [EMAIL PROTECTED] wrote: I'd like to know if I'm heading down the right path, so my questions are: * Has anyone tried searching a distributed Lucene index using a method like this before? It seems too easy. Are there any gotchas that I should look out for as I scale up to more nodes and a larger index? * Do you think that going ahead with this approach, which consists of 1) creating a Lucene index using the hadoop.contrib.index code (thanks, Ning!) and 2) leaving that index in-place on hdfs and searching over it using the client code below, is a good approach? Yes, the code works on a single index shard. There is the performance concern described above. More importantly, as your index scales out, there will be multiple shards, and there are the challenges of load balance and fault tolerance, etc. * What is the status of the bailey project? It seems to be working on the same type of problem. Should I wait until that project comes out with code? There is no timeline for Bailey right now. Ning
Forcing particular nodes to perform m/r tasks?
There might be a better way around this problem, and I'm all ears (eyes?) if you see one - but I'm running in a bit of a snag... Currently, we're running in millions of lines of log data and writing the parsed output into H2 (embedded) instances distributed across all nodes local file systems. The problem that I'm running into is that we don't know for sure that the reducers will be split evenly across the nodes/instances and that's causing all sorts of deadlocks and unpredictable performance. My question then is if there is a way to force a specific split of the labor across the nodes within the cluster? i.e - say I have 3 nodes with 6 max tasks... I want to ensure that each node is getting precisely 2 m/r tasks and is writing to /local/h2_db_1 and local/h2_db_2... is this possible? Speed is terribly important, so we've abandoned mySql cluster already. Also, we can't get the r/w speed we'd like in HDFS - so local H2 instances are what we've been playing with in this latest round. -- View this message in context: http://www.nabble.com/Forcing-particular-nodes-to-perform-m-r-tasks--tp19034754p19034754.html Sent from the Hadoop core-user mailing list archive at Nabble.com.
RE: Why is scaling HBase much simpler then scaling a relational db?
Please note that you will get a prompt response about HBase questions if you ask them on the HBase user list ( [EMAIL PROTECTED] ) -Original Message- From: Mork0075 [mailto:[EMAIL PROTECTED] Sent: Sunday, August 17, 2008 11:51 PM To: core-user@hadoop.apache.org Subject: Re: Why is scaling HBase much simpler then scaling a relational db? I've read some papers and tutorials this week and now got some conrete questions: (1) Sharding is also available in common relational systems. Often it is discribed that you need an application layer for the (shards) federation. I unterstand HBase like this layer, which implements the whole sharding thing. HBase distributes the shards (regions) over the region servers if a certain size increases. Wouldn't it be more practicabel to distribute the regions by load and not by size? So if a region gets more requests (load), it would be splittet and ditributed? Bigtable uses both. First it splits row ranges based on size. It also has the ability to detect hot row ranges and will split a region if it becomes too hot. This is tricky because you don't want to have a hot range split off and then have it drop below the too hot threshold forcing a range merge. If you get into this situation, you could end up with oscillation of split - merge - split - etc. (2) What happens if the ressources of one machine are not able to handle the load of the most used region (which is still splitted to the maximum) - then you need replication, so that one region is served by multiple region servers. Is this feature planed? HBase 0.2.0 provides simple load balancing. Currently, it only balances the number of regions per server. More sophisticated balancing is planned for the future (either 0.3.0 or 0.4.0). The advanced balancing will consider the load on regions as well. Allowing multiple servers to serve the same region is only practical if the region is read-only. Otherwise you run into distributed cache coherency problems. (3) I don't understand at the moment, how restoring after crash works. On each commit, a snippet of data is written into the commit log which is distributed/replicated over all region servers? So how many servers may die whithout loosing any data? No. Each region server keeps its own redo log. When a region server dies, the master splits its log into one log for each region the server was serving, before the master reassigns the regions. When the regions are reassigned, the first thing the region server does is apply the log for that region, before announcing that the region is back on-line. (4) How is backup of the data realised? In a perhaps MySQL environment you run the Admin console and perform a backup. Is this also available in an HBase enviroment? We currently do not support backup (other than taking HBase down and copying the files to some other location). As each data block is replicated three times (the default for HDFS), the probability of losing data is low.
Re: Integrate HADOOP and Map/Reduce paradigm into HPC environment
On 8/17/08 10:56 AM, Filippo Spiga [EMAIL PROTECTED] wrote: I read the tutorial about HOD (Hadoop on demand) but HOD use torque only for initial node allocation. I would use TORQUE also for computation, allowing users to load data into HDFS, submit a TORQUE JOB that execute a Map/Reduce task and after retrive results. It's important for me that Map/Reduce tasks run only on the subset of nodes selected by TORQUE. Can someone help me? This is essentially how we use HOD. We have a HDFS that run on all nodes and then use torque to allocate mini-mapreduce clusters on top of that. To limit the amount of nodes get used for MapReduce, IIRC you need to create a queue in torque to be used for MR. Then limit the amount of nodes that queue will be allowed to allocate at one time in Maui using the classcfg stuff.
Re: Integrate HADOOP and Map/Reduce paradigm into HPC environment
2008/8/18 Allen Wittenauer [EMAIL PROTECTED] This is essentially how we use HOD. We have a HDFS that run on all nodes and then use torque to allocate mini-mapreduce clusters on top of that. To limit the amount of nodes get used for MapReduce, IIRC you need to create a queue in torque to be used for MR. Then limit the amount of nodes that queue will be allowed to allocate at one time in Maui using the classcfg stuff. Well but I haven't understand how I should configurate HOD to work in this manner. For HDFS I folllow this sequence of steps - conf/master contain only master node of my cluster - conf/slaves contain all nodes - I start HDFS using bin/start-dfs.sh Potentially I would allow to use all nodes for MapReduce. For HOD which parameter should I set in contrib/hod/conf/hodrc? Should I change only the gridservice-hdfs section? Regards -- Filippo Spiga DISCo - FISLAB - Computational Physics and Complex Systems Laboratory Rappresentante degli Studenti presso la Facoltà di Scienze Matematiche, Fisiche e Naturali Università degli Studi di Milano-Bicocca mobile: +393408387735 Skype: filippo.spiga C'e' un solo modo di dimenticare il tempo: impiegarlo. -- Baudelaire, Diari intimi
Errors when hadoop.tmp.dir is sent to multiple directories
Hi, I guess it is not a rare use case to have hadoop dfs running with multiple native OS directories, by setting them in hadoop.tmp.dir and seperating them by commas. However, mine does not work. I got error like this at some datanodes 2008-08-18 15:05:35,167 ERROR org.apache.hadoop.dfs.DataNode: Exception: java.lang.NullPointerException at org.apache.hadoop.dfs.FSDataset$FSDir.getBlockInfo(FSDataset.java:154) at org.apache.hadoop.dfs.FSDataset$FSVolume.getBlockInfo(FSDataset.java:377) at org.apache.hadoop.dfs.FSDataset$FSVolumeSet.getBlockInfo(FSDataset.java:475) at org.apache.hadoop.dfs.FSDataset.getBlockReport(FSDataset.java:830) at org.apache.hadoop.dfs.DataNode.offerService(DataNode.java:671) at org.apache.hadoop.dfs.DataNode.run(DataNode.java:2667) at java.lang.Thread.run(Thread.java:595) when I tried to upload(put) a file. But everything is right when I use only one directory at each node. Does any one know about this issue? Thank you! Best, -Kevin
error:JAVA_HOME not set
hi.. i'm new to hadoop.. i'm following the same steps given in hadoop site. in hadoop-env.sh file i've set JAVA_HOME=/usr/java/jdk1.6.0_07 The stand alone operation example is working.. mkdir input $ cp conf/*.xml input $ bin/hadoop jar hadoop-*-examples.jar grep input output 'dfs[a-z.]+' $ cat output/* But in the distributed mode after formatting new dfs using bin/hadoop namenode -format and when i start the daemons using the command: # bin/start-all.sh i get this starting namenode, logging to /etc/hadoop_install/hadoop-0.16.4/bin/../logs/hadoop-root-namenode- localhost.localdomain.out localhost: starting datanode, logging to /etc/hadoop_install/hadoop-0.16.4/bin/../logs/hadoop-root-datanode- localhost.localdomain.out localhost: Error: JAVA_HOME is not set. localhost: starting secondarynamenode, logging to /etc/hadoop_install/hadoop-0.16.4/bin/../logs/hadoop-root- secondarynamenode-localhost.localdomain.out localhost: Error: JAVA_HOME is not set. starting jobtracker, logging to /etc/hadoop_install/hadoop-0.16.4/bin/../logs/hadoop-root-jobtracker- localhost.localdomain.out localhost: starting tasktracker, logging to /etc/hadoop_install/hadoop-0.16.4/bin/../logs/hadoop-root- tasktracker-localhost.localdomain.out localhost: Error: JAVA_HOME is not set. What should i do.. Kindly help me regarding this issue.. Regards, S.chandravadana This e-mail and any files transmitted with it are for the sole use of the intended recipient(s) and may contain confidential and privileged information. If you are not the intended recipient, please contact the sender by reply e-mail and destroy all copies of the original message. Any unauthorized review, use, disclosure, dissemination, forwarding, printing or copying of this email or any action taken in reliance on this e-mail is strictly prohibited and may be unlawful.
