Big thanks to everybody :) Sorry that I am not good at English communication.But It's really good that I can receive so many kind reply here.
Thanks to Michael for your kindly reply and recommending me Lars George and Nick Dimiduk's book,I think that is <HBase, The Definitive Guide> and <HBase in Action>, Roman,I really hoping for your solution on GItHub. JM,thanks for your encouragement :) to 谢良: 多谢回复，呵呵 Best Regards. Jack Chan. From: Jean-Marc Spaggiari Date: 2013-10-23 22:46 To: Michael Segel CC: user; cdj0579 Subject: Re: How can I insert large image or video into HBase? To reply to Jack: There is no stupid question. What is stupid is to go in the wrong direction, close its eyes, and run. Asking is always good. You got a very nicely detailed respons from Michael, so I don't think I have anything to add ;) JM 2013/10/23 Michael Segel <[email protected]> Actually you didn't. Its actually a good question and JM gave you the short answer. ;-) First to clarify... depending on the size of the image or video you may want to actually store the image/vid in a sequence file. Then use HBase to index the file. (Its pretty much the same idea as what JM was saying but that you have to remember that even Vid and image files may be too large to fit into HBase yet too small to be separate files. ) This is one of the patterns that we've talked about at conferences. (Big Data TechCon) and its covered in Boris Lublinsky, Kevin Smith and Alexey Y's new book. (Its a Wiley publication.) The longer answer... In HBase your row cannot exceed the width of the region. So you end up with very large regions, with a single row and you have to cap the size of the video or image. HBase doesn't really perform well and of course performance would suck when you tried to insert or read the row. (You can go through examples in either Lars George's book or Nick Dimiduk ?sp? book on what happens during reads and writes.) Unless the row is in cache, reads and writes to HBase are slower than reading or writing to HDFS. The reason I like the question is that it forces people to think about how HBase works and why it may not be the best option as a stand alone solution. ;-) On Oct 23, 2013, at 1:38 AM, Jack Chan <[email protected]> wrote: > Hi JM, > > Thanks for your reply,It seems I really asked a stupid question :) > > > > > Jack Chan. > > From: Jean-Marc Spaggiari > Date: 2013-10-23 12:12 > To: user; cdj0579 > Subject: Re: How can I insert large image or video into HBase? > Put your file into HDFS and store only the name into HBase. HBase is not done > do store large files. > > > JM > > > > > 2013/10/23 Jack Chan <[email protected]> > > Hi All: > > This could be a stupid question.But here it goes.... > We knew that we can use "put" to insert some small files by converting it to > bytes first. > But for a large file,I think we would better stream it first. > So,how can we insert the large file into HBase through Java code using the > stream way? > > Thanks and regards > > > > Jack Chan. The opinions expressed here are mine, while they may reflect a cognitive thought, that is purely accidental. Use at your own risk. Michael Segel michael_segel (AT) hotmail.com
