Re: readAtOffset error when reading from HFiles

2015-05-27 Thread Nick Dimiduk
s3:// and s3n:// are different things. Given the structure of your S3 key, I think you should be using s3n. For reference, http://wiki.apache.org/hadoop/AmazonS3

Re: readAtOffset error when reading from HFiles

2015-05-21 Thread Ted Yu
I tried to access http://pastebin.com/SJci7uQM (from your first email) but looks like the file has been removed. Can you post the error on pastebin again ? Cheers On Wed, May 20, 2015 at 9:38 PM, donmai dood...@gmail.com wrote: Hi Ted - would you happen to have any insight as to what issues

Re: readAtOffset error when reading from HFiles

2015-05-20 Thread donmai
Hi Ted - would you happen to have any insight as to what issues might cause a positional read failure for an HFile? Running the HFile tool seems to have normal output. On Wed, May 13, 2015 at 12:40 PM, donmai dood...@gmail.com wrote: Ahhh, thanks for that. Yep, all flushes will (or should be)

Re: readAtOffset error when reading from HFiles

2015-05-13 Thread donmai
Ahhh, thanks for that. Yep, all flushes will (or should be) going to S3. I'm working through it and it seems that it's defaulting to the positional read instead of seek+read - is this accurate? On Tue, May 12, 2015 at 12:00 PM, Ted Yu yuzhih...@gmail.com wrote: -w is shorthand for --seekToRow

readAtOffset error when reading from HFiles

2015-05-12 Thread donmai
Hi, I'm getting this error when trying to read from HFiles: http://pastebin.com/SJci7uQM Any idea what's going on here? Thanks!

Re: readAtOffset error when reading from HFiles

2015-05-12 Thread donmai
Thanks for the help! I am getting 'okay' for everything. I am using S3 for the store, do you think it could be an issue with how S3 is reading the files? On Tue, May 12, 2015 at 9:45 AM, Talat Uyarer ta...@uyarer.com wrote: Hi Donmai, Seems to be something wrong with this file. Can you try to

Re: readAtOffset error when reading from HFiles

2015-05-12 Thread Ted Yu
What release of hbase are you using ? Please read http://hbase.apache.org/book.html#hfile where you can find description about HFile tool. This tool would allow you to investigate given HFile. Cheers On Tue, May 12, 2015 at 6:02 AM, donmai dood...@gmail.com wrote: Hi, I'm getting this error

Re: readAtOffset error when reading from HFiles

2015-05-12 Thread Talat Uyarer
Hi Donmai, Seems to be something wrong with this file. Can you try to reach the HFile ? Try to check cluster with hbck tool (http://hbase.apache.org/book.html#hbck), use -details flag to see the errors. 2015-05-12 16:02 GMT+03:00 donmai dood...@gmail.com: Hi, I'm getting this error when

Re: readAtOffset error when reading from HFiles

2015-05-12 Thread donmai
Thanks Ted, this actually helped me out alot! I'm running 0.98.6 and was able to determine that the HFiles are perfectly okay and can be scanned through without issue - it looks like there's something else going on, since after a compaction everything works... On Tue, May 12, 2015 at 9:55 AM, Ted

Re: readAtOffset error when reading from HFiles

2015-05-12 Thread donmai
Actually, looking deeper into it, things don't seem to be making sense. The error message is this: Caused by: java.io.IOException: Positional read of 65723 bytes failed at offset 394218 (returned 16384) As such, I try to do a read for 65723 bytes using the tool to see if it fails at that offset: