Hi Michel,

I agree it doesn't make sense, but then I believe we are tracking a bug.

I don't know about speculative execution, but I certainly did not switch it on.

I am just counting the number of rows that come back in the Result.

If you are interested in this, try my Unit test. I'd be very interested to see if behaves the same for others.

http://dl.dropbox.com/u/68001072/HBaseScanCacheBug.java


Here is the output.  It shows how the number of results and key value pairs 
varies as caching in changed, and families are included.  It shows the bug 
starting with 3 families and 5000 caching.  It also shows a new bug, where the 
query always fails with an IOException with 4 families.

CacheSize FamilyCount ResultCount KeyValueCount
1000 1 10000 10
5000 1 10000 10



On 3/23/12 7:55 AM, Michel Segel wrote:
Peter, that doesnt make sense.

I mean I believe you in what you are saying, but don't see how a VPN in would 
cause this variance in results.

Do you have any speculative execution turned on?

Are you counting just the numbers of rows in the result set, or are you using 
counters in the map reduce? (I'm assuming that you are running a map/reduce, 
and not just a simple connection and single threaded scan...).

I apologize if this had already been answered, I hadn't been following this too 
closely.

Sent from a remote device. Please excuse any typos...

Mike Segel

On Mar 22, 2012, at 8:01 PM, Peter Wolf<[email protected]>  wrote:

Hello again Lars and Lars,

Here is some additional information that may help you track this down.

I think this behavior has something to do with my VPN.  My servers are on the 
Amazon Cloud and I normally run my client on my laptop via a VPN (Tunnelblick: 
OS X 10.7.3; Tunnelblick 3.2.3 (build 2891.2932)).  This is where I see the 
buggy behavior I describe.

However, when my Client is running on an EC2 machine, then I get different 
behavior.  I can not prove that it is always correct, but in at least one case 
my current code does not work on my laptop, but gets the correct number of 
results on an EC2 machine.  Note that my scans are also much faster on the EC2 
machine.

I will do more tests to see if I can localize it further.

Hope this helps
Thank you again
Peter


On 3/19/12 2:24 PM, Peter Wolf wrote:
Hello Lars and Lars,

Thank you for you help and attention.

I wrote a standalone test that exhibits the bug.

http://dl.dropbox.com/u/68001072/HBaseScanCacheBug.java

Here is the output.  It shows how the number of results and key value pairs 
varies as caching in changed, and families are included.  It shows the bug 
starting with 3 families and 5000 caching.  It also shows a new bug, where the 
query always fails with an IOException with 4 families.

CacheSize FamilyCount ResultCount KeyValueCount
1000 1 10000 10
5000 1 10000 10

Reply via email to