James,
thanks for the details, an interesting case.
It looks to me the OutOfOrderScannerNextException thrown by RS is correct
-- note the internal cursor in scanner has moved on, even if the client
side retries, client will not get the correct data(will lose data that the
scanner already scanned)
It seems each region is a split in current TableInputFormat. We have large
regions and it's suboptimal.
Is there a TableInputFormat implementation that supports multiple splits
for each region?
Thanks,
--
Jianshi Huang
LinkedIn: jianshi
Twitter: @jshuang
Github Blog:
I don't think we have it currently.
On Wed, Oct 15, 2014 at 6:21 PM, Jianshi Huang jianshi.hu...@gmail.com
wrote:
It seems each region is a split in current TableInputFormat. We have large
regions and it's suboptimal.
Is there a TableInputFormat implementation that supports multiple splits
+user@hbase
2014-10-15 20:48 GMT+08:00 Fengyun RAO raofeng...@gmail.com:
We use Spark 1.1, and HBase 0.98.1-cdh5.1.0, and need to read and write an
HBase table in Spark program.
I notice there are:
spark.driver.extraClassPath
spark.executor.extraClassPathproperties to manage extra
I am writing to HBase, following are my options:
export SPARK_CLASSPATH=/opt/cloudera/parcels/CDH/lib/hbase/hbase-protocol.jar
spark-submit \
--jars
Done. It's HBASE-12270
Please let me know if you have any comments/questions.
Date: Mon, 13 Oct 2014 22:42:37 -0500
Subject: RE: HBase read performance
From: bus...@cloudera.com
To: user@hbase.apache.org
Yes, please file a JIRA and attach the test
Hi all,
I'm trying to get a custom filter to work on HBase 0.96. After some
searching, I found that starting from 0.96, the implementer is required to
implement toByteArray and parseFrom methods, using Protocol Buffers.
But I'm having trouble with the how.
The proto file for the existing filters
Hi Ted ,
Since I am also working on similar thing is there a way we can first test
the filter on client side?.You know what I mean without disrupting others
who are using the same cluster for other work?
Thanks,
Nishanth
On Wed, Oct 15, 2014 at 3:17 PM, Ted Yu yuzhih...@gmail.com wrote:
bq.
Nishanth:
Good question.
As a general coding guide, writing unit test is always a good start. Using
Matt's case as an example, take a look at TestPrefixFilter.
There're various unit tests for Filters in hbase code.
Cheers
On Wed, Oct 15, 2014 at 2:30 PM, Nishanth S nishanth.2...@gmail.com
Ted, sorry, I have no idea what you mean. Could you please elaborate?
On Wednesday, October 15, 2014, Ted Yu yuzhih...@gmail.com wrote:
Nishanth:
Good question.
As a general coding guide, writing unit test is always a good start. Using
Matt's case as an example, take a look at
Matt:
I was answering Nishanth's question - testing custom filter on client side.
Cheers
On Wed, Oct 15, 2014 at 2:54 PM, Matt K matvey1...@gmail.com wrote:
Ted, sorry, I have no idea what you mean. Could you please elaborate?
On Wednesday, October 15, 2014, Ted Yu yuzhih...@gmail.com wrote:
Thanks Ted .I will take a look.
-Nishanth
On Wed, Oct 15, 2014 at 3:43 PM, Ted Yu yuzhih...@gmail.com wrote:
Nishanth:
Good question.
As a general coding guide, writing unit test is always a good start. Using
Matt's case as an example, take a look at TestPrefixFilter.
There're various
Hi,
I have a hbase table( say test1) with 3 cfs ( a,b,c) and i have bunch of
cqs in each of these cf.
I also have one more table ( say test2) with some cfs on same cluster.
My requirement is to move data from test1 to test2 through my custom logic
( as in which cf:cq from test1 will go to which
13 matches
Mail list logo
