date:20121120


[ 
https://issues.apache.org/jira/browse/HBASE-7191?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13501214#comment-13501214
 ] 

Oleg Zhurakousky commented on HBASE-7191:
-

Actually, this affects other versions as well
{code}
HBase 0.94.1
{code}
Further more; FSUtils spits out a misleading message
{code}
 String msg = File system needs to be upgraded.
  +   You have version  + version
  +  and I want version  + HConstants.FILE_SYSTEM_VERSION
  + .  Run the '${HBASE_HOME}/bin/hbase migrate' script.;
{code}
As far as I can tell 'migrate' script is no longer available

One would think 'hbase hbck -fixVersionFile' would do the trick, but that 
requires a running instance of HBase which we can't start unless those two 
files are in tact 

The current workaround is to temporarily change the value of 'hbase.rootdir' - 
start/stop hbase (this will re-create the two files) - copy those files to the 
original 'hbase.rootdir' and change 'hbase.rootdir' to point back to the 
original one. However, this is obviously a hack. May what really needs to 
happen is to have and offline option of 'hbase hbck -fixVersionFile'.


 HBCK - Add offline create/fix hbase.version and hbase.id 
 -

 Key: HBASE-7191
 URL: https://issues.apache.org/jira/browse/HBASE-7191
 Project: HBase
  Issue Type: Improvement
  Components: hbck
Affects Versions: 0.96.0
Reporter: Enis Soztutar

 One of our clients run into a problem, in which they have the hbase.root.dir, 
 and cluster data, but their hbase.id and hbase.version files are corrupted. 
 HMaster creates those on start, but not if there is already existing data.
 We can add smt like --fixIdFile, and ability for HBCK to do some offline 
 repairs for the version file. 

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Created] (HBASE-7196) 'hbase shell status' throws exception when HBase is not running

Oleg Zhurakousky created HBASE-7196:
---

 Summary: 'hbase shell status' throws exception when HBase is not 
running
 Key: HBASE-7196
 URL: https://issues.apache.org/jira/browse/HBASE-7196
 Project: HBase
  Issue Type: Bug
Affects Versions: 0.94.1
Reporter: Oleg Zhurakousky


Its kind of a nuisance bug. One would assume that 'status' command should 
simply  return something along the lines of HBase is not running

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-7146) Fix the wrong reference to getReader survived in theFSHLog javadoc

2012-11-20 Thread nkeywal (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-7146?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13501223#comment-13501223
 ] 

nkeywal commented on HBASE-7146:


Hi Gustavo,

I think the new doc is ok. I will commit this if no ones object.

Thanks;

Nicolas

 Fix the wrong reference to getReader survived in theFSHLog javadoc
 --

 Key: HBASE-7146
 URL: https://issues.apache.org/jira/browse/HBASE-7146
 Project: HBase
  Issue Type: Sub-task
  Components: documentation
Affects Versions: 0.96.0
Reporter: nkeywal
Priority: Trivial
 Attachments: HBASE-7146.patch


  In FSHLog, a wrong reference to getReader survived in the javadoc:
 {noformat}
  * To read an HLog, call {@link #getReader(org.apache.hadoop.fs.FileSystem,
  * org.apache.hadoop.fs.Path, org.apache.hadoop.conf.Configuration)}.
 {noformat}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-7146) Fix the wrong reference to getReader survived in theFSHLog javadoc

2012-11-20 Thread nkeywal (JIRA)


 [ 
https://issues.apache.org/jira/browse/HBASE-7146?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

nkeywal updated HBASE-7146:
---

Assignee: Gustavo Anatoly

 Fix the wrong reference to getReader survived in theFSHLog javadoc
 --

 Key: HBASE-7146
 URL: https://issues.apache.org/jira/browse/HBASE-7146
 Project: HBase
  Issue Type: Sub-task
  Components: documentation
Affects Versions: 0.96.0
Reporter: nkeywal
Assignee: Gustavo Anatoly
Priority: Trivial
 Attachments: HBASE-7146.patch


  In FSHLog, a wrong reference to getReader survived in the javadoc:
 {noformat}
  * To read an HLog, call {@link #getReader(org.apache.hadoop.fs.FileSystem,
  * org.apache.hadoop.fs.Path, org.apache.hadoop.conf.Configuration)}.
 {noformat}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-7196) 'hbase shell status' throws exception when HBase is not running


[ 
https://issues.apache.org/jira/browse/HBASE-7196?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13501230#comment-13501230
 ] 

Oleg Zhurakousky commented on HBASE-7196:
-

Also
{code}
HBaseAdmin.isMasterRunning()
{code}
is throwing an exception if Master is not running. By definition boolean 
methods should never throw an exception, don't you agree?

 'hbase shell status' throws exception when HBase is not running
 ---

 Key: HBASE-7196
 URL: https://issues.apache.org/jira/browse/HBASE-7196
 Project: HBase
  Issue Type: Bug
Affects Versions: 0.94.1
Reporter: Oleg Zhurakousky

 Its kind of a nuisance bug. One would assume that 'status' command should 
 simply  return something along the lines of HBase is not running

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-7196) 'hbase shell status' throws exception when HBase is not running


[ 
https://issues.apache.org/jira/browse/HBASE-7196?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13501257#comment-13501257
 ] 

Ted Yu commented on HBASE-7196:
---

Take a look at this method in FileSystem from hadoop:
{code}
  public abstract boolean delete(Path f) throws IOException;
{code}

 'hbase shell status' throws exception when HBase is not running
 ---

 Key: HBASE-7196
 URL: https://issues.apache.org/jira/browse/HBASE-7196
 Project: HBase
  Issue Type: Bug
Affects Versions: 0.94.1
Reporter: Oleg Zhurakousky

 Its kind of a nuisance bug. One would assume that 'status' command should 
 simply  return something along the lines of HBase is not running

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-6580) New HTable pool, based on HBase(byte[], HConnection, ExecutorService) constructor

2012-11-20 Thread Lars Hofhansl (JIRA)

[
https://issues.apache.org/jira/browse/HBASE-6580?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13501302#comment-13501302
]

Lars Hofhansl commented on HBASE-6580:
--

Not really. After HBASE-4805 you'd create a single HConnection (via
HConnectionManager.createConnection(...)) and then use this for each HTable
created afterwards. HTable creation is very cheap when both HConnection and
ExecutorService are passed... Cheaper indeed than then the HTablePool protocol.

Maybe HConnection should just finally get a getTable(...) method.

New HTable pool, based on HBase(byte[], HConnection, ExecutorService)
constructor
-

Key: HBASE-6580
URL: https://issues.apache.org/jira/browse/HBASE-6580
Project: HBase
Issue Type: Improvement
Affects Versions: 0.92.2, 0.94.2
Reporter: Lars Hofhansl
Priority: Minor
Attachments: HBASE-6580_v1.patch

Here I propose a very simple TablePool.
It could be called LightHTablePool (or something - if you have a better name).
Internally it would maintain an HConnection and an Executor service and each
invocation of getTable(...) would create a new HTable and close() would just
close it.
In testing I find this more light weight than HTablePool and easier to
monitor in terms of resources used.
It would hardly be more than a few dozen lines of code.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-7146) Fix the wrong reference to getReader survived in theFSHLog javadoc

2012-11-20 Thread Gustavo Anatoly (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-7146?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13501308#comment-13501308
 ] 

Gustavo Anatoly commented on HBASE-7146:


Hi, Nicolas.

It's ok, to me. I don't have objections.

Thanks for your review.

 Fix the wrong reference to getReader survived in theFSHLog javadoc
 --

 Key: HBASE-7146
 URL: https://issues.apache.org/jira/browse/HBASE-7146
 Project: HBase
  Issue Type: Sub-task
  Components: documentation
Affects Versions: 0.96.0
Reporter: nkeywal
Assignee: Gustavo Anatoly
Priority: Trivial
 Attachments: HBASE-7146.patch


  In FSHLog, a wrong reference to getReader survived in the javadoc:
 {noformat}
  * To read an HLog, call {@link #getReader(org.apache.hadoop.fs.FileSystem,
  * org.apache.hadoop.fs.Path, org.apache.hadoop.conf.Configuration)}.
 {noformat}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-7196) 'hbase shell status' throws exception when HBase is not running


[ 
https://issues.apache.org/jira/browse/HBASE-7196?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13501313#comment-13501313
 ] 

Oleg Zhurakousky commented on HBASE-7196:
-

Different question.
Is there a reason why HBaseAdmin initializes itself in the constructor? It 
makes the above problem even more counter intuitive:
The javadoc for the constructor says:
{code}
@throws MasterNotRunningException if the master is not running
{code}
which means the _isMasterRunning()_ method is meaningless since you can never 
call it unless you have a valid instance of HBaseAdmin, but you'll never have 
one unless master is running.

Wouldn't it be better to always create HBaseAdmin and perform check for running 
master when necessary (of course without exception being thrown by the 
isMasterRunning() method)

 'hbase shell status' throws exception when HBase is not running
 ---

 Key: HBASE-7196
 URL: https://issues.apache.org/jira/browse/HBASE-7196
 Project: HBase
  Issue Type: Bug
Affects Versions: 0.94.1
Reporter: Oleg Zhurakousky

 Its kind of a nuisance bug. One would assume that 'status' command should 
 simply  return something along the lines of HBase is not running

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-7196) 'hbase shell status' throws exception when HBase is not running


[ 
https://issues.apache.org/jira/browse/HBASE-7196?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13501322#comment-13501322
 ] 

Oleg Zhurakousky commented on HBASE-7196:
-

[~yuzhih...@gmail.com]
That is different. Delete may or may not succeed, while during the deletion the 
IOException may be thrown.
In the above case we have a method that by its name and thus definition suppose 
to return 'true' if Master is running and 'false' if Master is not running. And 
it doesn't do that. And as described above it its meaningless since it can 
never be called unless Master is running which is the prerequisite for creating 
HBaseAdmin instance. But once the instance is created its already implied that 
Master is running. Don't you agree?

 'hbase shell status' throws exception when HBase is not running
 ---

 Key: HBASE-7196
 URL: https://issues.apache.org/jira/browse/HBASE-7196
 Project: HBase
  Issue Type: Bug
Affects Versions: 0.94.1
Reporter: Oleg Zhurakousky

 Its kind of a nuisance bug. One would assume that 'status' command should 
 simply  return something along the lines of HBase is not running

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-7191) HBCK - Add offline create/fix hbase.version and hbase.id

2012-11-20 Thread Jimmy Xiang (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-7191?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13501323#comment-13501323
 ] 

Jimmy Xiang commented on HBASE-7191:


We can add -fixVersionFile to the offline meta repair tool.

 HBCK - Add offline create/fix hbase.version and hbase.id 
 -

 Key: HBASE-7191
 URL: https://issues.apache.org/jira/browse/HBASE-7191
 Project: HBase
  Issue Type: Improvement
  Components: hbck
Affects Versions: 0.96.0
Reporter: Enis Soztutar

 One of our clients run into a problem, in which they have the hbase.root.dir, 
 and cluster data, but their hbase.id and hbase.version files are corrupted. 
 HMaster creates those on start, but not if there is already existing data.
 We can add smt like --fixIdFile, and ability for HBCK to do some offline 
 repairs for the version file. 

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-7196) 'hbase shell status' throws exception when HBase is not running


[ 
https://issues.apache.org/jira/browse/HBASE-7196?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13501327#comment-13501327
 ] 

Ted Yu commented on HBASE-7196:
---

I think the purpose of isMasterRunning() is for checking whether master is 
running after HBaseAdmin instance is successfully obtained.

 'hbase shell status' throws exception when HBase is not running
 ---

 Key: HBASE-7196
 URL: https://issues.apache.org/jira/browse/HBASE-7196
 Project: HBase
  Issue Type: Bug
Affects Versions: 0.94.1
Reporter: Oleg Zhurakousky

 Its kind of a nuisance bug. One would assume that 'status' command should 
 simply  return something along the lines of HBase is not running

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-7196) 'hbase shell status' throws exception when HBase is not running


[ 
https://issues.apache.org/jira/browse/HBASE-7196?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13501335#comment-13501335
 ] 

Oleg Zhurakousky commented on HBASE-7196:
-

It would be great if that was the case, but unfortunately it is not and that is 
what I've been trying to explain.
Based on the current code HBaseAdmin instance can NOT be created unless master 
is running. 

 'hbase shell status' throws exception when HBase is not running
 ---

 Key: HBASE-7196
 URL: https://issues.apache.org/jira/browse/HBASE-7196
 Project: HBase
  Issue Type: Bug
Affects Versions: 0.94.1
Reporter: Oleg Zhurakousky

 Its kind of a nuisance bug. One would assume that 'status' command should 
 simply  return something along the lines of HBase is not running

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-6580) New HTable pool, based on HBase(byte[], HConnection, ExecutorService) constructor

2012-11-20 Thread Gary Helmling (JIRA)

[
https://issues.apache.org/jira/browse/HBASE-6580?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13501343#comment-13501343
]

Gary Helmling commented on HBASE-6580:
--

bq. Maybe HConnection should just finally get a getTable(...) method.

+1. Using HConnection to get an HTable seems more intuitive and more
composable than our old monolithic HTable. Lightweight HTables that are easily
GC'd would also be very nice. Agree that at that point we don't really need
HTablePool.

New HTable pool, based on HBase(byte[], HConnection, ExecutorService)
constructor
-

[jira] [Updated] (HBASE-7197) Add multi get to RemoteHTable


 [ 
https://issues.apache.org/jira/browse/HBASE-7197?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Elliott Clark updated HBASE-7197:
-

  Component/s: REST
   Client
Affects Version/s: 0.96.0
   0.94.2

 Add multi get to RemoteHTable
 -

 Key: HBASE-7197
 URL: https://issues.apache.org/jira/browse/HBASE-7197
 Project: HBase
  Issue Type: Improvement
  Components: Client, REST
Affects Versions: 0.94.2, 0.96.0
Reporter: Elliott Clark
Priority: Minor

 REST server supports multi-get, so the RemoteHTable class should as well.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Created] (HBASE-7197) Add multi get to RemoteHTable

Elliott Clark created HBASE-7197:


 Summary: Add multi get to RemoteHTable
 Key: HBASE-7197
 URL: https://issues.apache.org/jira/browse/HBASE-7197
 Project: HBase
  Issue Type: Improvement
Reporter: Elliott Clark
Priority: Minor


REST server supports multi-get, so the RemoteHTable class should as well.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-6423) Writes should not block reads on blocking updates to memstores

[
https://issues.apache.org/jira/browse/HBASE-6423?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13501354#comment-13501354
]

stack commented on HBASE-6423:
--

Needs release note of new config.

You think Long.MAX_VALUE as default? In trunk why not have it a reasonable
value and in 0.94 have your MAX_VALUE?

Should default be socket timeout?

What is this comment about?

+// block waiting for the lock for closing
lock.writeLock().lock();

No busy waiting when closing?

I see the comment in a few places.

Removing getRecentFlushInfo is ok because HRegion is annotated private?

This is a little ugly:

+ int multiplier = Math.min(acquiredLocks.size(), maxBusyWaitMultiplier);
+ lock(this.updatesLock.readLock(), multiplier);

You pass in multiplier. Why not pass in calculated time to busy wait? Have a
little function that consults multiplier and
does Math.min, etc., and figures how long to wait?

+ int multiplier = Math.min(acquiredLocks.size(), maxBusyWaitMultiplier);

For example, put the above into the lock method? Pass in size?

Fix your capialization of the 'N' in + * @throws RegioNTooBusyException if
failed to get the lock in time

Can we backport?

So write locks are unbounded, its just the reads that we'll give up on. How
does this address Karthik's original comment in the issue?
Reads fail faster releasing handlers so we can read/write against other
memstores other than the one that has a write lock on it (and is
currently at its limit)?

Good patch Jimmy

Writes should not block reads on blocking updates to memstores
--

Key: HBASE-6423
URL: https://issues.apache.org/jira/browse/HBASE-6423
Project: HBase
Issue Type: Bug
Reporter: Karthik Ranganathan
Assignee: Jimmy Xiang
Attachments: trunk-6423.patch, trunk-6423_v2.1.patch,
trunk-6423_v2.patch

We have a big data use case where we turn off WAL and have a ton of reads and
writes. We found that:
1. flushing a memstore takes a while (GZIP compression)
2. incoming writes cause the new memstore to grow in an unbounded fashion
3. this triggers blocking memstore updates
4. in turn, this causes all the RPC handler threads to block on writes to
that memstore
5. we are not able to read during this time as RPC handlers are blocked
At a higher level, we should not hold up the RPC threads while blocking
updates, and we should build in some sort of rate control.

[jira] [Commented] (HBASE-7180) RegionScannerImpl.next() is inefficient.

[
https://issues.apache.org/jira/browse/HBASE-7180?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13501355#comment-13501355
]

stack commented on HBASE-7180:
--

getReadPt has to be public?

nextInternal has to be public too?

closeRegionOperation and startRegionOperation too?

So CPs can get at them?

You add KeyValue to RegionScanner when it did not need it previous. You have
to?

And a nextInternal in an Interface like RegionScanner seems wrong?

I am for speedup but as you say, this one is ugly

RegionScannerImpl.next() is inefficient.

Key: HBASE-7180
URL: https://issues.apache.org/jira/browse/HBASE-7180
Project: HBase
Issue Type: Bug
Reporter: Lars Hofhansl
Attachments: 7180-0.94-SKETCH.txt, 7180-0.94-v1.txt

We just came across a special scenario.
For our Phoenix project (SQL runtime for HBase), we push a lot of work into
HBase via coprocessors. One method is to wrap RegionScanner in coprocessor
hooks and then do processing in the hook to avoid returning a lot of data to
the client unnecessarily.
In this specific case this is pretty bad. Since the wrapped RegionScanner's
next() does not know that it is called this way is still does all of this
on each invocation:
# Starts a RegionOperation
# Increments the request count
# set the current read point on a thread local (because generally each call
could come from a different thread)
# Finally does the next on its StoreScanner(s)
# Ends the RegionOperation
When this is done in a tight loop millions of times (as is the case for us)
it starts to become significant.
Not sure what to do about this, really. Opening this issue for discussion.
One way is to extend the RegionScanner with an internal next() method of
sorts, so that all this overhead can be avoided. The coprocessor could call
the regular next() methods once and then just call the cheaper internal
version.
Are there better/cleaner ways?

[jira] [Commented] (HBASE-7185) Create hbase-rpc module


[ 
https://issues.apache.org/jira/browse/HBASE-7185?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13501356#comment-13501356
 ] 

stack commented on HBASE-7185:
--

Highlevel, this is rpc module so why the pb stuff in here?  Its scope is messy 
when it has stuff like ClusterIdProtos and AggregateProtos and zk protos?
Did you have to pull in stuff to make things like AggregateProtos happy?

Is it odd having RegionServerProto in here, in this module that hbase-server, 
another module downstream on the dependency river, depends on?

Any particular reason you need it beyond what is listed in issue?

It does not move the content of the rpc package?  Just the pbs?


So the below makes it so I can skip unit tests in rpc?

idskip-rpc-tests/id

Its a profile?  -Pskip-rpc-tests  Or I do it w/ -Dskip-rpc-tests=true

Should be in release notes how to do above.

If above is correct, why doesn't skipTests work for this module?  It does but 
you just want flag for skipping rpcs?


 Create hbase-rpc module
 ---

 Key: HBASE-7185
 URL: https://issues.apache.org/jira/browse/HBASE-7185
 Project: HBase
  Issue Type: Sub-task
Reporter: Elliott Clark
Assignee: Elliott Clark
 Fix For: 0.96.0

 Attachments: HBASE-7185-0.patch, HBASE-7185-1.patch


 Create an hbase-rpc module where protobuf's that can cross the wire(either to 
 zk or from a server) will be stored.  This should allow others to create 
 clients without relying on too much of the hbase code base.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-2231) Compaction events should be written to HLog


[ 
https://issues.apache.org/jira/browse/HBASE-2231?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13501364#comment-13501364
 ] 

stack commented on HBASE-2231:
--

So, test passes because we are able to read from deleted file because dfsclient 
has block locations cached.  I suppose this makes sense it is just a little 
unexpected (It took me a while to figure it).

Not sure how test failed in the past.  My guess is that in the past we were 
strict when you went to compact that all referenced files had to be present.  
That is no longer the case in that we seem to be loosey-goosey about it to 
handle the case where a compaction may have run before ours.  Let me look into 
this more.

 Compaction events should be written to HLog
 ---

 Key: HBASE-2231
 URL: https://issues.apache.org/jira/browse/HBASE-2231
 Project: HBase
  Issue Type: Improvement
  Components: regionserver
Reporter: Todd Lipcon
Assignee: Todd Lipcon
Priority: Blocker
  Labels: moved_from_0_20_5
 Fix For: 0.96.0

 Attachments: 2231-testcase-0.94.txt, 2231-testcase_v2.txt, 
 2231v2.txt, hbase-2231-testcase.txt, hbase-2231.txt


 The sequence for a compaction should look like this:
 # Compact region to new files
 # Write a Compacted Region entry to the HLog
 # Delete old files
 This deals with a case where the RS has paused between step 1 and 2 and the 
 regions have since been reassigned.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-7185) Create hbase-rpc module

[
https://issues.apache.org/jira/browse/HBASE-7185?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13501365#comment-13501365
]

Elliott Clark commented on HBASE-7185:
--

Talked with stack and I think we both agree that rpc is probably a bad name for
this module. Thinking hbase-protocol would be better. It would convey more
that this module contains only the code of protobuf's that are pushed across
the wire to ZK and to/from the servers. That's also why the ipc namespace is
not moved into this module. The ipc module it much more tied to our client and
server implementations. I wanted to make it possible for an external client to
use this module.

The skipping stuff is copied from the rest of the modules. -Dskip-server-tests
will skip the server tests, -Dskip-common-tests will skip common etc.

Create hbase-rpc module
---

Key: HBASE-7185
URL: https://issues.apache.org/jira/browse/HBASE-7185
Project: HBase
Issue Type: Sub-task
Reporter: Elliott Clark
Assignee: Elliott Clark
Fix For: 0.96.0

Attachments: HBASE-7185-0.patch, HBASE-7185-1.patch

Create an hbase-rpc module where protobuf's that can cross the wire(either to
zk or from a server) will be stored. This should allow others to create
clients without relying on too much of the hbase code base.

[jira] [Commented] (HBASE-6777) Snapshot Restore interface


[ 
https://issues.apache.org/jira/browse/HBASE-6777?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13501371#comment-13501371
 ] 

Jesse Yates commented on HBASE-6777:


Looks like this is a little behind the current branch; I guess with the changes 
to HMaster from offline, this got a little off. I took the opportunity to 
rebase the snapshot branch onto the current trunk (its pushed up now).

Any chance you can rebase this patch Matteo (and maybe HBASE-6230 on this as 
well)?

 Snapshot Restore interface
 --

 Key: HBASE-6777
 URL: https://issues.apache.org/jira/browse/HBASE-6777
 Project: HBase
  Issue Type: Sub-task
  Components: Client, master, snapshots
Affects Versions: 0.96.0
Reporter: Matteo Bertozzi
Assignee: Matteo Bertozzi
 Fix For: hbase-6055

 Attachments: hbase-6777-v9.patch, HBASE--v1.patch


 Add interfaces for restoring a snapshot

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-7009) Port HBaseCluster interface/tests to 0.94


[ 
https://issues.apache.org/jira/browse/HBASE-7009?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13501373#comment-13501373
 ] 

Sergey Shelukhin commented on HBASE-7009:
-

I am going to remove the backport of 7109 from the patch (it was started here 
but review in 7109 resulted in changes, so it's easier to commit them 
separately, 7109 in 7109 and 6241 here).

 Port HBaseCluster interface/tests to 0.94
 -

 Key: HBASE-7009
 URL: https://issues.apache.org/jira/browse/HBASE-7009
 Project: HBase
  Issue Type: Sub-task
  Components: test
Affects Versions: 0.94.3
Reporter: Sergey Shelukhin
Assignee: Sergey Shelukhin
 Fix For: 0.94.4

 Attachments: HBASE-7009-p1.patch, HBASE-7009.patch, 
 HBASE-7009-v2-squashed.patch, HBASE-7009-v3-squashed.patch


 Need to port. I am porting V5 patch from the original JIRA; I have a 
 partially ported (V3) patch from Enis with protocol buffers being reverted to 
 HRegionInterface/HMasterInterface

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-7180) RegionScannerImpl.next() is inefficient.

2012-11-20 Thread Lars Hofhansl (JIRA)

[
https://issues.apache.org/jira/browse/HBASE-7180?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13501375#comment-13501375
]

Lars Hofhansl commented on HBASE-7180:
--

Yeah I don't like it either. We somehow need to expose more internals to
coprocessors in a clean way.

* KeyValue already is needed for RegionScanner, since it extends internal
scanner.
* start/closeRegionOperation should be available to coprocessors anyway (I
think). Otherwise it is hard to implement these types of things in coprocessors.
* I mainly do not like nextInternal on the interface. Is there a better way to
expose the inner workings of RegionScannerImpl to avoid expensive setup at each
iteration?

Another option is to keep the RegionScanner interface as it, and just make
these methods public in RegionScannerImpl. A coprocessor can then cast the
RegionScanner to RegionScannerImpl and access the stuff it needs.

RegionScannerImpl.next() is inefficient.

Key: HBASE-7180
URL: https://issues.apache.org/jira/browse/HBASE-7180
Project: HBase
Issue Type: Bug
Reporter: Lars Hofhansl
Attachments: 7180-0.94-SKETCH.txt, 7180-0.94-v1.txt

[jira] [Commented] (HBASE-6055) Snapshots in HBase 0.96

2012-11-20 Thread Jonathan Hsieh (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-6055?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13501387#comment-13501387
 ] 

Jonathan Hsieh commented on HBASE-6055:
---

Here's a repo with the online branch merged to the snapshot branch with 
snapshot unit tests passing now.  

https://github.com/jmhsieh/hbase/tree/snapshots-online-merge
hash  d1299347c0c1afcc0264b14ee12beee170efc4c2

mvn clean
mvn test -Dtest=errorhandling/* -PlocalTests
mvn test -Dtest=Test*Procedure* -PlocalTests
mvn test -Dtest=Test*Snapshot*,snapshot/*,TestFSUtils -PlocalTests

It needs some cleanup (cleanup duplicate/commented code from merge) but patches 
of pieces will be coming out later today / tommorrow.

Likely pieces:  ( might break a few more of these down if they are excessive)
* External Exceptions + snapshot manager refactor
* Barrier Procedure
* online Timestamp snapshots tasks
* online timestampe snapshots + snapshot manager


 Snapshots in HBase 0.96
 ---

 Key: HBASE-6055
 URL: https://issues.apache.org/jira/browse/HBASE-6055
 Project: HBase
  Issue Type: New Feature
  Components: Client, master, regionserver, snapshots, Zookeeper
Reporter: Jesse Yates
Assignee: Jesse Yates
 Fix For: hbase-6055, 0.96.0

 Attachments: Snapshots in HBase.docx


 Continuation of HBASE-50 for the current trunk. Since the implementation has 
 drastically changed, opening as a new ticket.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-6055) Snapshots in HBase 0.96

2012-11-20 Thread Micah Whitacre (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-6055?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13501391#comment-13501391
 ] 

Micah Whitacre commented on HBASE-6055:
---

I will be out of the office on November 19 - 25, 2012 with limited access to 
email.  If this is a support issue please use the appropriate on-call 
procedures.[1] If this is an emergency please contact Greg Whitsitt.

[1] - http://wiki.cloud.cerner.corp/index.php/Big_Data/On_Call_Support

CONFIDENTIALITY NOTICE This message and any included attachments are from 
Cerner Corporation and are intended only for the addressee. The information 
contained in this message is confidential and may constitute inside or 
non-public information under international, federal, or state securities laws. 
Unauthorized forwarding, printing, copying, distribution, or use of such 
information is strictly prohibited and may be unlawful. If you are not the 
addressee, please promptly delete this message and notify the sender of the 
delivery error by e-mail or you may call Cerner's corporate offices in Kansas 
City, Missouri, U.S.A at (+1) (816)221-1024.


 Snapshots in HBase 0.96
 ---

 Key: HBASE-6055
 URL: https://issues.apache.org/jira/browse/HBASE-6055
 Project: HBase
  Issue Type: New Feature
  Components: Client, master, regionserver, snapshots, Zookeeper
Reporter: Jesse Yates
Assignee: Jesse Yates
 Fix For: hbase-6055, 0.96.0

 Attachments: Snapshots in HBase.docx


 Continuation of HBASE-50 for the current trunk. Since the implementation has 
 drastically changed, opening as a new ticket.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-6055) Snapshots in HBase 0.96

2012-11-20 Thread Andrew Purtell (JIRA)

[
https://issues.apache.org/jira/browse/HBASE-6055?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Andrew Purtell updated HBASE-6055:
--

Comment: was deleted

(was: I will be out of the office on November 19 - 25, 2012 with limited access
to email. If this is a support issue please use the appropriate on-call
procedures.[1] If this is an emergency please contact Greg Whitsitt.

[1] - http://wiki.cloud.cerner.corp/index.php/Big_Data/On_Call_Support

CONFIDENTIALITY NOTICE This message and any included attachments are from
Cerner Corporation and are intended only for the addressee. The information
contained in this message is confidential and may constitute inside or
non-public information under international, federal, or state securities laws.
Unauthorized forwarding, printing, copying, distribution, or use of such
information is strictly prohibited and may be unlawful. If you are not the
addressee, please promptly delete this message and notify the sender of the
delivery error by e-mail or you may call Cerner's corporate offices in Kansas
City, Missouri, U.S.A at (+1) (816)221-1024.
)

Snapshots in HBase 0.96
---

Key: HBASE-6055
URL: https://issues.apache.org/jira/browse/HBASE-6055
Project: HBase
Issue Type: New Feature
Components: Client, master, regionserver, snapshots, Zookeeper
Reporter: Jesse Yates
Assignee: Jesse Yates
Fix For: hbase-6055, 0.96.0

Attachments: Snapshots in HBase.docx

Continuation of HBASE-50 for the current trunk. Since the implementation has
drastically changed, opening as a new ticket.

[jira] [Updated] (HBASE-7185) Create hbase-protocol module


 [ 
https://issues.apache.org/jira/browse/HBASE-7185?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Elliott Clark updated HBASE-7185:
-

Attachment: HBASE-7185-2.patch

Moved to hbase-protocol

 Create hbase-protocol module
 

 Key: HBASE-7185
 URL: https://issues.apache.org/jira/browse/HBASE-7185
 Project: HBase
  Issue Type: Sub-task
Reporter: Elliott Clark
Assignee: Elliott Clark
 Fix For: 0.96.0

 Attachments: HBASE-7185-0.patch, HBASE-7185-1.patch, 
 HBASE-7185-2.patch


 Create an hbase-rpc module where protobuf's that can cross the wire(either to 
 zk or from a server) will be stored.  This should allow others to create 
 clients without relying on too much of the hbase code base.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-7185) Create hbase-protocol module


 [ 
https://issues.apache.org/jira/browse/HBASE-7185?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Elliott Clark updated HBASE-7185:
-

Summary: Create hbase-protocol module  (was: Create hbase-rpc module)

 Create hbase-protocol module
 

 Key: HBASE-7185
 URL: https://issues.apache.org/jira/browse/HBASE-7185
 Project: HBase
  Issue Type: Sub-task
Reporter: Elliott Clark
Assignee: Elliott Clark
 Fix For: 0.96.0

 Attachments: HBASE-7185-0.patch, HBASE-7185-1.patch, 
 HBASE-7185-2.patch


 Create an hbase-rpc module where protobuf's that can cross the wire(either to 
 zk or from a server) will be stored.  This should allow others to create 
 clients without relying on too much of the hbase code base.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-6777) Snapshot Restore interface

2012-11-20 Thread Matteo Bertozzi (JIRA)


 [ 
https://issues.apache.org/jira/browse/HBASE-6777?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Matteo Bertozzi updated HBASE-6777:
---

Attachment: HBASE-6777-v10.patch

Attached new patch, no changes just regenerated the MasterAdminProtos

also the branch with the patch applied is here:
https://github.com/matteobertozzi/hbase/commits/offline-snapshot-review-v4

 Snapshot Restore interface
 --

 Key: HBASE-6777
 URL: https://issues.apache.org/jira/browse/HBASE-6777
 Project: HBase
  Issue Type: Sub-task
  Components: Client, master, snapshots
Affects Versions: 0.96.0
Reporter: Matteo Bertozzi
Assignee: Matteo Bertozzi
 Fix For: hbase-6055

 Attachments: HBASE-6777-v10.patch, hbase-6777-v9.patch, 
 HBASE--v1.patch


 Add interfaces for restoring a snapshot

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-7009) Port HBaseCluster interface/tests to 0.94


 [ 
https://issues.apache.org/jira/browse/HBASE-7009?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sergey Shelukhin updated HBASE-7009:


Attachment: HBASE-7009-v3-wo-classfinder.patch

 Port HBaseCluster interface/tests to 0.94
 -

 Key: HBASE-7009
 URL: https://issues.apache.org/jira/browse/HBASE-7009
 Project: HBase
  Issue Type: Sub-task
  Components: test
Affects Versions: 0.94.3
Reporter: Sergey Shelukhin
Assignee: Sergey Shelukhin
 Fix For: 0.94.4

 Attachments: HBASE-7009-p1.patch, HBASE-7009.patch, 
 HBASE-7009-v2-squashed.patch, HBASE-7009-v3-squashed.patch, 
 HBASE-7009-v3-wo-classfinder.patch


 Need to port. I am porting V5 patch from the original JIRA; I have a 
 partially ported (V3) patch from Enis with protocol buffers being reverted to 
 HRegionInterface/HMasterInterface

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-7185) Create hbase-protocol module

[
https://issues.apache.org/jira/browse/HBASE-7185?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13501424#comment-13501424
]

Hadoop QA commented on HBASE-7185:
--

{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12554391/HBASE-7185-2.patch
against trunk revision .

{color:green}+1 @author{color}. The patch does not contain any @author
tags.

{color:red}-1 tests included{color}. The patch doesn't appear to include
any new or modified tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.

{color:green}+1 hadoop2.0{color}. The patch compiles against the hadoop
2.0 profile.

{color:red}-1 javadoc{color}. The javadoc tool appears to have generated
99 warning messages.

{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.

{color:red}-1 findbugs{color}. The patch appears to introduce 24 new
Findbugs (version 1.3.9) warnings.

{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.

{color:red}-1 core tests{color}. The patch failed these unit tests:
org.apache.hadoop.hbase.regionserver.TestSplitTransaction

Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/3374//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/3374//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/3374//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/3374//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/3374//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/3374//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/3374//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/3374//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/3374//console

This message is automatically generated.

Create hbase-protocol module

Key: HBASE-7185
URL: https://issues.apache.org/jira/browse/HBASE-7185
Project: HBase
Issue Type: Sub-task
Reporter: Elliott Clark
Assignee: Elliott Clark
Fix For: 0.96.0

Attachments: HBASE-7185-0.patch, HBASE-7185-1.patch,
HBASE-7185-2.patch

[jira] [Commented] (HBASE-7009) Port HBaseCluster interface/tests to 0.94

[
https://issues.apache.org/jira/browse/HBASE-7009?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13501429#comment-13501429
]

Hadoop QA commented on HBASE-7009:
--

{color:red}-1 overall{color}. Here are the results of testing the latest
attachment

http://issues.apache.org/jira/secure/attachment/12554396/HBASE-7009-v3-wo-classfinder.patch
against trunk revision .

{color:green}+1 @author{color}. The patch does not contain any @author
tags.

{color:green}+1 tests included{color}. The patch appears to include 71 new
or modified tests.

{color:red}-1 patch{color}. The patch command could not apply the patch.

Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/3376//console

This message is automatically generated.

Port HBaseCluster interface/tests to 0.94
-

Key: HBASE-7009
URL: https://issues.apache.org/jira/browse/HBASE-7009
Project: HBase
Issue Type: Sub-task
Components: test
Affects Versions: 0.94.3
Reporter: Sergey Shelukhin
Assignee: Sergey Shelukhin
Fix For: 0.94.4

Attachments: HBASE-7009-p1.patch, HBASE-7009.patch,
HBASE-7009-v2-squashed.patch, HBASE-7009-v3-squashed.patch,
HBASE-7009-v3-wo-classfinder.patch

Need to port. I am porting V5 patch from the original JIRA; I have a
partially ported (V3) patch from Enis with protocol buffers being reverted to
HRegionInterface/HMasterInterface

[jira] [Commented] (HBASE-6777) Snapshot Restore interface

[
https://issues.apache.org/jira/browse/HBASE-6777?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13501430#comment-13501430
]

Hadoop QA commented on HBASE-6777:
--

{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12554397/HBASE-6777-v10.patch
against trunk revision .

{color:green}+1 @author{color}. The patch does not contain any @author
tags.

{color:red}-1 patch{color}. The patch command could not apply the patch.

Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/3375//console

This message is automatically generated.

Snapshot Restore interface
--

Key: HBASE-6777
URL: https://issues.apache.org/jira/browse/HBASE-6777
Project: HBase
Issue Type: Sub-task
Components: Client, master, snapshots
Affects Versions: 0.96.0
Reporter: Matteo Bertozzi
Assignee: Matteo Bertozzi
Fix For: hbase-6055

Attachments: HBASE-6777-v10.patch, hbase-6777-v9.patch,
HBASE--v1.patch

Add interfaces for restoring a snapshot

[jira] [Updated] (HBASE-6777) Snapshot Restore interface


 [ 
https://issues.apache.org/jira/browse/HBASE-6777?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jesse Yates updated HBASE-6777:
---

Resolution: Fixed
Status: Resolved  (was: Patch Available)

Committed to snapshot branch

 Snapshot Restore interface
 --

 Key: HBASE-6777
 URL: https://issues.apache.org/jira/browse/HBASE-6777
 Project: HBase
  Issue Type: Sub-task
  Components: Client, master, snapshots
Affects Versions: 0.96.0
Reporter: Matteo Bertozzi
Assignee: Matteo Bertozzi
 Fix For: hbase-6055

 Attachments: HBASE-6777-v10.patch, hbase-6777-v9.patch, 
 HBASE--v1.patch


 Add interfaces for restoring a snapshot

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-6777) Snapshot Restore interface


[ 
https://issues.apache.org/jira/browse/HBASE-6777?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13501431#comment-13501431
 ] 

Jesse Yates commented on HBASE-6777:


[~mbertozzi] hmm, odd. When I tried applying the first time it was getting mad 
at the HMaster. Oh well, thanks for the rebase!

 Snapshot Restore interface
 --

 Key: HBASE-6777
 URL: https://issues.apache.org/jira/browse/HBASE-6777
 Project: HBase
  Issue Type: Sub-task
  Components: Client, master, snapshots
Affects Versions: 0.96.0
Reporter: Matteo Bertozzi
Assignee: Matteo Bertozzi
 Fix For: hbase-6055

 Attachments: HBASE-6777-v10.patch, hbase-6777-v9.patch, 
 HBASE--v1.patch


 Add interfaces for restoring a snapshot

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-6230) [brainstorm] Restore snapshots for HBase 0.96

2012-11-20 Thread Matteo Bertozzi (JIRA)


 [ 
https://issues.apache.org/jira/browse/HBASE-6230?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Matteo Bertozzi updated HBASE-6230:
---

Attachment: HBASE-6230-v3.patch

 [brainstorm] Restore snapshots for HBase 0.96
 ---

 Key: HBASE-6230
 URL: https://issues.apache.org/jira/browse/HBASE-6230
 Project: HBase
  Issue Type: Brainstorming
Reporter: Jesse Yates
Assignee: Matteo Bertozzi
 Attachments: HBASE-6230-v1.patch, HBASE-6230-v2.patch, 
 HBASE-6230-v3.patch, SnapshotRestore-v0.pdf


 Discussion ticket around the definitions/expectations of different parts of 
 snapshot restoration.  This is complementary, but separate from the _how_ of 
 taking a snapshot of a table.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-7191) HBCK - Add offline create/fix hbase.version and hbase.id


 [ 
https://issues.apache.org/jira/browse/HBASE-7191?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Oleg Zhurakousky updated HBASE-7191:


Affects Version/s: (was: 0.96.0)
   0.94.1
   Status: Patch Available  (was: Open)

 HBCK - Add offline create/fix hbase.version and hbase.id 
 -

 Key: HBASE-7191
 URL: https://issues.apache.org/jira/browse/HBASE-7191
 Project: HBase
  Issue Type: Improvement
  Components: hbck
Affects Versions: 0.94.1
Reporter: Enis Soztutar

 One of our clients run into a problem, in which they have the hbase.root.dir, 
 and cluster data, but their hbase.id and hbase.version files are corrupted. 
 HMaster creates those on start, but not if there is already existing data.
 We can add smt like --fixIdFile, and ability for HBCK to do some offline 
 repairs for the version file. 

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-7009) Port HBaseCluster interface/tests to 0.94

[
https://issues.apache.org/jira/browse/HBASE-7009?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13501437#comment-13501437
]

Hadoop QA commented on HBASE-7009:
--

{color:red}-1 overall{color}. Here are the results of testing the latest
attachment

http://issues.apache.org/jira/secure/attachment/12554396/HBASE-7009-v3-wo-classfinder.patch
against trunk revision .

{color:green}+1 @author{color}. The patch does not contain any @author
tags.

{color:green}+1 tests included{color}. The patch appears to include 71 new
or modified tests.

{color:red}-1 patch{color}. The patch command could not apply the patch.

Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/3377//console

This message is automatically generated.

Port HBaseCluster interface/tests to 0.94
-

Attachments: HBASE-7009-p1.patch, HBASE-7009.patch,
HBASE-7009-v2-squashed.patch, HBASE-7009-v3-squashed.patch,
HBASE-7009-v3-wo-classfinder.patch

Need to port. I am porting V5 patch from the original JIRA; I have a
partially ported (V3) patch from Enis with protocol buffers being reverted to
HRegionInterface/HMasterInterface

[jira] [Updated] (HBASE-7191) HBCK - Add offline create/fix hbase.version and hbase.id


 [ 
https://issues.apache.org/jira/browse/HBASE-7191?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Oleg Zhurakousky updated HBASE-7191:


Attachment: 7191.patch

Attaching a patch which basically adds two options to OfflineMetaRepair
{code}
$ hbase org.apache.hadoop.hbase.util.hbck.OfflineMetaRepair -fixVersionFile

$ hbase org.apache.hadoop.hbase.util.hbck.OfflineMetaRepair -fixClusterIdFile
{code}

Also fixed a few other minor things such as System.exit(..) which prevented 
proper testing and quite frankly not needed as access to the executor can still 
be obtain via Reflection.

 HBCK - Add offline create/fix hbase.version and hbase.id 
 -

 Key: HBASE-7191
 URL: https://issues.apache.org/jira/browse/HBASE-7191
 Project: HBase
  Issue Type: Improvement
  Components: hbck
Affects Versions: 0.94.1
Reporter: Enis Soztutar
 Attachments: 7191.patch


 One of our clients run into a problem, in which they have the hbase.root.dir, 
 and cluster data, but their hbase.id and hbase.version files are corrupted. 
 HMaster creates those on start, but not if there is already existing data.
 We can add smt like --fixIdFile, and ability for HBCK to do some offline 
 repairs for the version file. 

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-7191) HBCK - Add offline create/fix hbase.version and hbase.id

[
https://issues.apache.org/jira/browse/HBASE-7191?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13501442#comment-13501442
]

Hadoop QA commented on HBASE-7191:
--

{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12554402/7191.patch
against trunk revision .

{color:green}+1 @author{color}. The patch does not contain any @author
tags.

{color:green}+1 tests included{color}. The patch appears to include 3 new
or modified tests.

{color:red}-1 patch{color}. The patch command could not apply the patch.

Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/3378//console

This message is automatically generated.

HBCK - Add offline create/fix hbase.version and hbase.id
-

Key: HBASE-7191
URL: https://issues.apache.org/jira/browse/HBASE-7191
Project: HBase
Issue Type: Improvement
Components: hbck
Affects Versions: 0.94.1
Reporter: Enis Soztutar
Attachments: 7191.patch

One of our clients run into a problem, in which they have the hbase.root.dir,
and cluster data, but their hbase.id and hbase.version files are corrupted.
HMaster creates those on start, but not if there is already existing data.
We can add smt like --fixIdFile, and ability for HBCK to do some offline
repairs for the version file.

[jira] [Resolved] (HBASE-6230) [brainstorm] Restore snapshots for HBase 0.96


 [ 
https://issues.apache.org/jira/browse/HBASE-6230?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jesse Yates resolved HBASE-6230.


Resolution: Fixed

Committed to https://github.com/jyates/hbase/tree/snapshots. 

 [brainstorm] Restore snapshots for HBase 0.96
 ---

 Key: HBASE-6230
 URL: https://issues.apache.org/jira/browse/HBASE-6230
 Project: HBase
  Issue Type: Brainstorming
Reporter: Jesse Yates
Assignee: Matteo Bertozzi
 Attachments: HBASE-6230-v1.patch, HBASE-6230-v2.patch, 
 HBASE-6230-v3.patch, SnapshotRestore-v0.pdf


 Discussion ticket around the definitions/expectations of different parts of 
 snapshot restoration.  This is complementary, but separate from the _how_ of 
 taking a snapshot of a table.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-7191) HBCK - Add offline create/fix hbase.version and hbase.id


 [ 
https://issues.apache.org/jira/browse/HBASE-7191?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Oleg Zhurakousky updated HBASE-7191:


Attachment: (was: 7191.patch)

 HBCK - Add offline create/fix hbase.version and hbase.id 
 -

 Key: HBASE-7191
 URL: https://issues.apache.org/jira/browse/HBASE-7191
 Project: HBase
  Issue Type: Improvement
  Components: hbck
Affects Versions: 0.94.1
Reporter: Enis Soztutar

 One of our clients run into a problem, in which they have the hbase.root.dir, 
 and cluster data, but their hbase.id and hbase.version files are corrupted. 
 HMaster creates those on start, but not if there is already existing data.
 We can add smt like --fixIdFile, and ability for HBCK to do some offline 
 repairs for the version file. 

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-7191) HBCK - Add offline create/fix hbase.version and hbase.id


 [ 
https://issues.apache.org/jira/browse/HBASE-7191?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Oleg Zhurakousky updated HBASE-7191:


Attachment: 7191-2.patch

The new patch includes modified hbase-site.xml file which has hbase.root set 
which is required by the test

 HBCK - Add offline create/fix hbase.version and hbase.id 
 -

 Key: HBASE-7191
 URL: https://issues.apache.org/jira/browse/HBASE-7191
 Project: HBase
  Issue Type: Improvement
  Components: hbck
Affects Versions: 0.94.1
Reporter: Enis Soztutar
 Attachments: 7191-2.patch


 One of our clients run into a problem, in which they have the hbase.root.dir, 
 and cluster data, but their hbase.id and hbase.version files are corrupted. 
 HMaster creates those on start, but not if there is already existing data.
 We can add smt like --fixIdFile, and ability for HBCK to do some offline 
 repairs for the version file. 

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-7191) HBCK - Add offline create/fix hbase.version and hbase.id

[
https://issues.apache.org/jira/browse/HBASE-7191?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13501462#comment-13501462
]

Hadoop QA commented on HBASE-7191:
--

{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12554407/7191-2.patch
against trunk revision .

{color:green}+1 @author{color}. The patch does not contain any @author
tags.

{color:green}+1 tests included{color}. The patch appears to include 6 new
or modified tests.

{color:red}-1 patch{color}. The patch command could not apply the patch.

Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/3379//console

This message is automatically generated.

HBCK - Add offline create/fix hbase.version and hbase.id
-

Key: HBASE-7191
URL: https://issues.apache.org/jira/browse/HBASE-7191
Project: HBase
Issue Type: Improvement
Components: hbck
Affects Versions: 0.94.1
Reporter: Enis Soztutar
Attachments: 7191-2.patch

[jira] [Updated] (HBASE-6802) Export Snapshot


 [ 
https://issues.apache.org/jira/browse/HBASE-6802?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jesse Yates updated HBASE-6802:
---

Resolution: Fixed
Status: Resolved  (was: Patch Available)

Committed to snapshot branch

 Export Snapshot
 ---

 Key: HBASE-6802
 URL: https://issues.apache.org/jira/browse/HBASE-6802
 Project: HBase
  Issue Type: Sub-task
  Components: snapshots
Reporter: Matteo Bertozzi
Assignee: Matteo Bertozzi
 Fix For: hbase-6055, 0.96.0

 Attachments: HBASE-6802-v1.patch


 Export a snapshot to another cluster.
  - Copy the .snapshot/name folder with all the references
  - Copy the hfiles/hlogs needed by the snapshot
 Once the other cluster has the files and the snapshot information it can 
 restore the snapshot.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-7191) HBCK - Add offline create/fix hbase.version and hbase.id


[ 
https://issues.apache.org/jira/browse/HBASE-7191?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13501489#comment-13501489
 ] 

Oleg Zhurakousky commented on HBASE-7191:
-

Not sure what I can do. The test does pass but there are other failing tests:
{code}
Failed tests:   
testSplitCalculatorEq(org.apache.hadoop.hbase.util.TestRegionSplitCalculator): 
expected:2 but was:1
  testNativeSizes(org.apache.hadoop.hbase.io.TestHeapSize): expected:56 but 
was:64
  testSizes(org.apache.hadoop.hbase.io.TestHeapSize): expected:192 but 
was:200
{code}

 and looking at the Jenkins report there hasn't been a successful build since 
October 30 (as far as I can see).

The irony is that for example TestRegionSplitCalculator test passes locally if 
ran in isolation 

 HBCK - Add offline create/fix hbase.version and hbase.id 
 -

 Key: HBASE-7191
 URL: https://issues.apache.org/jira/browse/HBASE-7191
 Project: HBase
  Issue Type: Improvement
  Components: hbck
Affects Versions: 0.94.1
Reporter: Enis Soztutar
 Attachments: 7191-2.patch


 One of our clients run into a problem, in which they have the hbase.root.dir, 
 and cluster data, but their hbase.id and hbase.version files are corrupted. 
 HMaster creates those on start, but not if there is already existing data.
 We can add smt like --fixIdFile, and ability for HBCK to do some offline 
 repairs for the version file. 

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-7185) Create hbase-protocol module


[ 
https://issues.apache.org/jira/browse/HBASE-7185?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13501510#comment-13501510
 ] 

stack commented on HBASE-7185:
--

+1

 Create hbase-protocol module
 

 Key: HBASE-7185
 URL: https://issues.apache.org/jira/browse/HBASE-7185
 Project: HBase
  Issue Type: Sub-task
Reporter: Elliott Clark
Assignee: Elliott Clark
 Fix For: 0.96.0

 Attachments: HBASE-7185-0.patch, HBASE-7185-1.patch, 
 HBASE-7185-2.patch


 Create an hbase-rpc module where protobuf's that can cross the wire(either to 
 zk or from a server) will be stored.  This should allow others to create 
 clients without relying on too much of the hbase code base.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-7191) HBCK - Add offline create/fix hbase.version and hbase.id

2012-11-20 Thread Devaraj Das (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-7191?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13501523#comment-13501523
 ] 

Devaraj Das commented on HBASE-7191:


Haven't looked at the patch in detail but some things that you should take care 
of: 
(1) indentation of code (two spaces, look at the existing code for getting a 
feel of how it should look like).
(2) patch should be submitted for trunk first (ideally)
There are flaky tests in the codebase. If you see an unrelated test failing on 
hudson, and the test passes locally, please make a note of it as a comment on 
jira.

 HBCK - Add offline create/fix hbase.version and hbase.id 
 -

 Key: HBASE-7191
 URL: https://issues.apache.org/jira/browse/HBASE-7191
 Project: HBase
  Issue Type: Improvement
  Components: hbck
Affects Versions: 0.94.1
Reporter: Enis Soztutar
 Attachments: 7191-2.patch


 One of our clients run into a problem, in which they have the hbase.root.dir, 
 and cluster data, but their hbase.id and hbase.version files are corrupted. 
 HMaster creates those on start, but not if there is already existing data.
 We can add smt like --fixIdFile, and ability for HBCK to do some offline 
 repairs for the version file. 

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-7191) HBCK - Add offline create/fix hbase.version and hbase.id


[ 
https://issues.apache.org/jira/browse/HBASE-7191?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13501528#comment-13501528
 ] 

Ted Yu commented on HBASE-7191:
---

For OfflineVersionAndClusterIdFixTests.java, usually test class starts with 
Test. Name it TestOfflineVersionAndClusterIdFix
It should carry the following annotation:
{code}
@Category(SmallTests.class)
{code}
{code}
-  System.exit(success ? 0 : 1);
+ shutDownFsck(fsck);
{code}
Can you log the value of success above ?
For shutDownFsck(), can you add a method in HBaseFsck which shuts down executor 
?

 HBCK - Add offline create/fix hbase.version and hbase.id 
 -

 Key: HBASE-7191
 URL: https://issues.apache.org/jira/browse/HBASE-7191
 Project: HBase
  Issue Type: Improvement
  Components: hbck
Affects Versions: 0.94.1
Reporter: Enis Soztutar
 Attachments: 7191-2.patch


 One of our clients run into a problem, in which they have the hbase.root.dir, 
 and cluster data, but their hbase.id and hbase.version files are corrupted. 
 HMaster creates those on start, but not if there is already existing data.
 We can add smt like --fixIdFile, and ability for HBCK to do some offline 
 repairs for the version file. 

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-7191) HBCK - Add offline create/fix hbase.version and hbase.id


[ 
https://issues.apache.org/jira/browse/HBASE-7191?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13501529#comment-13501529
 ] 

Oleg Zhurakousky commented on HBASE-7191:
-

Thanks Deveraj
The 3 failing tests are mentioned in the previous comment. Actually they pass 
in isolation, but not as a full build (unrelated to the patch). I'll check the 
indentation and resubmit. Let me know if there is anything else you'll see in 
the meantime. I'll also do another one anyway from the trunk


 HBCK - Add offline create/fix hbase.version and hbase.id 
 -

 Key: HBASE-7191
 URL: https://issues.apache.org/jira/browse/HBASE-7191
 Project: HBase
  Issue Type: Improvement
  Components: hbck
Affects Versions: 0.94.1
Reporter: Enis Soztutar
 Attachments: 7191-2.patch


 One of our clients run into a problem, in which they have the hbase.root.dir, 
 and cluster data, but their hbase.id and hbase.version files are corrupted. 
 HMaster creates those on start, but not if there is already existing data.
 We can add smt like --fixIdFile, and ability for HBCK to do some offline 
 repairs for the version file. 

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-7191) HBCK - Add offline create/fix hbase.version and hbase.id


[ 
https://issues.apache.org/jira/browse/HBASE-7191?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13501530#comment-13501530
 ] 

Oleg Zhurakousky commented on HBASE-7191:
-

[~yuzhih...@gmail.com]
Sure, will do

 HBCK - Add offline create/fix hbase.version and hbase.id 
 -

 Key: HBASE-7191
 URL: https://issues.apache.org/jira/browse/HBASE-7191
 Project: HBase
  Issue Type: Improvement
  Components: hbck
Affects Versions: 0.94.1
Reporter: Enis Soztutar
 Attachments: 7191-2.patch


 One of our clients run into a problem, in which they have the hbase.root.dir, 
 and cluster data, but their hbase.id and hbase.version files are corrupted. 
 HMaster creates those on start, but not if there is already existing data.
 We can add smt like --fixIdFile, and ability for HBCK to do some offline 
 repairs for the version file. 

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-7191) HBCK - Add offline create/fix hbase.version and hbase.id


[ 
https://issues.apache.org/jira/browse/HBASE-7191?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13501568#comment-13501568
 ] 

Ted Yu commented on HBASE-7191:
---

@Oleg:
w.r.t. the test failures in TestHeapSize, did you run the tests using Java 1.7 ?
I encountered the same failure on MacBook with 1.7.0_07-b10

 HBCK - Add offline create/fix hbase.version and hbase.id 
 -

 Key: HBASE-7191
 URL: https://issues.apache.org/jira/browse/HBASE-7191
 Project: HBase
  Issue Type: Improvement
  Components: hbck
Affects Versions: 0.94.1
Reporter: Enis Soztutar
 Attachments: 7191-2.patch


 One of our clients run into a problem, in which they have the hbase.root.dir, 
 and cluster data, but their hbase.id and hbase.version files are corrupted. 
 HMaster creates those on start, but not if there is already existing data.
 We can add smt like --fixIdFile, and ability for HBCK to do some offline 
 repairs for the version file. 

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-7185) Create hbase-protocol module


 [ 
https://issues.apache.org/jira/browse/HBASE-7185?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Elliott Clark updated HBASE-7185:
-

  Resolution: Fixed
Release Note: Created hbase-protocol module to extract all protobuf files 
that could be transmitted. 
Hadoop Flags: Reviewed
  Status: Resolved  (was: Patch Available)

Committed revision 1411922.

 Create hbase-protocol module
 

 Key: HBASE-7185
 URL: https://issues.apache.org/jira/browse/HBASE-7185
 Project: HBase
  Issue Type: Sub-task
Reporter: Elliott Clark
Assignee: Elliott Clark
 Fix For: 0.96.0

 Attachments: HBASE-7185-0.patch, HBASE-7185-1.patch, 
 HBASE-7185-2.patch


 Create an hbase-rpc module where protobuf's that can cross the wire(either to 
 zk or from a server) will be stored.  This should allow others to create 
 clients without relying on too much of the hbase code base.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-7109) integration tests on cluster are not getting picked up from distribution


 [ 
https://issues.apache.org/jira/browse/HBASE-7109?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sergey Shelukhin updated HBASE-7109:


Attachment: HBASE-7109-v5.patch

Added tests... unfortunately I didn't realize when I started how tedious 
mucking with jar files and ClassLoader is... But now it's there :)

 integration tests on cluster are not getting picked up from distribution
 

 Key: HBASE-7109
 URL: https://issues.apache.org/jira/browse/HBASE-7109
 Project: HBase
  Issue Type: Sub-task
  Components: test
Affects Versions: 0.96.0
Reporter: Sergey Shelukhin
Assignee: Sergey Shelukhin
 Attachments: HBASE-7109-squashed.patch, HBASE-7109-v2-squashed.patch, 
 HBASE-7109-v3-squashed.patch, HBASE-7109-v4-squashed.patch, 
 HBASE-7109-v5.patch


 The method of finding test classes only works on local build (or its full 
 copy), not if the distribution is used.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-7109) integration tests on cluster are not getting picked up from distribution


 [ 
https://issues.apache.org/jira/browse/HBASE-7109?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sergey Shelukhin updated HBASE-7109:


Attachment: HBASE-7109-v5.patch

 integration tests on cluster are not getting picked up from distribution
 

 Key: HBASE-7109
 URL: https://issues.apache.org/jira/browse/HBASE-7109
 Project: HBase
  Issue Type: Sub-task
  Components: test
Affects Versions: 0.96.0
Reporter: Sergey Shelukhin
Assignee: Sergey Shelukhin
 Attachments: HBASE-7109-squashed.patch, HBASE-7109-v2-squashed.patch, 
 HBASE-7109-v3-squashed.patch, HBASE-7109-v4-squashed.patch, 
 HBASE-7109-v5.patch, HBASE-7109-v5.patch


 The method of finding test classes only works on local build (or its full 
 copy), not if the distribution is used.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Created] (HBASE-7198) fix lastSeqId logic in regionserver

2012-11-20 Thread Amitanand Aiyer (JIRA)

Amitanand Aiyer created HBASE-7198:
--

 Summary: fix lastSeqId logic in regionserver
 Key: HBASE-7198
 URL: https://issues.apache.org/jira/browse/HBASE-7198
 Project: HBase
  Issue Type: Bug
Affects Versions: 0.89-fb
Reporter: Amitanand Aiyer
Priority: Minor
 Fix For: 0.89-fb




--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-2376) Add special SnapshotScanner which presents view of all data at some time in the past

2012-11-20 Thread Kannan Muthukkaruppan (JIRA)


 [ 
https://issues.apache.org/jira/browse/HBASE-2376?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Kannan Muthukkaruppan updated HBASE-2376:
-

Assignee: Pritam Damania

 Add special SnapshotScanner which presents view of all data at some time in 
 the past
 

 Key: HBASE-2376
 URL: https://issues.apache.org/jira/browse/HBASE-2376
 Project: HBase
  Issue Type: New Feature
  Components: Client, regionserver
Affects Versions: 0.20.3
Reporter: Jonathan Gray
Assignee: Pritam Damania

 In order to support a particular kind of database snapshot feature which 
 doesn't require copying data, we came up with the idea for a special 
 SnapshotScanner that would present a view of your data at some point in the 
 past.  The primary use case for this would be to be able to recover 
 particular data/rows (but not all data, like a global rollback) should they 
 have somehow been messed up (application fault, application bug, user error, 
 etc.).

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-7185) Create hbase-protocol module

2012-11-20 Thread Hudson (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-7185?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13501603#comment-13501603
 ] 

Hudson commented on HBASE-7185:
---

Integrated in HBase-TRUNK #3554 (See 
[https://builds.apache.org/job/HBase-TRUNK/3554/])
HBASE-7185 Create hbase-protocol module (Revision 1411922)

 Result = FAILURE
eclark : 
Files : 
* /hbase/trunk/hbase-protocol
* /hbase/trunk/hbase-protocol/pom.xml
* /hbase/trunk/hbase-protocol/src
* /hbase/trunk/hbase-protocol/src/main
* /hbase/trunk/hbase-protocol/src/main/java
* /hbase/trunk/hbase-protocol/src/main/java/org
* /hbase/trunk/hbase-protocol/src/main/java/org/apache
* /hbase/trunk/hbase-protocol/src/main/java/org/apache/hadoop
* /hbase/trunk/hbase-protocol/src/main/java/org/apache/hadoop/hbase
* /hbase/trunk/hbase-protocol/src/main/java/org/apache/hadoop/hbase/protobuf
* 
/hbase/trunk/hbase-protocol/src/main/java/org/apache/hadoop/hbase/protobuf/generated
* 
/hbase/trunk/hbase-protocol/src/main/java/org/apache/hadoop/hbase/protobuf/generated/AccessControlProtos.java
* 
/hbase/trunk/hbase-protocol/src/main/java/org/apache/hadoop/hbase/protobuf/generated/AdminProtos.java
* 
/hbase/trunk/hbase-protocol/src/main/java/org/apache/hadoop/hbase/protobuf/generated/AggregateProtos.java
* 
/hbase/trunk/hbase-protocol/src/main/java/org/apache/hadoop/hbase/protobuf/generated/ClientProtos.java
* 
/hbase/trunk/hbase-protocol/src/main/java/org/apache/hadoop/hbase/protobuf/generated/ClusterIdProtos.java
* 
/hbase/trunk/hbase-protocol/src/main/java/org/apache/hadoop/hbase/protobuf/generated/ClusterStatusProtos.java
* 
/hbase/trunk/hbase-protocol/src/main/java/org/apache/hadoop/hbase/protobuf/generated/ComparatorProtos.java
* 
/hbase/trunk/hbase-protocol/src/main/java/org/apache/hadoop/hbase/protobuf/generated/FSProtos.java
* 
/hbase/trunk/hbase-protocol/src/main/java/org/apache/hadoop/hbase/protobuf/generated/FilterProtos.java
* 
/hbase/trunk/hbase-protocol/src/main/java/org/apache/hadoop/hbase/protobuf/generated/HBaseProtos.java
* 
/hbase/trunk/hbase-protocol/src/main/java/org/apache/hadoop/hbase/protobuf/generated/LoadBalancerProtos.java
* 
/hbase/trunk/hbase-protocol/src/main/java/org/apache/hadoop/hbase/protobuf/generated/MasterAdminProtos.java
* 
/hbase/trunk/hbase-protocol/src/main/java/org/apache/hadoop/hbase/protobuf/generated/MasterMonitorProtos.java
* 
/hbase/trunk/hbase-protocol/src/main/java/org/apache/hadoop/hbase/protobuf/generated/MasterProtos.java
* 
/hbase/trunk/hbase-protocol/src/main/java/org/apache/hadoop/hbase/protobuf/generated/MultiRowMutation.java
* 
/hbase/trunk/hbase-protocol/src/main/java/org/apache/hadoop/hbase/protobuf/generated/RPCProtos.java
* 
/hbase/trunk/hbase-protocol/src/main/java/org/apache/hadoop/hbase/protobuf/generated/RegionServerStatusProtos.java
* 
/hbase/trunk/hbase-protocol/src/main/java/org/apache/hadoop/hbase/protobuf/generated/Tracing.java
* 
/hbase/trunk/hbase-protocol/src/main/java/org/apache/hadoop/hbase/protobuf/generated/ZooKeeperProtos.java
* /hbase/trunk/hbase-protocol/src/main/protobuf
* /hbase/trunk/hbase-protocol/src/main/protobuf/AccessControl.proto
* /hbase/trunk/hbase-protocol/src/main/protobuf/Admin.proto
* /hbase/trunk/hbase-protocol/src/main/protobuf/Aggregate.proto
* /hbase/trunk/hbase-protocol/src/main/protobuf/BulkDelete.proto
* /hbase/trunk/hbase-protocol/src/main/protobuf/Client.proto
* /hbase/trunk/hbase-protocol/src/main/protobuf/ClusterId.proto
* /hbase/trunk/hbase-protocol/src/main/protobuf/ClusterStatus.proto
* /hbase/trunk/hbase-protocol/src/main/protobuf/Comparator.proto
* /hbase/trunk/hbase-protocol/src/main/protobuf/Examples.proto
* /hbase/trunk/hbase-protocol/src/main/protobuf/FS.proto
* /hbase/trunk/hbase-protocol/src/main/protobuf/Filter.proto
* /hbase/trunk/hbase-protocol/src/main/protobuf/LoadBalancer.proto
* /hbase/trunk/hbase-protocol/src/main/protobuf/Master.proto
* /hbase/trunk/hbase-protocol/src/main/protobuf/MasterAdmin.proto
* /hbase/trunk/hbase-protocol/src/main/protobuf/MasterMonitor.proto
* /hbase/trunk/hbase-protocol/src/main/protobuf/MultiRowMutation.proto
* /hbase/trunk/hbase-protocol/src/main/protobuf/README.txt
* /hbase/trunk/hbase-protocol/src/main/protobuf/RPC.proto
* /hbase/trunk/hbase-protocol/src/main/protobuf/RegionServerStatus.proto
* /hbase/trunk/hbase-protocol/src/main/protobuf/Tracing.proto
* /hbase/trunk/hbase-protocol/src/main/protobuf/ZooKeeper.proto
* /hbase/trunk/hbase-protocol/src/main/protobuf/hbase.proto
* /hbase/trunk/hbase-server/pom.xml
* 
/hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/protobuf/generated
* /hbase/trunk/hbase-server/src/main/protobuf
* /hbase/trunk/pom.xml


 Create hbase-protocol module
 

 Key: HBASE-7185
 URL: https://issues.apache.org/jira/browse/HBASE-7185
 Project: HBase
  Issue Type: Sub-task
Reporter: Elliott Clark

[jira] [Commented] (HBASE-2376) Add special SnapshotScanner which presents view of all data at some time in the past

2012-11-20 Thread Pritam Damania (JIRA)


[ 
https://issues.apache.org/jira/browse/HBASE-2376?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13501602#comment-13501602
 ] 

Pritam Damania commented on HBASE-2376:
---

Here are the relevant patches for this feature in the 89fb branch :
http://svn.apache.org/viewvc?view=revisionrevision=r1395032
http://svn.apache.org/viewvc?view=revisionrevision=r1406789
http://svn.apache.org/viewvc?view=revisionrevision=r1410118

 Add special SnapshotScanner which presents view of all data at some time in 
 the past
 

 Key: HBASE-2376
 URL: https://issues.apache.org/jira/browse/HBASE-2376
 Project: HBase
  Issue Type: New Feature
  Components: Client, regionserver
Affects Versions: 0.20.3
Reporter: Jonathan Gray
Assignee: Pritam Damania

 In order to support a particular kind of database snapshot feature which 
 doesn't require copying data, we came up with the idea for a special 
 SnapshotScanner that would present a view of your data at some point in the 
 past.  The primary use case for this would be to be able to recover 
 particular data/rows (but not all data, like a global rollback) should they 
 have somehow been messed up (application fault, application bug, user error, 
 etc.).

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HBASE-6423) Writes should not block reads on blocking updates to memstores

2012-11-20 Thread Jimmy Xiang (JIRA)


 [ 
https://issues.apache.org/jira/browse/HBASE-6423?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jimmy Xiang updated HBASE-6423:
---

Status: Open  (was: Patch Available)

 Writes should not block reads on blocking updates to memstores
 --

 Key: HBASE-6423
 URL: https://issues.apache.org/jira/browse/HBASE-6423
 Project: HBase
  Issue Type: Bug
Reporter: Karthik Ranganathan
Assignee: Jimmy Xiang
 Attachments: trunk-6423.patch, trunk-6423_v2.1.patch, 
 trunk-6423_v2.patch


 We have a big data use case where we turn off WAL and have a ton of reads and 
 writes. We found that:
 1. flushing a memstore takes a while (GZIP compression)
 2. incoming writes cause the new memstore to grow in an unbounded fashion
 3. this triggers blocking memstore updates
 4. in turn, this causes all the RPC handler threads to block on writes to 
 that memstore
 5. we are not able to read during this time as RPC handlers are blocked
 At a higher level, we should not hold up the RPC threads while blocking 
 updates, and we should build in some sort of rate control.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-7109) integration tests on cluster are not getting picked up from distribution

[
https://issues.apache.org/jira/browse/HBASE-7109?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13501616#comment-13501616
]

Hadoop QA commented on HBASE-7109:
--

{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12554426/HBASE-7109-v5.patch
against trunk revision .

{color:green}+1 @author{color}. The patch does not contain any @author
tags.

{color:green}+1 tests included{color}. The patch appears to include 13 new
or modified tests.

{color:green}+1 hadoop2.0{color}. The patch compiles against the hadoop
2.0 profile.

{color:red}-1 javadoc{color}. The javadoc tool appears to have generated
99 warning messages.

{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.

{color:red}-1 findbugs{color}. The patch appears to introduce 24 new
Findbugs (version 1.3.9) warnings.

{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.

{color:red}-1 core tests{color}. The patch failed these unit tests:
org.apache.hadoop.hbase.util.TestMiniClusterLoadParallel

Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/3380//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/3380//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/3380//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/3380//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/3380//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/3380//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/3380//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/3380//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/3380//console

This message is automatically generated.

integration tests on cluster are not getting picked up from distribution

Key: HBASE-7109
URL: https://issues.apache.org/jira/browse/HBASE-7109
Project: HBase
Issue Type: Sub-task
Components: test
Affects Versions: 0.96.0
Reporter: Sergey Shelukhin
Assignee: Sergey Shelukhin
Attachments: HBASE-7109-squashed.patch, HBASE-7109-v2-squashed.patch,
HBASE-7109-v3-squashed.patch, HBASE-7109-v4-squashed.patch,
HBASE-7109-v5.patch, HBASE-7109-v5.patch

The method of finding test classes only works on local build (or its full
copy), not if the distribution is used.

[jira] [Created] (HBASE-7199) hbck should check lingering reference hfile and have option to sideline them automatically

2012-11-20 Thread Jimmy Xiang (JIRA)

Jimmy Xiang created HBASE-7199:
--

 Summary: hbck should check lingering reference hfile and have 
option to sideline them automatically
 Key: HBASE-7199
 URL: https://issues.apache.org/jira/browse/HBASE-7199
 Project: HBase
  Issue Type: Improvement
  Components: hbck
Reporter: Jimmy Xiang


Sometimes, some lingering reference hfile points to some region doesn't exist 
any more.  This will prevent the region to open.  Hbck should be able to find 
these files and sideline them automatically if requested.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HBASE-7109) integration tests on cluster are not getting picked up from distribution