[jira] [Resolved] (HBASE-16702) TestBlockEvictionFromClient is broken

2016-09-23 Thread Duo Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-16702?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang resolved HBASE-16702. --- Resolution: Duplicate Duplicated with HBASE-16696. > TestBlockEvictionFromClient is broken >

[jira] [Created] (HBASE-16702) TestBlockEvictionFromClient is broken

2016-09-23 Thread Heng Chen (JIRA)
Heng Chen created HBASE-16702: - Summary: TestBlockEvictionFromClient is broken Key: HBASE-16702 URL: https://issues.apache.org/jira/browse/HBASE-16702 Project: HBase Issue Type: Bug

Backup Implementation (WAS => Re: [DISCUSSION] MR jobs started by Master or RS)

2016-09-23 Thread Stack
(Moved out of the Master doing MR DISCUSSION) On Fri, Sep 23, 2016 at 12:24 PM, Vladimir Rodionov wrote: > >> -1 on that backup be in core hbase > > Not sure I understand what it means. > > Sorry for the imprecision. The -1 is NOT against backup/restore. I am -1 on MR

[jira] [Created] (HBASE-16701) TestHRegion and TestHRegionWithInMemoryFlush timing out

2016-09-23 Thread Appy (JIRA)
Appy created HBASE-16701: Summary: TestHRegion and TestHRegionWithInMemoryFlush timing out Key: HBASE-16701 URL: https://issues.apache.org/jira/browse/HBASE-16701 Project: HBase Issue Type: Bug

Re: [DISCUSSION] MR jobs started by Master or RS

2016-09-23 Thread Andrew Purtell
It's simple: Use a tool (something derived from Tool, like our other utilities for import, export, etc.), or use a separate process, but don't "shell out" to MR from the Master. On Fri, Sep 23, 2016 at 12:24 PM, Vladimir Rodionov wrote: > >> -1 on that backup be in

[jira] [Created] (HBASE-16700) Allow for coprocessor whitelisting

2016-09-23 Thread Clay B. (JIRA)
Clay B. created HBASE-16700: --- Summary: Allow for coprocessor whitelisting Key: HBASE-16700 URL: https://issues.apache.org/jira/browse/HBASE-16700 Project: HBase Issue Type: Improvement

[jira] [Created] (HBASE-16699) Overflows in AverageIntervalRateLimiter's refill() and getWaitInterval()

2016-09-23 Thread huaxiang sun (JIRA)
huaxiang sun created HBASE-16699: Summary: Overflows in AverageIntervalRateLimiter's refill() and getWaitInterval() Key: HBASE-16699 URL: https://issues.apache.org/jira/browse/HBASE-16699 Project:

[jira] [Created] (HBASE-16698) Handlers stuck waiting for CountDownLatch inside WALKey#getWriteEntry under high writing workload

2016-09-23 Thread Yu Li (JIRA)
Yu Li created HBASE-16698: - Summary: Handlers stuck waiting for CountDownLatch inside WALKey#getWriteEntry under high writing workload Key: HBASE-16698 URL: https://issues.apache.org/jira/browse/HBASE-16698

Re: [DISCUSSION] MR jobs started by Master or RS

2016-09-23 Thread Vladimir Rodionov
>> -1 on that backup be in core hbase Not sure I understand what it means. 1. We are not allowed to use Master to orchestrate the whole process? We have already brought up all advantages of using Master and distributed procedures for backup and restore. Downside of moving this to client

[jira] [Resolved] (HBASE-16693) The commit of HBASE-16604 creates a unrelated folder

2016-09-23 Thread Enis Soztutar (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-16693?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Enis Soztutar resolved HBASE-16693. --- Resolution: Fixed Fix Version/s: 2.0.0 Should be resolved now. > The commit of

Re: [DISCUSSION] MR jobs started by Master or RS

2016-09-23 Thread Stack
On Fri, Sep 23, 2016 at 11:22 AM, Vladimir Rodionov wrote: > >> + MR is dead > > Does MR know that? :) > > Again. With all due respect, stack - still no suggestions what should we > use for "bulk data move and transformation" instead of MR? > Use whatever distributed

Re: [DISCUSSION] MR jobs started by Master or RS

2016-09-23 Thread Vladimir Rodionov
>> + MR is dead Does MR know that? :) Again. With all due respect, stack - still no suggestions what should we use for "bulk data move and transformation" instead of MR? Backup needs this, by default. We are talking about TBs of data in a single session. We not only copy data between src and

Re: [DISCUSSION] MR jobs started by Master or RS

2016-09-23 Thread Stack
On Fri, Sep 23, 2016 at 6:46 AM, Matteo Bertozzi wrote: > let me try to go back to my original topic. > this question was meant to be generic, and provide some rule for future > code. > > from what I can gather, a rule that may satisfy everyone can be: > - we don't want

Re: [DISCUSSION] MR jobs started by Master or RS

2016-09-23 Thread Devaraj Das
Great points. I am getting a feeling that we have beaten this topic to death and have spent enough and more time on it, and it's time to move on. My takeaway is that it's fine to do MR for backup/restore - backup/restore is an optional feature - unless you use it, you don't need MR. Fingers

[jira] [Resolved] (HBASE-16292) Can't start a local instance

2016-09-23 Thread Jerry He (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-16292?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jerry He resolved HBASE-16292. -- Resolution: Duplicate > Can't start a local instance > > >

[jira] [Created] (HBASE-16697) bump TestRegionServerMetrics to LargeTests

2016-09-23 Thread Matteo Bertozzi (JIRA)
Matteo Bertozzi created HBASE-16697: --- Summary: bump TestRegionServerMetrics to LargeTests Key: HBASE-16697 URL: https://issues.apache.org/jira/browse/HBASE-16697 Project: HBase Issue Type:

Re: [DISCUSSION] MR jobs started by Master or RS

2016-09-23 Thread Jerry He
That is the point, Matteo. Put it another way, there is nothing that stops a user from deploying custom procedure, custom co-processor that calls out MR job. The optional feature should satisfy some basic requirements. .e.g. No impact if not deployed or used. Limited impact if used. It can be

Re: [DISCUSSION] MR jobs started by Master or RS

2016-09-23 Thread Nick Dimiduk
So we support MR/YARN anyway through these optional features, thus burdening is with a baseline of maintenance. Since we have that anyway, do we accept _optional_ implementations of "core" features based on MR it even YARN? I'd guess HBase is co-deployed with a YARN or Spark cluster often enough

Successful: HBase Generate Website

2016-09-23 Thread Apache Jenkins Server
Build status: Successful If successful, the website and docs have been generated. To update the live site, follow the instructions below. If failed, skip to the bottom of this email. Use the following commands to download the patch and apply it to a clean branch based on origin/asf-site. If

[jira] [Created] (HBASE-16696) TestBlockEvictionFromClient fails in master branch

2016-09-23 Thread Ted Yu (JIRA)
Ted Yu created HBASE-16696: -- Summary: TestBlockEvictionFromClient fails in master branch Key: HBASE-16696 URL: https://issues.apache.org/jira/browse/HBASE-16696 Project: HBase Issue Type: Bug

[jira] [Created] (HBASE-16695) Procedure v2 - Support for parent holding locks

2016-09-23 Thread Matteo Bertozzi (JIRA)
Matteo Bertozzi created HBASE-16695: --- Summary: Procedure v2 - Support for parent holding locks Key: HBASE-16695 URL: https://issues.apache.org/jira/browse/HBASE-16695 Project: HBase Issue

Re: [DISCUSSION] MR jobs started by Master or RS

2016-09-23 Thread Matteo Bertozzi
let me try to go back to my original topic. this question was meant to be generic, and provide some rule for future code. from what I can gather, a rule that may satisfy everyone can be: - we don't want any core feature (e.g. compaction/log-split/log-reply) over MR, because some cluster may not

[jira] [Created] (HBASE-16694) Reduce garbage for onDiskChecksum in HFileBlock

2016-09-23 Thread binlijin (JIRA)
binlijin created HBASE-16694: Summary: Reduce garbage for onDiskChecksum in HFileBlock Key: HBASE-16694 URL: https://issues.apache.org/jira/browse/HBASE-16694 Project: HBase Issue Type:

Re: [DISCUSSION] MR jobs started by Master or RS

2016-09-23 Thread Ted Yu
I suggest you look at Matteo's work for AssignmentManager which is to make Master more stable. Cheers On Fri, Sep 23, 2016 at 5:32 AM, 张铎 wrote: > No, not your fault, at lease, not this time:) > > Why I call the code ugly? Can you simply tell me the sequence of calls

Re: [DISCUSSION] MR jobs started by Master or RS

2016-09-23 Thread Ted Yu
I read through HADOOP-13433 - the cited race condition is in jdk. Suggest pinging the reviewer on JIRA to get it moving. bq. But the ugly code in HMaster is readlly a problem... Can you be specific as to which code is ugly ? Is it in the

[jira] [Created] (HBASE-16692) Make ByteBufferUtils#equals more safe and correct

2016-09-23 Thread binlijin (JIRA)
binlijin created HBASE-16692: Summary: Make ByteBufferUtils#equals more safe and correct Key: HBASE-16692 URL: https://issues.apache.org/jira/browse/HBASE-16692 Project: HBase Issue Type:

[jira] [Created] (HBASE-16691) optimize KeyOnlyFilter

2016-09-23 Thread binlijin (JIRA)
binlijin created HBASE-16691: Summary: optimize KeyOnlyFilter Key: HBASE-16691 URL: https://issues.apache.org/jira/browse/HBASE-16691 Project: HBase Issue Type: Improvement Reporter:

[jira] [Created] (HBASE-16690) Move znode path configs to a separated class

2016-09-23 Thread Duo Zhang (JIRA)
Duo Zhang created HBASE-16690: - Summary: Move znode path configs to a separated class Key: HBASE-16690 URL: https://issues.apache.org/jira/browse/HBASE-16690 Project: HBase Issue Type: