[jira] [Created] (HBASE-14265) we should forbidden create table using 'hbase' namespace

2015-08-19 Thread Heng Chen (JIRA)
Heng Chen created HBASE-14265: - Summary: we should forbidden create table using 'hbase' namespace Key: HBASE-14265 URL: https://issues.apache.org/jira/browse/HBASE-14265 Project: HBase Issue Type

[jira] [Reopened] (HBASE-14225) Exclude **/target/** from RAT checks

2015-08-19 Thread Andrew Purtell (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-14225?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell reopened HBASE-14225: Assignee: Andrew Purtell Nope, we need this. Even after upgrading to RAT 0.11 (see HBASE-

Re: Testing and CI -- Apache Jenkins Builds

2015-08-19 Thread Stack
https://issues.apache.org/jira/browse/BUILDS-109 St.Ack On Wed, Aug 19, 2015 at 5:38 PM, Stack wrote: > On Wed, Aug 19, 2015 at 5:24 PM, Stack wrote: > >> On Wed, Aug 19, 2015 at 5:08 PM, Sean Busbey wrote: >> >>> I don't recall anything, but I'll go dig through the infra@ list later >>> this

Re: HBase and Accumulo

2015-08-19 Thread Jerry He
I definitely agree HBase has a broader base. Thanks, Ted. Jerry On Wed, Aug 19, 2015 at 4:42 PM, Ted Malaska wrote: > I would say most banks are hbase but there r a few with accumulo. I have > most bank, broker dealers and regulators in my region. Also I think we r > talking about the same fo

[jira] [Created] (HBASE-14264) Update RAT plugin version in 0.98 branch

2015-08-19 Thread Andrew Purtell (JIRA)
Andrew Purtell created HBASE-14264: -- Summary: Update RAT plugin version in 0.98 branch Key: HBASE-14264 URL: https://issues.apache.org/jira/browse/HBASE-14264 Project: HBase Issue Type: Task

[jira] [Resolved] (HBASE-14225) Exclude **/target/** from RAT checks

2015-08-19 Thread Andrew Purtell (JIRA)
[ https://issues.apache.org/jira/browse/HBASE-14225?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell resolved HBASE-14225. Resolution: Invalid Assignee: (was: Andrew Purtell) > Exclude **/target/** from R

Re: Testing and CI -- Apache Jenkins Builds

2015-08-19 Thread Stack
On Wed, Aug 19, 2015 at 5:24 PM, Stack wrote: > On Wed, Aug 19, 2015 at 5:08 PM, Sean Busbey wrote: > >> I don't recall anything, but I'll go dig through the infra@ list later >> this >> evening to make sure. >> >> Is this on all the H* build hosts? >> >> > Seemingly, yes. > > I can look at infr

Re: Testing and CI -- Apache Jenkins Builds

2015-08-19 Thread Stack
On Wed, Aug 19, 2015 at 5:08 PM, Sean Busbey wrote: > I don't recall anything, but I'll go dig through the infra@ list later > this > evening to make sure. > > Is this on all the H* build hosts? > > Seemingly, yes. I can look at infra changes, no prob. Sean. Was just wondering if anyone knew of

Re: Testing and CI -- Apache Jenkins Builds

2015-08-19 Thread Sean Busbey
I don't recall anything, but I'll go dig through the infra@ list later this evening to make sure. Is this on all the H* build hosts? On Wed, Aug 19, 2015 at 7:05 PM, Stack wrote: > Trunk builds are failing with OOME, can't create native thread > (HBASE-14262) > > Anyone know of any recent INFRA

Re: Testing and CI -- Apache Jenkins Builds

2015-08-19 Thread Stack
Trunk builds are failing with OOME, can't create native thread (HBASE-14262) Anyone know of any recent INFRA changes? Imperfect triage has our problem starting in build #15134, Aug 18, 2015 5:10:02 AM Otherwise, was going to mess with heap sizings and trying to make the big tests smaller so we c

[jira] [Created] (HBASE-14263) ExploringCompactionPolicy logic around file selection is broken

2015-08-19 Thread Vladimir Rodionov (JIRA)
Vladimir Rodionov created HBASE-14263: - Summary: ExploringCompactionPolicy logic around file selection is broken Key: HBASE-14263 URL: https://issues.apache.org/jira/browse/HBASE-14263 Project: HB

[jira] [Created] (HBASE-14262) Big Trunk unit tests failing with "OutOfMemoryError: unable to create new native thread"

2015-08-19 Thread stack (JIRA)
stack created HBASE-14262: - Summary: Big Trunk unit tests failing with "OutOfMemoryError: unable to create new native thread" Key: HBASE-14262 URL: https://issues.apache.org/jira/browse/HBASE-14262 Project: H

Re: HBase and Accumulo

2015-08-19 Thread Ted Malaska
I would say most banks are hbase but there r a few with accumulo. I have most bank, broker dealers and regulators in my region. Also I think we r talking about the same foreign bank ;) Ted Malaska On Aug 19, 2015 7:15 PM, "Jerry He" wrote: > Hi, folks > > Thanks so much for all the responses an

[jira] [Created] (HBASE-14261) Enhance Chaos Monkey framework by adding zookeeper and datanode fault injections.

2015-08-19 Thread Srikanth Srungarapu (JIRA)
Srikanth Srungarapu created HBASE-14261: --- Summary: Enhance Chaos Monkey framework by adding zookeeper and datanode fault injections. Key: HBASE-14261 URL: https://issues.apache.org/jira/browse/HBASE-14261

Re: HBase and Accumulo

2015-08-19 Thread Jerry He
Hi, folks Thanks so much for all the responses and comments. We don't have or support Accumulo yet We support HBase. There have been requests for Accumulo. Like Ted said, almost all from Federal sector and Banks (even foreign banks). They seem to have References or reference implementations for

Re: HBase and Accumulo

2015-08-19 Thread Ted Malaska
That being send what is the use case that u feel you need a nosql solution for? On Aug 19, 2015 6:54 PM, "Ted Malaska" wrote: > I'm on the side of benchmarking for the use case and with an expert. > There a so many ways to cheat a benchmark. And the bench mark may not be > anything like your use

Re: HBase and Accumulo

2015-08-19 Thread Ted Malaska
I'm on the side of benchmarking for the use case and with an expert. There a so many ways to cheat a benchmark. And the bench mark may not be anything like your use case. On Aug 19, 2015 5:43 PM, "Andrew Purtell" wrote: > I think someone who uses third party benchmarks to assess a system like >

Re: building javadoc jars on master

2015-08-19 Thread Andrew Purtell
Yeah I should have just not commented, Sean is right, need to switch on the profiles to get javadoc jars. On Wed, Aug 19, 2015 at 3:45 PM, Andrew Purtell wrote: > IIRC specifying the 'package' goal triggers javadoc. > > > On Wed, Aug 19, 2015 at 12:58 PM, Sean Busbey wrote: > >> to answer my ow

Re: HBase and Accumulo

2015-08-19 Thread Josh Elser
Ah right, I did forgot about that paper. Thanks for clarifying. Big +1 to Andy's comments, too. Jeremy Kepner wrote: Turning off the walog was mostly to shorten the benchmarking cycle (it allowed us to go from zero to peak ingest in a few seconds). BAH got pretty much the same performance resu

Re: building javadoc jars on master

2015-08-19 Thread Andrew Purtell
IIRC specifying the 'package' goal triggers javadoc. On Wed, Aug 19, 2015 at 12:58 PM, Sean Busbey wrote: > to answer my own question: > > mvn -Papache-release -Prelease clean package > > Looks like in our release candidate docs, that means they only get made > during the "stage a maven reposit

Re: HBase and Accumulo

2015-08-19 Thread Andrew Purtell
I think someone who uses third party benchmarks to assess a system like HBase or Accumulo (or Cassandra...) is taking a foolish shortcut, so perhaps we must agree to disagree. On Wed, Aug 19, 2015 at 2:34 PM, Jeremy Kepner wrote: > I agree, that performance on real apps is the most important fo

Re: HBase and Accumulo

2015-08-19 Thread Jeremy Kepner
I agree, that performance on real apps is the most important for any particular organization, but as technologists how do we measure ourselves? Hence imperfect benchmarking remains our only recourse. On Wed, Aug 19, 2015 at 12:34:44PM -0700, Andrew Purtell wrote: > I can't speak for anyone other t

Re: HBase and Accumulo

2015-08-19 Thread Jeremy Kepner
Turning off the walog was mostly to shorten the benchmarking cycle (it allowed us to go from zero to peak ingest in a few seconds). BAH got pretty much the same performance results in their paper, it just took longer for their experiments to run. So, in this case, we had two different teams doing

[jira] [Created] (HBASE-14260) building javadocs takes a long time

2015-08-19 Thread Sean Busbey (JIRA)
Sean Busbey created HBASE-14260: --- Summary: building javadocs takes a long time Key: HBASE-14260 URL: https://issues.apache.org/jira/browse/HBASE-14260 Project: HBase Issue Type: Improvement

Re: building javadoc jars on master

2015-08-19 Thread Sean Busbey
to answer my own question: mvn -Papache-release -Prelease clean package Looks like in our release candidate docs, that means they only get made during the "stage a maven repository" step. (which I wasn't doing before) On Wed, Aug 19, 2015 at 2:43 PM, Sean Busbey wrote: > Can someone save me a

building javadoc jars on master

2015-08-19 Thread Sean Busbey
Can someone save me a good deal of floundering and give me a maven invocation that makes the javadoc jars we publish in a release? I think I have a solution to https://issues.apache.org/jira/browse/HBASE-14251 but can't test it right now. I've gone through the "create a release candidate" steps

[jira] [Created] (HBASE-14259) Backport Namespace quota support to 98 branch

2015-08-19 Thread Vandana Ayyalasomayajula (JIRA)
Vandana Ayyalasomayajula created HBASE-14259: Summary: Backport Namespace quota support to 98 branch Key: HBASE-14259 URL: https://issues.apache.org/jira/browse/HBASE-14259 Project: HBase

Re: HBase and Accumulo

2015-08-19 Thread Ted Malaska
Hbase region splits can be done through a variety of strategies. Data size can be a component in those strategies. There's no hard and fast rule of how large a region can be. There's some tradeoffs with larger or smaller region sizes. A region split strategy will depend upon a number of factors. Me

Re: HBase and Accumulo

2015-08-19 Thread Andrew Purtell
I can't speak for anyone other than myself in the HBase community, but I'm much more interested and focused on performance analysis and developing/deploying for the use cases of my employer than participating in generic bench-marketing to make weapons for happy OSS warriors. Perhaps this does a dis

Re: HBase and Accumulo

2015-08-19 Thread Josh Elser
Alright, I have to ask... are you referring to the paper that cites Accumulo performance without write-ahead logs enabled? I have some serious reservations about the relevance of that paper to this conversation and just want to make sure people aren't led astray by what the actual takeaway shou

Re: HBase and Accumulo

2015-08-19 Thread Christopher
Forgive my ignorance about HBase, but wouldn't size of records count, also? Your response seems to imply that number of records is what matters for how many regions are needed. For what it's worth, Accumulo's tablets are split based on storage size, not number of records. I assumed the same was tru

Re: HBase and Accumulo

2015-08-19 Thread Josh Elser
He didn't ask just about security, FWIW "I am looking for real gap comparing HBase to Accumulo if there is any so that I can be prepared to address them. This is not limited to the security area." Sean Busbey wrote: Let's please stick to the topic Jerry asked about: security features. We ca

Re: HBase and Accumulo

2015-08-19 Thread William Slacum
If you drew a Venn diagram of HBase features compared to Accumulo features, it's pretty much going to be a single circle. If you want performance anecdotes, the most succinct summary I've seen is that Accumulo can handle heavier write loads whereas HBase will handle heavier read loads. From these

Re: HBase and Accumulo

2015-08-19 Thread Sean Busbey
Let's please stick to the topic Jerry asked about: security features. We can get into all sorts of discussions around scalability and read/write performance in a different joint thread if folks want. We all have lots of Opinions (and the YCSB community would love to see more of y'all show up to im

Re: HBase and Accumulo

2015-08-19 Thread Ted Malaska
Sorry Type-o So there might be issues when you pass the Quadrillion. But Like I said never ran into that issue of region limits. On Wed, Aug 19, 2015 at 2:29 PM, Ted Malaska wrote: > Sorry 10 billion a day so that is 7 Trillion records. So many issues > around 1000 Trillion > > On Wed, Aug 19

Re: HBase and Accumulo

2015-08-19 Thread Ted Malaska
Yeah is you have more then a Quadrillion records in you design let me know I would love to help out. Ted Malaska On Wed, Aug 19, 2015 at 2:30 PM, Josh Elser wrote: > Like I've said many times now, it's relative to your actual problem. If > you don't have that much data (or intend to grow into t

Re: HBase and Accumulo

2015-08-19 Thread Josh Elser
Like I've said many times now, it's relative to your actual problem. If you don't have that much data (or intend to grow into that much data), it's not an issue. Obviously, this is the case for you. However, it is an architectural difference between the two projects with known limitations for

Re: HBase and Accumulo

2015-08-19 Thread Ted Malaska
Sorry 10 billion a day so that is 7 Trillion records. So many issues around 1000 Trillion On Wed, Aug 19, 2015 at 2:28 PM, Ted Malaska wrote: > I've been doing HBase for a long time and never had an issue with region > count limits and I have clusters with 10s of billions of records. Many > th

Re: [VOTE] First release candidate for HBase 1.1.2 (RC0) is available

2015-08-19 Thread Josh Elser
Forgot to send out last night (figured I would now despite the -1) Ignoring licensing issues, (non-binding) +1 * Checked sig/xsums * Inspected compatibility report (thanks for including!) * Built source * Ran small+med tests (TestRegionServerHostname failed, passed on rerun) Nick Dimiduk wrote:

Re: HBase and Accumulo

2015-08-19 Thread Ted Malaska
I've been doing HBase for a long time and never had an issue with region count limits and I have clusters with 10s of billions of records. Many there would be issues around a couple Trillion records, but never got that high yet. Ted Malaska On Wed, Aug 19, 2015 at 2:24 PM, Josh Elser wrote: >

Re: HBase and Accumulo

2015-08-19 Thread Josh Elser
Oh, one other thing that I should mention (was prompted off-list). (definition time since cross-list now: HBase regions == Accumulo tablets) Accumulo will handle many more regions than HBase does now due to a splittable metadata table. While I was told this was a very long and arduous journey

Re: HBase and Accumulo

2015-08-19 Thread Josh Elser
Andrew Purtell wrote: Last I tried to play with the cell-level security APIs in HBase, it seemed very obtuse to me. Perhaps I was just dense and didn't find the right sort of instructions. I don't think anyone would debate that cell level security in HBase is a work-in-progress. We'd really wel

Re: HBase and Accumulo

2015-08-19 Thread Sean Busbey
+dev@accumulo (Though Josh and I are on this list, some other folks on dev@accumulo might have opinions) Hi Jerry! Do you have constraints on which version(s) of HBase and Accumulo you're comparing? Are you looking for currently shipping or for some expected future date? In very broad strokes:

Re: HBase and Accumulo

2015-08-19 Thread Andrew Purtell
> Last I tried to play with the cell-level security APIs in HBase, it seemed very obtuse to me. Perhaps I was just dense and didn't find the right sort of instructions. I don't think anyone would debate that cell level security in HBase is a work-in-progress. We'd really welcome your impressions a

Re: HBase and Accumulo

2015-08-19 Thread Josh Elser
IMO, they are very similar. Lots of smart people on both sides making really good changes. I think HBase has a lot more instrumentation and understanding on the resources a cluster will use. For example, I think it's much clearer the resources and threads that the RPC server will use. This is

Re: HBase and Accumulo

2015-08-19 Thread Vladimir Rodionov
Go with lemmings :) Lemmings prefer HBase (Cassandra, actually, but this is not Cassandra vs. Acumulo debate) -Vlad On Wed, Aug 19, 2015 at 10:02 AM, Ted Malaska wrote: > In the end my company supports both Accumulo and HBase so it doesn't matter > to me, but these are things I would consider

[jira] [Created] (HBASE-14258) Make region_mover.rb script case insensitive in regard to hostname

2015-08-19 Thread Vladimir Rodionov (JIRA)
Vladimir Rodionov created HBASE-14258: - Summary: Make region_mover.rb script case insensitive in regard to hostname Key: HBASE-14258 URL: https://issues.apache.org/jira/browse/HBASE-14258 Project:

Re: HBase and Accumulo

2015-08-19 Thread Ted Malaska
In the end my company supports both Accumulo and HBase so it doesn't matter to me, but these are things I would consider - I would think security should no longer be a factor. - Also be very careful when looking at benchmarks for I have found that some are bais ether way. If you want to do a real

HBase and Accumulo

2015-08-19 Thread Jerry He
Hi, folks We have people that are evaluating HBase vs Accumulo. Security is an important factor. But I think after the Cell security was added in HBase, there is no more real gap compared to Accumulo. I know we have both HBase and Accumulo experts on this list. Could someone shred more light? I

[jira] [Created] (HBASE-14257) Periodic flusher only handles hbase:meta, not other system tables

2015-08-19 Thread Lars George (JIRA)
Lars George created HBASE-14257: --- Summary: Periodic flusher only handles hbase:meta, not other system tables Key: HBASE-14257 URL: https://issues.apache.org/jira/browse/HBASE-14257 Project: HBase

[jira] [Created] (HBASE-14256) Flush task message may be confusing when region is recovered

2015-08-19 Thread Lars George (JIRA)
Lars George created HBASE-14256: --- Summary: Flush task message may be confusing when region is recovered Key: HBASE-14256 URL: https://issues.apache.org/jira/browse/HBASE-14256 Project: HBase I

[jira] [Created] (HBASE-14255) Simplify Cell creation post 1.0

2015-08-19 Thread Lars George (JIRA)
Lars George created HBASE-14255: --- Summary: Simplify Cell creation post 1.0 Key: HBASE-14255 URL: https://issues.apache.org/jira/browse/HBASE-14255 Project: HBase Issue Type: Improvement

[jira] [Created] (HBASE-14254) Wrong error message when throwing NamespaceNotFoundException in shell

2015-08-19 Thread Liu Shaohui (JIRA)
Liu Shaohui created HBASE-14254: --- Summary: Wrong error message when throwing NamespaceNotFoundException in shell Key: HBASE-14254 URL: https://issues.apache.org/jira/browse/HBASE-14254 Project: HBase