Re: using stateful iterators to do filtering during major compaction?

2017-04-10 Thread Adam Fuchs
Hi Jonathan, Did you consider simply not using the whole row iterator in the compaction scope? Iterators don't get torn down and rebuilt in the compaction scope, so partial row reads as you describe in the scan scope don't really exist in compaction scopes unless another iterator above your

Re: bulk load architecture

2016-08-18 Thread Adam Fuchs
y has any > thoughts on Adam's questions. > > On Mon, Aug 15, 2016 at 1:49 PM Adam Fuchs <afu...@apache.org> wrote: > > > I've been looking through the bulk load code lately related to some > > performance issues a customer of ours is experiencing, and I'm perplex

bulk load architecture

2016-08-15 Thread Adam Fuchs
I've been looking through the bulk load code lately related to some performance issues a customer of ours is experiencing, and I'm perplexed by a couple of things. Between o.a.a.master.tableOps.LoadFiles and o.a.a.server.client.BulkImporter we have 4 thread pools that are used in bulk load. It

Re: Snappy as default table.file.compress.type?

2016-08-15 Thread Adam Fuchs
t; disagree on that point. Thanks for clarifying your opinions on the topic. > > Adam Fuchs wrote: > >> If the crux of your argument was that snappy is always a better choice, >> then my retort was to say it is not, since sometimes compression ratio can >> be a dominant factor. C

Re: Snappy as default table.file.compress.type?

2016-08-13 Thread Adam Fuchs
houldn't use the > faster compression algorithm. > > I don't mean to be snarky, but your argument is shutting down conversation. > I appreciate you sharing the opinion but don't feel like it's encouraging > discussion. > > On Aug 13, 2016 11:18 PM, "Adam Fuchs" <afu..

Re: Snappy as default table.file.compress.type?

2016-08-13 Thread Adam Fuchs
In my experience gz gets roughly 1.5x to 2x better compression than snappy. Snappy is definitely not a pareto improvement (although we tend to use snappy by default). Since it's not always better I think you would need a more solid argument to change the default. Adam On Aug 13, 2016 8:06 PM,

Re: Accumulo folks at Hadoop Summit San Jose

2016-05-19 Thread Adam Fuchs
I'll be there. Adam On Thu, May 19, 2016 at 11:01 AM, Josh Elser wrote: > Out of curiosity, are there going to be any Accumulo-folks at Hadoop > Summit in San Jose, CA at the end of June? > > - Josh >

Re: Review Request 43957: ACCUMULO-1755: removed synchronized modifier from TabletServerBatchWriterstartProcessing()

2016-02-25 Thread Adam Fuchs
n have startProcessing() return a Runnable that then gets executed outside of a synchronized block in addMutation? - Adam Fuchs On Feb. 24, 2016, 11:10 p.m., Dave Marion wrote: > > --- > This is an automatically generated e-mail. To reply,

Re: total table rows

2015-11-09 Thread Adam Fuchs
+1 on a standard counting iterator. I have written this for tests way too many times. :) ACCUMULO-4049. Adam On Mon, Nov 9, 2015 at 10:52 AM, Josh Elser wrote: > Note that CountingIterator is in the system iterator package > (FirstEntryInRowIterator also isn't in the

Re: [DISCUSS] What to do about encryption at rest?

2015-11-05 Thread Adam Fuchs
then define common access patterns for > applications > > > that want to use multiple keys for encryption. > > > > > > > > > > > > On Wed, Nov 4, 2015 at 8:10 PM, Adam Fuchs<afu...@apache.org> wrote: > > > > > >> Bill, > >

Re: [DISCUSS] What to do about encryption at rest?

2015-11-05 Thread Adam Fuchs
ping between locality groups and column families (as well as an extra > mapping to files). > > Did I miss anything? > > On Thu, Nov 5, 2015 at 1:27 PM, Adam Fuchs <afu...@apache.org> wrote: > > > Camps two and three are the same camp, really. If we can identify a clear > >

Re: [DISCUSS] What to do about encryption at rest?

2015-11-04 Thread Adam Fuchs
ncrypted-HDFS-encrypted-RFile? Wondering if there are any > > > unexpected gotchas w/ this. > > > > > > > I was discussing my questions w/ Christopher today and he mentioned an > > experiment that I thought was interesting. What is the random seek > >

Re: [DISCUSS] What to do about encryption at rest?

2015-11-01 Thread Adam Fuchs
There's another way to look at the state of Accumulo's encryption at rest: 1. Encryption at rest works great for what it does, and the code being "at rest" isn't necessarily a problem 2. Several organizations are using Accumulo's encryption at rest effectively in operations 3. Encryption at rest

Re: [DISCUSS] What to do about encryption at rest?

2015-11-01 Thread Adam Fuchs
4. Has Accumulo's been evaluated for security and performance? By whom? Is > it published? Yes, there have been several talks at meetups and conferences that discuss the security and performance of the current solution. > > On Sun, Nov 1, 2015, 08:55 Adam Fuchs <afu...@apache.org> wrote: &

Re: new committers!

2015-10-07 Thread Adam Fuchs
Welcome, Dylan and Russ! Glad to have you both aboard. Adam On Oct 7, 2015 2:39 PM, "Billie Rinaldi" wrote: > Join me in welcoming Dylan Hutchison and Russ Weeks as new Apache Accumulo > committers and PMC members! Dylan and Russ, feel free to tell us about > yourselves and

Re: [VOTE] Accumulo 1.5.4-rc1

2015-09-10 Thread Adam Fuchs
I agree with Chris on this one. I don't see the pre-existing licensing/copyright notification issue as a blocker to a bugfix release. Definitely something we should fix as soon as we can, though. Sorry I can't give a +1 to the release -- I haven't done a thorough enough review to officially vote.

Re: table size questions

2015-09-08 Thread Adam Fuchs
Accumulo sorts keys and then compresses them in blocks. Each block is compressed without any information external to the block. You're going to see different compression ratios depending on the relative entropy of keys inside of each compressed block. For purpose of discussion, let's simplify this

Re: exception thrown during minor compaction

2015-09-01 Thread Adam Fuchs
In your case, there shouldn't be a delete marker unless you're explicitly writing one. The tricky thing about deletes in a summing combiner is that sums and deletes together are not commutative, and combiners require associativity and commutativity. If I have three operations: add 1 to x, delete

Re: using combiner vs. building stats cache

2015-08-28 Thread Adam Fuchs
Calling flush after every write will probably slow you down by more than 1000x, since the flush call is on the order of 10-100ms. Keeping a buffer of data at your client and only flushing when the buffer is full is usually a pretty decent strategy. That way you can replay from the buffer in case

Re: Thrift as Public API?

2015-06-04 Thread Adam Fuchs
IMO we do not. The thrift interface that we currently have is not easily consumable (you have to understand a lot about the metadata and cluster layout), and we don't include it in SemVer. Additional features like GetMasterStats should be plumbed through to the Java API if we want to expose them.

Re: Scan-time iterators returning out-of-order rows

2015-04-13 Thread Adam Fuchs
Russ, Sorry for being a bit late to the discussion here. I think it's technically fine for scan-time iterators to return keys out of order, but there are a few things you should be aware of: 1. Returning keys outside of the current row can cause Accumulo to skip over data without your knowledge,

Re: [VOTE] Establishing a contrib repo for upgrade testing

2015-03-10 Thread Adam Fuchs
+1 Adam On Mar 10, 2015 2:48 AM, Sean Busbey bus...@cloudera.com wrote: Hi Accumulo! This is the VOTE thread following our DISCUSS thread on establishing a new contrib for upgrade testing. For more details, please see the prior DISCUSS thread on this topic[1]. Cloudera has recently made

Re: [DISCUSS] Establishing a contrib repo for upgrade testing

2015-03-06 Thread Adam Fuchs
I've seen some of the problems this has found, and I'm all for it! Are there opportunities to share this project with other projects? Would this be something that starts out as a contrib and then graduates to top level project over time? Adam On Fri, Mar 6, 2015 at 10:15 AM, Sean Busbey

Re: Google analytics summaries

2014-12-11 Thread Adam Fuchs
I'm for it. Also, does anyone know how I get access to the google analytics reports page for our site? Thanks, Adam On Thu, Dec 11, 2014 at 11:14 AM, Sean Busbey bus...@cloudera.com wrote: I just got the mail summary of our google analytics stats for accumulo.apache.org. Anyone opposed to

Re: Official Guidance to users regarding removal of functions

2014-12-11 Thread Adam Fuchs
Has anybody looked into separating the public API a bit more from the core? It seems to me that a large number of the deprecation removal issues are related to people failing to read section 9 of the README. It would be great if we built an API jar that people could build against, but didn't leak

Re: Official Guidance to users regarding removal of functions

2014-12-11 Thread Adam Fuchs
Awesome -- ACCUMULO-2589 gets us at least halfway there. Given this, what would be the challenges in having and maintaining one API project for each major version ever released? Adam On Thu, Dec 11, 2014 at 2:24 PM, Josh Elser josh.el...@gmail.com wrote: Adam Fuchs wrote: Has anybody looked

Re: [VOTE] adoption of semver

2014-12-09 Thread Adam Fuchs
+1 for semver version 2.0.0 Assuming this passes, let's make sure we grab a copy for posterity and post it on our site. Adam On Tue, Dec 9, 2014 at 1:23 PM, Billie Rinaldi bil...@apache.org wrote: I would like to call a vote on adopting semantic versioning ( http://semver.org/) for future

Re: [VOTE] adoption of semver

2014-12-09 Thread Adam Fuchs
Yup, all future releases should use semver version 2.0.0. Adam On Tue, Dec 9, 2014 at 3:59 PM, John Vines vi...@apache.org wrote: Adam, the vote isn't for 2.0.0, it's for all future releases. Can you please clarify your vote? On Tue, Dec 9, 2014 at 3:40 PM, Adam Fuchs afu...@apache.org wrote

Re: Updated Accumulo Benchmarking Paper

2014-11-24 Thread Adam Fuchs
Looks good to me. Another option would be to put multiple links under one entry. That might be a bit cleaner. Adam On Sun, Nov 23, 2014 at 10:15 PM, Josh Elser josh.el...@gmail.com wrote: +1 publish that sucker. Thanks for updating the site with the new paper. Drew Farris wrote: Ok,

Re: C++ accumulo client -- native clients for Python, Go, Ruby etc

2014-10-08 Thread Adam Fuchs
John, Do you have any performance numbers that you can share around your use of the existing proxy solution? One of the reasons that Thrift is performant for Accumulo is that messages are batched by the client library and sent over a smaller number of RPCs. A C++ client will also need to have

Re: Deprecation removal for 1.7.0

2014-10-08 Thread Adam Fuchs
On Tue, Oct 7, 2014 at 3:52 PM, Josh Elser josh.el...@gmail.com wrote: I like the idea of leaving deprecated things until the next major release (2.0.0 in this case). It would encourage us to choose carefully what APIs we add :) +1 Despite the fact that certain people (ahem Josh) are not

Re: Deprecation removal for 1.7.0

2014-10-08 Thread Adam Fuchs
What's the right level of review? Should we have a public announcement board of some sort on the website, or is a request for comment on the user list sufficient? On Wed, Oct 8, 2014 at 5:35 PM, Jeremy Kepner kep...@ll.mit.edu wrote: Perhaps the process should be changed to require review prior

Re: Deprecation removal for 1.7.0

2014-10-06 Thread Adam Fuchs
So, I think we can make a general argument to set policy, and when removing a specific method we should make a specific argument. Personally, I would set the bar at identifying the specific harm cause by the retention of the method, as well as polling the community and considering objections.

Re: Review Request 24855: ACCUMULO-1454 design doc

2014-08-19 Thread Adam Fuchs
On Tue, Aug 19, 2014 at 4:17 PM, ke...@deenlo.com wrote: ... Eric looked into locality once when running continuous ingest and found that ~50% of tablets had local data.This matches expectations as the default balancer will try to migrate one child after a split. I thought he found it

Re: Column size limit

2014-08-18 Thread Adam Fuchs
Joe, I would say that a rule of thumb would be tens of megabytes for a single cell. There are two limits that affect this: 1) Amount of memory used: This includes ingesting into the batchwriter, buffering in the in-memory maps, scanning RFiles, and preparing query responses. At any given point,

Re: web master?

2014-07-21 Thread Adam Fuchs
- Kepner et al ICASSP 2012 Paper: http://www.mit.edu/~kepner/pubs/Kepner_2012_D4M_Paper.pdf Slides: http://www.mit.edu/~kepner/pubs/Kepner_2012_D4M_Slides.pdf On Jul 14, 2014, at 5:24 PM, Adam Fuchs afu...@apache.org wrote: Jeremy, If you send me a list of links to papers I would be happy

Re: web master?

2014-07-21 Thread Adam Fuchs
://www.datatables.net/ On 7/21/14, 11:22 AM, Adam Fuchs wrote: I put up a new table of papers and presentations as sort of a prototype format. Comments? http://accumulo.apache.org/papers.html Still to be done: make it sortable, look into better content management strategies, experiment with tags Does

Re: web master?

2014-07-21 Thread Adam Fuchs
Found it! It was missing dataTables.bootstrap.js. Adam On Mon, Jul 21, 2014 at 2:24 PM, Josh Elser josh.el...@gmail.com wrote: On 7/21/14, 2:14 PM, Keith Turner wrote: On Mon, Jul 21, 2014 at 2:10 PM, Adam Fuchsafu...@apache.org wrote: Thanks Josh, a new version with datatables is now

Re: web master?

2014-07-14 Thread Adam Fuchs
Jeremy, If you send me a list of links to papers I would be happy to put them up. Also, if you (and anyone else) have a model that you like for organizing that page send me a link and we can work on that too. Given the amount of content that's out there now it probably makes sense to spend some

Re: [VOTE] Website update

2014-04-15 Thread Adam Fuchs
-0 In general, I really like the look and feel of the new site. However, it has some serious issues when viewed on small windows, like mobile devices. These things start to be an issue at ~800 pixels width: * Menu bar across the top takes up a significant portion of the screen real estate and

Re: [VOTE] Website update

2014-04-15 Thread Adam Fuchs
. Bootstrap is designed for handling mobile displays, so there should be easy ways to improve the view. On Tue, Apr 15, 2014 at 5:09 PM, Adam Fuchs afu...@apache.org wrote: -0 In general, I really like the look and feel of the new site. However, it has some serious issues when

Re: [VOTE] Define CTR in Bylaws

2014-04-07 Thread Adam Fuchs
+1 Adam On Mon, Apr 7, 2014 at 10:11 AM, Billie Rinaldi bil...@apache.org wrote: Please vote on applying the following changes to the Accumulo bylaws ( http://accumulo.apache.org/bylaws.html). If the Code Change action has been removed, it will be reintroduced along with these changes.

Re: Which is stable?

2014-04-07 Thread Adam Fuchs
With the stable tag, I think there is a perception that we might do experimental or dev releases like many other open source projects do. Since we don't do that, maybe we should drop the tags to reduce our web page maintenance and avoid confusion. I think latest is always stable for us unless

Re: [VOTE] Apache Accumulo 1.5.1-RC3

2014-02-25 Thread Adam Fuchs
Thanks for running that checker, Keith. Should we not be worried about the removal of InputFormatBase.RangeInputSplit? If I read correctly this will break both binary (runtime) compatibility and code (compile-time) compatibility. Can somebody make an argument for why this is not a problem in a

Re: [VOTE] Apache Accumulo 1.5.1-RC3

2014-02-25 Thread Adam Fuchs
, Adam Fuchs afu...@apache.org wrote: Thanks for running that checker, Keith. Should we not be worried about the removal of InputFormatBase.RangeInputSplit? If I read correctly this will break both binary (runtime) compatibility and code (compile-time) compatibility. Can somebody make

Re: Rack and Datacenter Awareness

2014-01-21 Thread Adam Fuchs
Accumulo also has a pluggable load balancer that could be used to balance with rack and data-center awareness. This can be done using the currently released versions of Accumulo. What problem are you trying to solve, exactly? Are you concerned about durability and availability issues like

Re: Review Request 15846: Prototype implementation of ACCUMULO-1787.

2013-11-25 Thread Adam Fuchs
be done if the alternative compression type is enabled. If we default the threshold to 0B and flip the to a that would be sufficient. - Adam Fuchs On Nov. 25, 2013, 9:37 p.m., Chris McCubbin wrote: --- This is an automatically generated

Re: Provided dependencies

2013-11-06 Thread Adam Fuchs
On Nov 6, 2013 6:04 PM, Joey Echeverria joey...@clouderagovt.com wrote: ... If I depend on Accumulo in my maven project, then I shouldn't need to depend on Hadoop unless the APIs I'm using leak that dependency or I have an explicit dependency on Hadoop elsewhere. We currently leak the Text

Re: Table entry count confusion

2013-10-02 Thread Adam Fuchs
The count that displays in the monitor is the sum of all the key/value pairs that are in the files that back Accumulo. You can also get this count by doing a scan of the !METADATA table and looking at the values associated with keys in the file column family. Inserting the same key twice could

broken svn links on accumulo site

2013-09-30 Thread Adam Fuchs
Folks, I just fixed a couple of links to the CHANGES files for 1.5.0 and 1.3.6 that referenced the old svn instead of the new git. I'm trying to do a quick evaluation of whether there might be any other broken links on our site, and I came across some files in

Re: [VOTE] 1.6.0 Feature freeze.

2013-09-28 Thread Adam Fuchs
+1 On Sep 27, 2013 1:40 PM, John Vines vi...@apache.org wrote: Please vote on a feature freeze date of Nov 1 23:59 PDT for the master branch. Shortly after this time we will branch 1.6.0-SNAPSHOT from master and increment the version in master. Feature Freeze means only bug fixes and

Re: Backporting policy proposal

2013-06-18 Thread Adam Fuchs
On Jun 17, 2013 1:07 PM, Christopher ctubb...@apache.org wrote: I propose we adopt a more structured policy beyond simple lazy consensus to be apply to backporting features. Some guidelines I'd like to see in this policy, include: 1. Back-porting bugfixes to a prior release line that is not

Re: Backporting policy proposal

2013-06-18 Thread Adam Fuchs
On Jun 18, 2013 11:05 AM, Josh Elser josh.el...@gmail.com wrote: On 6/18/13 10:53 AM, Adam Fuchs wrote: 2. Back-porting performance improvements to a prior release line that is not EOL (end-of-life) is usually okay (subject to normal lazy consensus), so long as it does not change user

Re: [VOTE] JDK 1.7 - Switch for Accumulo 1.6.0

2013-06-03 Thread Adam Fuchs
Let's be specific: which Java-1.7-only feature is motivating this change? Without knowing the upside I am inclined to vote no because of the number of platforms that I regularly see that are running java 1.6. Adam On Mon, Jun 3, 2013 at 4:51 PM, Christopher ctubb...@apache.org wrote: Given

Re: 1.6 discussion

2013-05-29 Thread Adam Fuchs
I second 1430 EDT on Wednesday. Adam On Wed, May 29, 2013 at 1:53 PM, Billie Rinaldi billie.rina...@gmail.comwrote: I can't do it then. I counter-propose Wednesday at 1430 ET. Two-ish hours sounds reasonable. Billie On Wed, May 29, 2013 at 10:46 AM, Adam Fuchs afu...@apache.org wrote

Re: Congratulations!

2013-05-28 Thread Adam Fuchs
Andres, That's great news! Please let me know if there's anything I or anyone else at Sqrrl can do to help. Cheers, Adam On Mon, May 27, 2013 at 3:34 PM, Andres Danter adan...@gmail.com wrote: Hello all, Just wanted to inform you all that my proposal to add support for Accumulo to

Re: 1.6 discussion

2013-05-28 Thread Adam Fuchs
I'm interested. Tuesday's no good for me, but my other days are fairly flexible. Adam On Tue, May 28, 2013 at 12:37 PM, Billie Rinaldi billie.rina...@gmail.comwrote: Would anyone be interested in getting together next week in person or via hangout to discuss 1.6 features? If we have such

Re: hadoop-2.0 incompatibility

2013-05-21 Thread Adam Fuchs
We still have the option of putting out a separate build for 1.5.0 compatibility with hadoop 2. Should we vote on that release separately? Seems like it should be easy to add more binary packages that correspond to the same source release, even after the initial vote. Adam On Tue, May 21, 2013

Re: Is C++ code still part of 1.5 release?

2013-05-17 Thread Adam Fuchs
Folks, Sorry to be late to the party, but did we come to a consensus on this? Seems like we still have opinions both ways as to whether the cpp code should be packaged with the binary distribution. I would argue that cpp code is a special case, since the build is so platform dependent. It's

Re: Is C++ code still part of 1.5 release?

2013-05-17 Thread Adam Fuchs
L Tubbs II http://gravatar.com/ctubbsii On Fri, May 17, 2013 at 10:49 AM, Billie Rinaldi billie.rina...@gmail.com wrote: On Fri, May 17, 2013 at 7:26 AM, Adam Fuchs afu...@apache.org wrote: Folks, Sorry to be late to the party, but did we come to a consensus on this? Seems like we

Re: Is C++ code still part of 1.5 release?

2013-05-17 Thread Adam Fuchs
a no brainer, but I have yet to here a reason that is not 'no source code in a binary release!' On Fri, May 17, 2013 at 12:11 PM, Adam Fuchs afu...@apache.org wrote: Just to solidify the decision that Chris is already leaning towards, let me try to clarify my position: 1

Re: Hadoop 2 compatibility issues

2013-05-16 Thread Adam Fuchs
before then... or, by the next RC if RC3 fails to pass a vote. -- Christopher L Tubbs II http://gravatar.com/ctubbsii On Wed, May 15, 2013 at 5:31 PM, Adam Fuchs afu...@apache.org wrote: It seems like the ideal option would be to have one binary build that determines

Re: Hadoop 2 compatibility issues

2013-05-15 Thread Adam Fuchs
It seems like the ideal option would be to have one binary build that determines Hadoop version and switches appropriately at runtime. Has anyone attempted to do this yet, and do we have an enumeration of the places in Accumulo code where the incompatibilities show up? One of the

Re: Hadoop 2 compatibility issues

2013-05-14 Thread Adam Fuchs
I tend to agree with Sean, John, and Benson. Option 4 works for now, and until we can define something that works better (e.g. runtime compatibility with both hadoop 1 and 2 using reflection and crazy class loaders) we should not delay the release. Good docs are always helpful where engineering is

Re: New committer!

2013-05-13 Thread Adam Fuchs
Welcome aboard, Corey! Adam On May 11, 2013 11:00 AM, Billie Rinaldi bil...@apache.org wrote: I am pleased to announce that Corey Nolet has been voted to become a new committer for Apache Accumulo. Welcome, Corey! Feel free to say a few words about your development interests. Billie

Re: Geographic Plot of Accumulo Downloads

2013-05-03 Thread Adam Fuchs
? On Fri, May 3, 2013 at 10:35 AM, Adam Fuchs afu...@apache.org wrote: A link should work better: http://people.apache.org/~afuchs/accumulo_visits_20130502.png Cheers, Adam On Fri, May 3, 2013 at 10:07 AM, Eric Newton eric.new...@gmail.com wrote: Attachments are being stripped

Re: Geographic Plot of Accumulo Downloads

2013-05-03 Thread Adam Fuchs
Visits. Adam On Fri, May 3, 2013 at 2:58 PM, Mike Drob md...@mdrob.com wrote: Is it visits or visitors? On Fri, May 3, 2013 at 12:16 PM, Adam Fuchs afu...@apache.org wrote: This covers about the last year: all time since Accumulo became a top-level project. The metric is visits

Re: asciidoc manual

2013-05-01 Thread Adam Fuchs
This looks good to me. Out of curiosity, what made you decide to use asciidoc rather than the multitude of other options? Cross-platform support looks good (support for Windows, Mac, and Linux). There also seems to be a big enough user base for long term supportability. A few things I noticed: 1.

Re: multi-table isolated batch scanner

2013-04-15 Thread Adam Fuchs
: How are you expecting to get results back? Guava's Iterables could concat a bunch of a Scanners together, if you didn't care about the throughput aspect of it and simply wanted results from multiple tables. On Mon, Apr 15, 2013 at 3:00 PM, Adam Fuchs afu...@apache.org wrote: Is anyone else

Re: multi-table isolated batch scanner

2013-04-15 Thread Adam Fuchs
:06 PM, Adam Fuchs afu...@apache.org wrote: Chris, The desire for isolation stems from the desire to amortize some computation over a number of results. Say it takes 5 seconds to compute an intersection Would increasing the size of the key/value buffer help in your case? The iterator

Re: [ANNOUNCE] New committer : Brian Loss

2013-02-13 Thread Adam Fuchs
Welcome Brian, glad to have you officially aboard! Adam On Wed, Feb 13, 2013 at 12:27 PM, Keith Turner ke...@deenlo.com wrote: I am pleased to announce that Brian Loss has been voted to become a new committer for Apache Accumulo. Welcome, Brian! Feel free to say a few words about your

Re: ACCUMULO-958 - Pluggable encryption in walogs

2013-01-30 Thread Adam Fuchs
Josh, Mike Allen is still working on ACCUMULO-958, and will have an updated patch in the next couple of weeks. We were hoping to get the more complete encryption strategy into 1.5, but were not able to complete it by feature freeze. However, the WAL encryption as is, when configured with the

Re: ACCUMULO-958 - Pluggable encryption in walogs

2013-01-30 Thread Adam Fuchs
Josh, Mike Allen is still working on ACCUMULO-958, and will have an updated patch in the next couple of weeks. We were hoping to get the more complete encryption strategy into 1.5, but were not able to complete it by feature freeze. However, the WAL encryption as is, when configured with the

Re: ACCUMULO-958 - Pluggable encryption in walogs

2013-01-30 Thread Adam Fuchs
Let me attempt to make another argument for why the 958 patch should be included in 1.5.0. What this patch represents is not an encryption solution for WAL, but an experimental extension point that will be used for building an encryption solution as a pluggable module. We need to judge its merit

Re: ACCUMULO-958 - Pluggable encryption in walogs

2013-01-30 Thread Adam Fuchs
not exist yet, the code is freely available to be modified, distributed and open to public review. Adam, I strongly disagree that forking the code is bad, considering the progress that other projects make specifically because they have experimental forks (HBase). On Wed, Jan 30, 2013 at 10:40 AM, Adam

Re: Accumulo 1.6 and beyond feature summit

2013-01-29 Thread Adam Fuchs
PM, Adam Fuchs afu...@apache.org wrote: Folks, Now that we're past feature freeze for 1.5, I'd like to set up a discussion of the features that Accumulo developers are interested in for 1.6 and beyond. The purpose of this discussion will be to raise awareness of projects that you think

Accumulo 1.6 and beyond feature summit

2013-01-26 Thread Adam Fuchs
Folks, Now that we're past feature freeze for 1.5, I'd like to set up a discussion of the features that Accumulo developers are interested in for 1.6 and beyond. The purpose of this discussion will be to raise awareness of projects that you think are cool and important, and to rally the

Re: Extend code freeze for a few days?

2013-01-21 Thread Adam Fuchs
consider quality control) commits? -- Christopher L Tubbs II http://gravatar.com/ctubbsii On Mon, Jan 21, 2013 at 4:26 PM, Adam Fuchs afu...@apache.org wrote: Also a big +1 for Friday. By the way, as much as I would like to we can't blame this one on John. He's largely asking on my

Re: Releasing 1.5.0

2012-11-21 Thread Adam Fuchs
+1 My wishlist includes: +Hadoop 2.0 support +kerberos support +generic authentication/authorization plugin infrastructure +thrift proxy +time and visibility block indexing in RFile +HDFS classloader +Tested compatibility with MapR, HDP, and CDH Other nice-to-haves would be: +Memory-to-memory

Re: Q: re Accumulo Commit Policy? (was Re: Setting charset in getBytes())

2012-10-31 Thread Adam Fuchs
I think the core policy should be if you think your change is at all likely to be rolled back then don't commit it. This applies to tickets with active debates. I also don't think we need to be heavy handed in policy -- shame of roll back is enough motivation and the cost isn't that high. This

Re: Q: re Accumulo Commit Policy? (was Re: Setting charset in getBytes())

2012-10-31 Thread Adam Fuchs
...@gmail.comwrote: On Wed, Oct 31, 2012 at 12:18 PM, Adam Fuchs afu...@apache.org wrote: I think the core policy should be if you think your change is at all likely to be rolled back then don't commit it. This applies to tickets with active debates. I also don't think we need

Re: Setting Charset in getBytes() call.

2012-10-30 Thread Adam Fuchs
On Mon, Oct 29, 2012 at 9:22 PM, Drew Farris d...@apache.org wrote: I have always wondered if there were cases in the API where users are forced to use Text when they would otherwise prefer byte[], e.g: stuffing a non utf8 byte[] into a Text object to facilitate storage or sorting. Not

new contributors recognized

2012-10-26 Thread Adam Fuchs
Folks, I did a quick search to see who has posted patches on ACCUMULO tickets in the last few months, and I found that we have several new names to add to the contributors list. Thanks to the following people for their contributions: Chris McCubbin Laura Peaslee Luke Brassard Phil Eberhardt You

Re: Release 1.4.2

2012-10-09 Thread Adam Fuchs
+1 I'll take a pass at the tickets tonight to see what I can see. Adam On Oct 9, 2012 6:56 PM, Keith Turner ke...@deenlo.com wrote: I think we should consider releasing 1.4.2 to make fixes for ACCUMULO-779, ACCUMULO-766, and ACCUMULO-409 available. Thoughts? Are there other fixes people

Re: Intermittent error connecting to Accumulo Instance

2012-09-11 Thread Adam Fuchs
Duane, The INVALID_INSTANCEID message implies that your client is trying to contact a server that thinks the instance id is something else. The 1 out of 5 likelihood probably indicates that one out of the five server processes (one master and four tablet servers) has some misconfiguration. When

Re: What's going on?

2012-09-06 Thread Adam Fuchs
+1! Adam On Sep 6, 2012 9:59 PM, Jim Klucar klu...@gmail.com wrote: Did Chris get hacked or did somebody lock him in a room with a laptop, a pizza, and case of red bull?

Re: complement in security label expression?

2012-09-01 Thread Adam Fuchs
Hi Rob, I think that a complement operator would fit naturally into Accumulo's cell-level security mechanism. The computation wouldn't be any harder, and the language wouldn't be significantly more complex. This would be a great project for you to take on. Please let us know if you need any

Re: [jira] [Commented] (ACCUMULO-739) Please add dot (.) as a valid character in column visibility tokens

2012-08-28 Thread Adam Fuchs
I think we should reserve single quotes, double quotes, tilde, and backslash in addition to the parenthesis, ampersand and pipe that are already used. If we need more special characters in the future we have the option of using backslash to change their meaning. For example, we could later decide

[jira] [Updated] (ACCUMULO-729) refactor core to reduce client application dependency version conflicts

2012-08-15 Thread Adam Fuchs (JIRA)
[ https://issues.apache.org/jira/browse/ACCUMULO-729?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Adam Fuchs updated ACCUMULO-729: Description: Accumulo client applications often include things like thrift servers

[jira] [Created] (ACCUMULO-729) refactor core to reduce client application dependency version conflicts

2012-08-14 Thread Adam Fuchs (JIRA)
Adam Fuchs created ACCUMULO-729: --- Summary: refactor core to reduce client application dependency version conflicts Key: ACCUMULO-729 URL: https://issues.apache.org/jira/browse/ACCUMULO-729 Project

Re: PMC Membership

2012-08-01 Thread Adam Fuchs
Josh, One of the tasks we have not yet taken on is to write bylaws for the Accumulo project. This is where things like PMC membership, subprojects, committer requirements, etc. are found. Here's an example: http://hadoop.apache.org/bylaws.html Cheers, Adam On Wed, Aug 1, 2012 at 2:07 PM, Josh

Re: Python client lib for Accumulo?

2012-07-27 Thread Adam Fuchs
One of the big challenges of connecting directly to the existing thrift services is that there is a lot of logic imbedded in the Java client libraries that would have to be recreated. This includes things like finding tablets, managing multiple connections, handling tablet migration, handling read

Re: Embedded error: /home/medined/workspace/accumulo_trunk/examples/wikisearch/ingest/target/classes (Is a directory)

2012-07-25 Thread Adam Fuchs
The general symptom is that mvn clean compile doesn't always work, but mvn clean package does. This is because the dependencies between Accumulo sub-projects require jars to exist rather than just compiled classes. The root cause seems to be our current configuration of the maven-dependency-plugin

[jira] [Commented] (ACCUMULO-697) Break Scanner parameterization from Key,Value to Key,{Something}

2012-07-21 Thread Adam Fuchs (JIRA)
[ https://issues.apache.org/jira/browse/ACCUMULO-697?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13419922#comment-13419922 ] Adam Fuchs commented on ACCUMULO-697: - I like the concept, but does this go far

[jira] [Commented] (ACCUMULO-429) Clean up startup scripts

2012-07-20 Thread Adam Fuchs (JIRA)
[ https://issues.apache.org/jira/browse/ACCUMULO-429?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13419513#comment-13419513 ] Adam Fuchs commented on ACCUMULO-429: - In the postinstall for debian packaging, we

[jira] [Commented] (ACCUMULO-600) Move wikisearch example to contrib

2012-07-19 Thread Adam Fuchs (JIRA)
[ https://issues.apache.org/jira/browse/ACCUMULO-600?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13418849#comment-13418849 ] Adam Fuchs commented on ACCUMULO-600: - That would also be a good reason to keep

[jira] [Updated] (ACCUMULO-652) support block-based filtering within RFile

2012-07-17 Thread Adam Fuchs (JIRA)
[ https://issues.apache.org/jira/browse/ACCUMULO-652?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Adam Fuchs updated ACCUMULO-652: Fix Version/s: 1.5.0 support block-based filtering within RFile

Re: change to jira notifications

2012-07-13 Thread Adam Fuchs
Todd, The big problem we're trying to solve is that we want the reply-to for jira messages to go to j...@apache.org so that email replies automatically get posted as comments. It seems we can only do this for messages posted to the new notifications list. Do you know how to solve both problems at

[jira] [Commented] (ACCUMULO-683) Accumulo ignores HDFS max replication configuration

2012-07-10 Thread Adam Fuchs (JIRA)
[ https://issues.apache.org/jira/browse/ACCUMULO-683?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13410436#comment-13410436 ] Adam Fuchs commented on ACCUMULO-683: - A workaround should be to set

[jira] [Commented] (ACCUMULO-652) support block-based filtering within RFile

2012-07-09 Thread Adam Fuchs (JIRA)
[ https://issues.apache.org/jira/browse/ACCUMULO-652?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13409808#comment-13409808 ] Adam Fuchs commented on ACCUMULO-652: - We uncovered another tricky point today

  1   2   >