Re: [DISCUSS] Would a visibility histogram on a table be harmful?

2016-10-12 Thread Marc P.
What prevents the owners of the system from doing this in their own table? Keeping track of that information is a use case of Accumulo. I think this may be an example of external code that the user must install. Placing the onus on the consumer mitigates concern that Mike "Mike" Drob and others may

Re: [DISCUSS] Would a visibility histogram on a table be harmful?

2016-10-12 Thread ivan bella
Yes the "owners" could create a visibility counting mechanism separately, however if we make this RFile metadata a part of the system then we increase the "ease of use". Unfortunately, system designers rarely think about the metadata they need from their system up front. That being said, if the

Re: [DISCUSS] Would a visibility histogram on a table be harmful?

2016-10-12 Thread Marc P.
How does it increase ease of use? On Wed, Oct 12, 2016, 10:34 AM ivan bella wrote: > Yes the "owners" could create a visibility counting mechanism separately, > however if we make this RFile metadata a part of the system then we > increase the "ease of use". Unfortunately, system designers rarel

Re: [DISCUSS] Would a visibility histogram on a table be harmful?

2016-10-12 Thread Marc P.
Beyond adding a tool on the side. It doesn't fit in metadata as that requires aggregated reads vs table aggregates data. On Wed, Oct 12, 2016, 11:02 AM Marc P. wrote: > How does it increase ease of use? > > On Wed, Oct 12, 2016, 10:34 AM ivan bella wrote: > > Yes the "owners" could create a vis

Re: [DISCUSS] Would a visibility histogram on a table be harmful?

2016-10-12 Thread Josh Elser
Again, can we please bring this discussion back from discussions of implementations to security? Does the fact that you three were discussing implementations imply that you do not think this invalidates one of the core tenets (security first) of Accumulo? Christopher wrote: Keith, Russ, mys

[GitHub] accumulo issue #166: ACCUMULO-4498 Remove vestigial security permission stuf...

2016-10-12 Thread joshelser
Github user joshelser commented on the issue: https://github.com/apache/accumulo/pull/166 > I was particularly interested in having the removal of that doPrivileged block reviewed Yes, I did notice that removal. I'm not super familiar with the AccessController outside of the

Re: [DISCUSS] Would a visibility histogram on a table be harmful?

2016-10-12 Thread Marc P.
My point for discussing implementation outside of accumulo is because I think it does invalidate a core tenant On Wed, Oct 12, 2016, 12:26 PM Josh Elser wrote: > Again, can we please bring this discussion back from discussions of > implementations to security? > > Does the fact that you three we

Re: [DISCUSS] Would a visibility histogram on a table be harmful?

2016-10-12 Thread Keith Turner
On Wed, Oct 12, 2016 at 10:40 AM, ivan bella wrote: > Yes the "owners" could create a visibility counting mechanism separately, > however if we make this RFile metadata a part of the system then we increase > the "ease of use". Unfortunately, system designers rarely think about the > metadata

Re: [DISCUSS] Would a visibility histogram on a table be harmful?

2016-10-12 Thread Keith Turner
On Wed, Oct 12, 2016 at 12:56 AM, Christopher wrote: > Keith, Russ, myself (and possible others) were discussing this at the > hackathon after the Accumulo Summit, and I think our consensus were > basically this: > > We need a generic pluggable mechanism for injecting arbitrary user counters > int

Re: [DISCUSS] Would a visibility histogram on a table be harmful?

2016-10-12 Thread Josh Elser
Thanks, Marc. Follow-on question(s) for you: Do you think _any_ such approach should never be pursued by Accumulo (reading into your other replies about doing it outside of Accumulo)? Are the permissions that we have in place not sufficient to protect such "metadata"? Or, would such a featur

[GitHub] accumulo pull request #160: Simplify Accumulo scripts and config files

2016-10-12 Thread billierinaldi
Github user billierinaldi commented on a diff in the pull request: https://github.com/apache/accumulo/pull/160#discussion_r83066536 --- Diff: assemble/bin/accumulo --- @@ -51,137 +75,179 @@ locationByProgram() eval "${2}=${RESULT}" } -test -z "${JAVA_HOME}"

Re: [DISCUSS] Would a visibility histogram on a table be harmful?

2016-10-12 Thread ivan bella
I do not see how this invalidates any security of the system unless you are summarizing these counters and making them available through a thrift or other call; don't do that unless other security is put in place. To get a summary I would think you would have to use a separate utility to scrape

[GitHub] accumulo pull request #163: ACCUMULO-2278 Return value from in.read() should...

2016-10-12 Thread mstair
Github user mstair commented on a diff in the pull request: https://github.com/apache/accumulo/pull/163#discussion_r83072327 --- Diff: core/src/main/java/org/apache/accumulo/core/security/crypto/NonCachingSecretKeyEncryptionStrategy.java --- @@ -81,7 +81,10 @@ private void doKeyEn

[GitHub] accumulo pull request #163: ACCUMULO-2278 Return value from in.read() should...

2016-10-12 Thread mm376
Github user mm376 commented on a diff in the pull request: https://github.com/apache/accumulo/pull/163#discussion_r83072563 --- Diff: core/src/main/java/org/apache/accumulo/core/security/crypto/NonCachingSecretKeyEncryptionStrategy.java --- @@ -81,7 +81,10 @@ private void doKeyEnc

[GitHub] accumulo pull request #160: Simplify Accumulo scripts and config files

2016-10-12 Thread mikewalch
Github user mikewalch commented on a diff in the pull request: https://github.com/apache/accumulo/pull/160#discussion_r83081054 --- Diff: INSTALL.md --- @@ -26,33 +25,41 @@ source code. Unpack as follows. tar xzf /accumulo-X.Y.Z-bin.tar.gz cd accumulo-X.Y.Z

Re: [DISCUSS] Would a visibility histogram on a table be harmful?

2016-10-12 Thread Keith Turner
We did discuss making this info available through the public API (and adding thrift calls to gather it). We discussed the possibility of adding a new permission. On Wed, Oct 12, 2016 at 2:35 PM, ivan bella wrote: > I do not see how this invalidates any security of the system unless you are > s

[GitHub] accumulo pull request #160: Simplify Accumulo scripts and config files

2016-10-12 Thread joshelser
Github user joshelser commented on a diff in the pull request: https://github.com/apache/accumulo/pull/160#discussion_r83083196 --- Diff: INSTALL.md --- @@ -26,33 +25,41 @@ source code. Unpack as follows. tar xzf /accumulo-X.Y.Z-bin.tar.gz cd accumulo-X.Y.Z

Re: [DISCUSS] Would a visibility histogram on a table be harmful?

2016-10-12 Thread Josh Elser
I was envisioning public API protected by a system permission (implying some Thrift RPC as well) if that is an important distinction for those with concerns. I am hoping to get more info from Mike/Marc about why they feel this is insufficient WRT Accumulo's security model. Keith Turner wrote:

[GitHub] accumulo pull request #163: ACCUMULO-2278 Return value from in.read() should...

2016-10-12 Thread joshelser
Github user joshelser commented on a diff in the pull request: https://github.com/apache/accumulo/pull/163#discussion_r83087033 --- Diff: core/src/main/java/org/apache/accumulo/core/security/crypto/NonCachingSecretKeyEncryptionStrategy.java --- @@ -81,7 +81,10 @@ private void doKe

Re: [DISCUSS] Would a visibility histogram on a table be harmful?

2016-10-12 Thread Christopher
I think SystemPermission.SYSTEM permission should probably be required for any public API retrieving this data. It is, after all, code run on servers, generating data directly from the RFiles. This would also imply that caution is needed if we were to cache the data in, say, the metadata table. On

[GitHub] accumulo pull request #163: ACCUMULO-2278 Return value from in.read() should...

2016-10-12 Thread ctubbsii
Github user ctubbsii commented on a diff in the pull request: https://github.com/apache/accumulo/pull/163#discussion_r83102811 --- Diff: core/src/main/java/org/apache/accumulo/core/security/crypto/NonCachingSecretKeyEncryptionStrategy.java --- @@ -81,7 +81,10 @@ private void doKey

[GitHub] accumulo pull request #160: Simplify Accumulo scripts and config files

2016-10-12 Thread ctubbsii
Github user ctubbsii commented on a diff in the pull request: https://github.com/apache/accumulo/pull/160#discussion_r83104635 --- Diff: assemble/bin/accumulo --- @@ -51,137 +75,179 @@ locationByProgram() eval "${2}=${RESULT}" } -test -z "${JAVA_HOME}" &&

[GitHub] accumulo pull request #160: Simplify Accumulo scripts and config files

2016-10-12 Thread ctubbsii
Github user ctubbsii commented on a diff in the pull request: https://github.com/apache/accumulo/pull/160#discussion_r83104105 --- Diff: assemble/bin/accumulo --- @@ -51,137 +75,179 @@ locationByProgram() eval "${2}=${RESULT}" } -test -z "${JAVA_HOME}" &&

[GitHub] accumulo issue #166: ACCUMULO-4498 Remove vestigial security permission stuf...

2016-10-12 Thread ctubbsii
Github user ctubbsii commented on the issue: https://github.com/apache/accumulo/pull/166 From what I can tell, this class came from Hadoop TFile, which does not have it. I checked the pre-ASF history, and there's no explanation for it, so I guess I'll include the removal of that block

[GitHub] accumulo pull request #166: ACCUMULO-4498 Remove vestigial security permissi...

2016-10-12 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/accumulo/pull/166 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is en