Re: Gfsh Command enhancement - show missing-disk-stores

[Dan Smith]

I think it should be a single command because the user is trying to
diagnose the same problem - what persistent data is missing that is
preventing system recovery? I'm not sure what the best name of the command
is.

-Dan

On Fri, Sep 9, 2016 at 1:40 PM, Kenneth Howe  wrote:

> GEODE-1128  requests
> the addition of missing colocated region information to the gfsh show
> missing-disk-stores command. This is of course doable, however with
> additional information not directly related to disk stores, the command
> name would be misleading. You may have missing colocated regions without
> missing disk stores, or the converse, missing stores without missing
> regions.
>
> So my question is: Should the command be renamed to better indicate the
> types of information reported? While working on this Jira, I have been
> using the new command name ‘show persistent-recovery-failures’. (Please
> suggest a better name!)
>
> Alternatives to renaming the command are
> 1) Do nothing with the command name. Add the missing colocated region
> information, but leave the command name as is.
> 2) Add a new command with both missing disk stores and missing colocated
> regions, and leave the existing missing-disk-stores command as is.
>
> Looking for a consensus on the best approach.
>
> Thanks,
> Ken Howe
>

Re: M3 is out - New release manager for 1.0 ?

[Anthony Baker]

Thanks for starting this thread.  There are a few recent conversations on this 
topic [1][2].

My thought is that we should bear down and focus on the issues that help geode 
become “graduation-ready”.  Here’s a candidate list:

v1.0.0-incubating:
-
GEODE-37 (package renaming)
GEODE-1791 (LICENSE update)
+ some bugs fixes as they come in

v1.0.0+
-
everything else


Looking forward to hearing your thoughts!

Anthony

[1] 
https://mail-archives.apache.org/mod_mbox/incubator-geode-dev/201609.mbox/%3cef2278f9-a216-4f64-8576-c9d091747...@pivotal.io%3e
[2] 
http://mail-archives.apache.org/mod_mbox/incubator-geode-dev/201609.mbox/%3ccaheksulw1j6rwexqvf_v1fw9ajimaexy7gdy5fmjtfkwb2_...@mail.gmail.com%3e

> On Sep 8, 2016, at 4:24 PM, Swapnil Bawaskar  wrote:
> 
> I can take on this task for 1.0
> 
> On Thu, Sep 8, 2016 at 4:20 PM, William Markito  wrote:
> 
>> With M3 out and all the discussions we're having around what's needed for
>> 1.0, I think we should continue the rotation of release managers and get a
>> new volunteer for 1.0.
>> 
>> As part of this conversation we should also discuss the scope and roadmap
>> for 1.0 and could use the thread [1] for M3 as a starting point.
>> 
>> Here is what got discussed:
>> 
>> -
>> Guys, restarting this thread to get a discussion going about M3, 1.0.0 and
>> next -  As the release manager for M3 here is what I'd like to propose.
>> 
>> Any feedback is welcome and let's also reuse this thread to talk a little
>> bit about roadmap as well ?
>> 
>> # Current M3 Scope (https://cwiki.apache.org/confluence/display/GEODE/1.0.
>> 0-incubating.M3+Release)
>> 
>> GEODE-33
>> GEODE-823
>> GEODE-835
>> GEODE-919
>> GEODE-1146
>> GEODE-1168
>> GEODE-1203
>> GEODE-1259
>> GEODE-1278
>> GEODE-1293
>> GEODE-1316
>> GEODE-1256
>> GEODE-1267
>> 
>> == Proposed scope & roadmap ==
>> 
>> I'd like to breakdown the release a little bit and already start planning
>> the next releases.
>> 
>> # Geode 1.0.0-incubating M3
>> 
>> GEODE-1316 Update @since tags to include GemFire or Geode in the version
>> name
>> GEODE-1293 Align code and docs for modules
>> GEODE-1278 AbstractPeerTXRegionStub should throw
>> TransactionDataNodeHasDeparted when remote cache is closed
>> GEODE-1267 NOTICE file improvements
>> GEODE-1256 Geode website - Unapproved licenses
>> GEODE-1203 gfsh connect --use-http reports a ClassNotFoundException
>> GEODE-919 GEODE-823 Remove checksums from .asc files (asc.md5, asc.sha1)
>> GEODE-835 Replace joptsimple source with a binary dependency
>> GEODE-823 RC Feedback: Fix build artifacts
>> GEODE-33-1 Create project examples
>> 
>> # Geode 1.0.0-incubating
>> 
>> GEODE-33-2 Create project examples
>> GEODE-1331 gfsh.bat on Windows is incorrect
>> GEODE-1168 geode-dependencies manifest is missing jars that are present in
>> the lib directory
>> GEODE-629 Replace use of org.json with Jackson JSON library
>> GEODE-607 the offheap package needs better unit test coverage
>> GEODE-136 Fix possible NullPointerException in Gfsh's 'list regions'
>> command's GetRegionsFunction.
>> 
>> # Geode 1.X.0-incubating
>> 
>> GEODE-17 Provide Integrated Security
>> GEODE-11 Lucene Integration
>> GEODE-33-3 Create project examples
>> 
>> # Geode 2.0.0-incubating
>> 
>> GEODE-72 Remove deprecated APIs from Geode
>> GEODE-37 Package renaming
>> -
>> 
>> 
>> [1] http://markmail.org/message/6roww5yc3xhdpdck
>> --
>> ~/William
>> 



signature.asc
Description: Message signed with OpenPGP using GPGMail

Gfsh Command enhancement - show missing-disk-stores

[Kenneth Howe]

GEODE-1128  requests the 
addition of missing colocated region information to the gfsh show 
missing-disk-stores command. This is of course doable, however with additional 
information not directly related to disk stores, the command name would be 
misleading. You may have missing colocated regions without missing disk stores, 
or the converse, missing stores without missing regions.

So my question is: Should the command be renamed to better indicate the types 
of information reported? While working on this Jira, I have been using the new 
command name ‘show persistent-recovery-failures’. (Please suggest a better 
name!)

Alternatives to renaming the command are
1) Do nothing with the command name. Add the missing colocated region 
information, but leave the command name as is.
2) Add a new command with both missing disk stores and missing colocated 
regions, and leave the existing missing-disk-stores command as is.

Looking for a consensus on the best approach.

Thanks,
Ken Howe

Re: Review Request 51764: GEODE-1880 DistributedSystem.java: typo repair in javadocs

[Karen Miller]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/51764/#review148365
---


Ship it!




Ship It!

- Karen Miller


On Sept. 9, 2016, 6:30 p.m., Dave Barnes wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/51764/
> ---
> 
> (Updated Sept. 9, 2016, 6:30 p.m.)
> 
> 
> Review request for geode, Joey McAllister and Karen Miller.
> 
> 
> Repository: geode
> 
> 
> Description
> ---
> 
> GEODE-1880 DistributedSystem.java: typo repair in javadocs
> 
> 
> Diffs
> -
> 
>   
> geode-core/src/main/java/com/gemstone/gemfire/distributed/DistributedSystem.java
>  cb6851f4382950ffe3b9ba3a4c04a1288acb14be 
> 
> Diff: https://reviews.apache.org/r/51764/diff/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Dave Barnes
> 
>

Re: Review Request 51764: GEODE-1880 DistributedSystem.java: typo repair in javadocs

[Joey McAllister]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/51764/#review148357
---


Ship it!




Ship It!

- Joey McAllister


On Sept. 9, 2016, 6:30 p.m., Dave Barnes wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/51764/
> ---
> 
> (Updated Sept. 9, 2016, 6:30 p.m.)
> 
> 
> Review request for geode, Joey McAllister and Karen Miller.
> 
> 
> Repository: geode
> 
> 
> Description
> ---
> 
> GEODE-1880 DistributedSystem.java: typo repair in javadocs
> 
> 
> Diffs
> -
> 
>   
> geode-core/src/main/java/com/gemstone/gemfire/distributed/DistributedSystem.java
>  cb6851f4382950ffe3b9ba3a4c04a1288acb14be 
> 
> Diff: https://reviews.apache.org/r/51764/diff/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Dave Barnes
> 
>

Review Request 51764: GEODE-1880 DistributedSystem.java: typo repair in javadocs

[Dave Barnes]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/51764/
---

Review request for geode, Joey McAllister and Karen Miller.


Repository: geode


Description
---

GEODE-1880 DistributedSystem.java: typo repair in javadocs


Diffs
-

  
geode-core/src/main/java/com/gemstone/gemfire/distributed/DistributedSystem.java
 cb6851f4382950ffe3b9ba3a4c04a1288acb14be 

Diff: https://reviews.apache.org/r/51764/diff/


Testing
---


Thanks,

Dave Barnes

Re: Review Request 51741: [GEODE-1817] Prepare for 'release quality' publishing

[Dan Smith]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/51741/#review148351
---



The changes look good, but I don't think we want to disable release signing by 
default. Geode releases should be signed with gpg.

- Dan Smith


On Sept. 8, 2016, 7:35 p.m., Jens Deppe wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/51741/
> ---
> 
> (Updated Sept. 8, 2016, 7:35 p.m.)
> 
> 
> Review request for geode, Anthony Baker and Dan Smith.
> 
> 
> Repository: geode
> 
> 
> Description
> ---
> 
> This also disables default archive signing which requires GPG keys
> 
> 
> Diffs
> -
> 
>   build.gradle e112eb77847371f730ba72febfa350bfe9466832 
>   gradle.properties 06855c7a3782cb83cce9731460d3197749de42f3 
>   gradle/publish.gradle 2258da6537b69afad0c3188d06f91da9fd8c08c3 
> 
> Diff: https://reviews.apache.org/r/51741/diff/
> 
> 
> Testing
> ---
> 
> Local building. The changes come into effect when using 'uploadArchives'.
> 
> 
> Thanks,
> 
> Jens Deppe
> 
>

Re: securing geode components

[John Blum]

I agree with Udo here.  Securing the channel between component/services has
really very little to do with Authentication/Authorization, and by
"Authentication", I mean in the user-centric sense, not the SSL "trusted"
sense (with "trusted" keys and such).

Whether a user can or cannot do something (in other words, is "authorized",
or allowed to invoke an action, given an ACL) is a function of the action
being performed and the privileges/rights that a user has been granted.
That is independent of whether or not the channel has been secured.

I also agree with Kirk that HTTP should perhaps be renamed to WEB.

Finally, while SSL is usually a cause to reboot the system, Auth has no
such restrictions.  I could easily change Auth credentials of a user (e.g.
revoke privileges) at runtime.

My $0.02,
-John


On Fri, Sep 9, 2016 at 10:00 AM, Michael Stolz  wrote:

> In a pure world I would go for all or nothing, but I always worry about the
> upgrade path. If I have to redeploy and restart EVERYTHING around the whole
> world simultaneously, it's a non-starter.
>
> --
> Mike Stolz
> Principal Engineer, GemFire Product Manager
> Mobile: 631-835-4771
>
> On Fri, Sep 9, 2016 at 12:51 PM, Anthony Baker  wrote:
>
> > Mike, I was suggesting ON | OFF only for RBAC security, not for SSL
> > configuration.  Any thoughts on that?
> >
> > Anthony
> >
> > > On Sep 9, 2016, at 9:44 AM, Michael Stolz  wrote:
> > >
> > > I think a reason that we might need to be less than all-or-nothing is
> for
> > > at least these two situations:
> > >
> > > 1. a user who started out with SSL disabled, and now wants to enable
> it,
> > > but can't take a full global outage, so needs to get it enabled for the
> > WAN
> > > first, and then for server-to-server and then for client/server.
> > >
> > > 2. SSL enabled over the WAN because that is not a trusted network, but
> > they
> > > can live without SSL for the server/server and client/server
> connections
> > > because they ARE in a trusted network and they don't need to pay the
> > > overhead for SSL on those links.
> > >
> > > There are probably other scenarios as well, but these are the two that
> > come
> > > to mind quickly.
> > >
> > >
> > >
> > > --
> > > Mike Stolz
> > > Principal Engineer, GemFire Product Manager
> > > Mobile: 631-835-4771
> > >
> > > On Fri, Sep 9, 2016 at 12:05 PM, Jinmei Liao 
> wrote:
> > >
> > >> That is my original thought as well. If we are protecting resources
> > >> (CLUSTER and DATA), it should be protected no matter which way user is
> > >> trying to access it. I guess I'll leave this to the PMs to decide.
> > >>
> > >> On Thu, Sep 8, 2016 at 7:37 PM, Anthony Baker 
> > wrote:
> > >>
> > >>> Udo, Kirk - this makes sense and thanks for the discussion to help
> > >> clarify
> > >>> the issue.
> > >>>
> > >>> Regarding GEODE-1648, does it make sense to do at all?  That is, what
> > if
> > >>> role-based access control is either ON | OFF instead of per
> component.
> > >> If
> > >>> we allow disabling RBAC for certain components, wouldn’t that make it
> > >>> possible to create a backdoor?  Could we start with a binary option
> and
> > >>> enable more granular control when requested by users?
> > >>>
> > >>> Anthony
> > >>>
> >  On Sep 8, 2016, at 4:11 PM, Kirk Lund  wrote:
> > 
> >  +1 overall with some feedback...
> > 
> >  1) I think the list is reasonable with a few nitpicks below
> > 
> >  2) If these are Channels and not Components, then I would probably
> > name
> > >>> it
> >  SecurableChannels or SecurableCommunicationChannels or whatever.
> > 
> >  3) I'd prefer HTTP be renamed to Web or other non-protocol word --
> > HTTP
> > >>> is
> >  a protocol and the names of the other channels are conceptual
> channels
> >  rather than a protocol (the others use TCP/IP or RMI but we don't
> > label
> >  them as such). Or am I missing something here?
> > 
> >  4) JMX is probably ok. Currently we are using (and securing) JMX
> over
> > >> RMI
> >  (javax.management.remote.rmi.RMIConnectorServer). There are other
> >  connectors for JMX including HTTP (ex: mx4j.tools.adaptor.http.
> > >>> HttpAdaptor)
> >  and SNMP (ex: com.sun.jmx.snmp.daemon.SnmpAdaptorServer). We only
> > need
> > >>> JMX
> >  over RMI for now, but would we add those others as new enums to
> >  SecurableChannels later if we add anything like that to Geode? Or
> > would
> > >>> we
> >  try to group those all together under the name JMX? Or decide when
> the
> > >>> time
> >  comes?
> > 
> >  I think we should try to steer away from being overly controlled by
> > >> specs
> >  especially for reasonable changes. We all follow agile process, so a
> >  decision made one iteration could easily be undone or changed in the
> > >> next
> >  iteration and most of us are following weekly 

Re: securing geode components

[Anthony Baker]

Mike, I was suggesting ON | OFF only for RBAC security, not for SSL 
configuration.  Any thoughts on that?

Anthony

> On Sep 9, 2016, at 9:44 AM, Michael Stolz  wrote:
> 
> I think a reason that we might need to be less than all-or-nothing is for
> at least these two situations:
> 
> 1. a user who started out with SSL disabled, and now wants to enable it,
> but can't take a full global outage, so needs to get it enabled for the WAN
> first, and then for server-to-server and then for client/server.
> 
> 2. SSL enabled over the WAN because that is not a trusted network, but they
> can live without SSL for the server/server and client/server connections
> because they ARE in a trusted network and they don't need to pay the
> overhead for SSL on those links.
> 
> There are probably other scenarios as well, but these are the two that come
> to mind quickly.
> 
> 
> 
> --
> Mike Stolz
> Principal Engineer, GemFire Product Manager
> Mobile: 631-835-4771
> 
> On Fri, Sep 9, 2016 at 12:05 PM, Jinmei Liao  wrote:
> 
>> That is my original thought as well. If we are protecting resources
>> (CLUSTER and DATA), it should be protected no matter which way user is
>> trying to access it. I guess I'll leave this to the PMs to decide.
>> 
>> On Thu, Sep 8, 2016 at 7:37 PM, Anthony Baker  wrote:
>> 
>>> Udo, Kirk - this makes sense and thanks for the discussion to help
>> clarify
>>> the issue.
>>> 
>>> Regarding GEODE-1648, does it make sense to do at all?  That is, what if
>>> role-based access control is either ON | OFF instead of per component.
>> If
>>> we allow disabling RBAC for certain components, wouldn’t that make it
>>> possible to create a backdoor?  Could we start with a binary option and
>>> enable more granular control when requested by users?
>>> 
>>> Anthony
>>> 
 On Sep 8, 2016, at 4:11 PM, Kirk Lund  wrote:
 
 +1 overall with some feedback...
 
 1) I think the list is reasonable with a few nitpicks below
 
 2) If these are Channels and not Components, then I would probably name
>>> it
 SecurableChannels or SecurableCommunicationChannels or whatever.
 
 3) I'd prefer HTTP be renamed to Web or other non-protocol word -- HTTP
>>> is
 a protocol and the names of the other channels are conceptual channels
 rather than a protocol (the others use TCP/IP or RMI but we don't label
 them as such). Or am I missing something here?
 
 4) JMX is probably ok. Currently we are using (and securing) JMX over
>> RMI
 (javax.management.remote.rmi.RMIConnectorServer). There are other
 connectors for JMX including HTTP (ex: mx4j.tools.adaptor.http.
>>> HttpAdaptor)
 and SNMP (ex: com.sun.jmx.snmp.daemon.SnmpAdaptorServer). We only need
>>> JMX
 over RMI for now, but would we add those others as new enums to
 SecurableChannels later if we add anything like that to Geode? Or would
>>> we
 try to group those all together under the name JMX? Or decide when the
>>> time
 comes?
 
 I think we should try to steer away from being overly controlled by
>> specs
 especially for reasonable changes. We all follow agile process, so a
 decision made one iteration could easily be undone or changed in the
>> next
 iteration and most of us are following weekly iterations.
 
 After a release anything exposed in a User API is very difficult to
>>> change
 due to backwards compatibility constraints. I think we should be much
>>> more
 careful with User APIs in Geode going forward to avoid some of the
>>> problems
 we have with pre-existing Geode User APIs that we inherited.
 
 -Kirk
 
 On Thu, Sep 8, 2016 at 2:07 PM, Udo Kohlmeyer 
>>> wrote:
 
> As GEODE-420 deals with SSL comms configuration and GEODE-1648 with
> Authentication I think we need to be careful in what is
> feasible and what is logical.
> 
> For SSL comms it was decided that the following components are
>> relevant
> [1]  SL+properties>:
> 
> * Locator => The comms channel between Locators + the initial comms
>  channel between clients and locator
> * Cluster => Internode comms channel (peer to peer)
> * Server => Client-server comms channel
> * Gateway => Comms channel between WAN Gateway senders/receivers
> * HTTP => Any HTTP comms. incl REST and Pulse
> * JMX => Any JMX comms
> 
> These components were selected as they seem to be logical boundaries
>> and
> communication interfaces.
> 
> I think the specialization of HTTP, for Authentication
>> are
> functions of those interfaces:
> 
> * REST-admin
> * REST-dev
> * Pulse
> 
> I think that comms and functions exposed by those comms should not be
> mixed. I think that securing the comms channel is a factor of "trust".

Re: securing geode components

[Jinmei Liao]

That is my original thought as well. If we are protecting resources
(CLUSTER and DATA), it should be protected no matter which way user is
trying to access it. I guess I'll leave this to the PMs to decide.

On Thu, Sep 8, 2016 at 7:37 PM, Anthony Baker  wrote:

> Udo, Kirk - this makes sense and thanks for the discussion to help clarify
> the issue.
>
> Regarding GEODE-1648, does it make sense to do at all?  That is, what if
> role-based access control is either ON | OFF instead of per component.  If
> we allow disabling RBAC for certain components, wouldn’t that make it
> possible to create a backdoor?  Could we start with a binary option and
> enable more granular control when requested by users?
>
> Anthony
>
> > On Sep 8, 2016, at 4:11 PM, Kirk Lund  wrote:
> >
> > +1 overall with some feedback...
> >
> > 1) I think the list is reasonable with a few nitpicks below
> >
> > 2) If these are Channels and not Components, then I would probably name
> it
> > SecurableChannels or SecurableCommunicationChannels or whatever.
> >
> > 3) I'd prefer HTTP be renamed to Web or other non-protocol word -- HTTP
> is
> > a protocol and the names of the other channels are conceptual channels
> > rather than a protocol (the others use TCP/IP or RMI but we don't label
> > them as such). Or am I missing something here?
> >
> > 4) JMX is probably ok. Currently we are using (and securing) JMX over RMI
> > (javax.management.remote.rmi.RMIConnectorServer). There are other
> > connectors for JMX including HTTP (ex: mx4j.tools.adaptor.http.
> HttpAdaptor)
> > and SNMP (ex: com.sun.jmx.snmp.daemon.SnmpAdaptorServer). We only need
> JMX
> > over RMI for now, but would we add those others as new enums to
> > SecurableChannels later if we add anything like that to Geode? Or would
> we
> > try to group those all together under the name JMX? Or decide when the
> time
> > comes?
> >
> > I think we should try to steer away from being overly controlled by specs
> > especially for reasonable changes. We all follow agile process, so a
> > decision made one iteration could easily be undone or changed in the next
> > iteration and most of us are following weekly iterations.
> >
> > After a release anything exposed in a User API is very difficult to
> change
> > due to backwards compatibility constraints. I think we should be much
> more
> > careful with User APIs in Geode going forward to avoid some of the
> problems
> > we have with pre-existing Geode User APIs that we inherited.
> >
> > -Kirk
> >
> > On Thu, Sep 8, 2016 at 2:07 PM, Udo Kohlmeyer 
> wrote:
> >
> >> As GEODE-420 deals with SSL comms configuration and GEODE-1648 with
> >> Authentication I think we need to be careful in what is
> >> feasible and what is logical.
> >>
> >> For SSL comms it was decided that the following components are relevant
> >> [1]  >> SL+properties>:
> >>
> >> * Locator => The comms channel between Locators + the initial comms
> >>   channel between clients and locator
> >> * Cluster => Internode comms channel (peer to peer)
> >> * Server => Client-server comms channel
> >> * Gateway => Comms channel between WAN Gateway senders/receivers
> >> * HTTP => Any HTTP comms. incl REST and Pulse
> >> * JMX => Any JMX comms
> >>
> >> These components were selected as they seem to be logical boundaries and
> >> communication interfaces.
> >>
> >> I think the specialization of HTTP, for Authentication are
> >> functions of those interfaces:
> >>
> >> * REST-admin
> >> * REST-dev
> >> * Pulse
> >>
> >> I think that comms and functions exposed by those comms should not be
> >> mixed. I think that securing the comms channel is a factor of "trust".
> Do I
> >> implicitly trust the interface/system that I am connected to or are
> >> connecting to.
> >>
> >> I think concepts like "management" is a concept in function. Do I allow
> a
> >> user to access admin API's? The function of management should not
> determine
> >> if a system trusts another systems connection. When a new comms
> interface
> >> is added (say messaging), we want to be able to trust that comms
> channel.
> >> The "management" function should still work regardless of interface, be
> it
> >> jmx,http/rest,prop tcp,messaging.
> >>
> >> --Udo
> >>
> >> [1]: https://cwiki.apache.org/confluence/display/GEODE/Revised+SS
> >> L+properties
> >>
> >>
> >>
> >> On 9/09/2016 5:49 AM, Swapnil Bawaskar wrote:
> >>
> >>> GEODE-1648 and GEODE-420 are both trying to add geode properties to
> secure
> >>> only some components.
> >>> GEODE-1648 is intending to add a property named
> >>> "security-enabled-components" that will allow users to turn off
> >>> authentication/authorization for some components
> >>> GEODE-420 is intending to add a property named "ssl-enabled-components"
> >>> that will allow users to turn off ssl. for either client/server,
> >>> peer-to-peer or wan communication.
> >>>
> >>>