Re: [Proposal] Changes to the experimental cluster configuration API.

[Michael William Dodge]

Perhaps org.apache.geode.DataSerializable or 
org.apache.geode.internal.DataSerializableFixedID instead of 
java.io.Serializable?

Sarge

> On 19 Apr, 2018, at 17:04, Patrick Rhomberg  wrote:
> 
> E) Every cluster configuration object should be Serializable so that the
> objects may be sent from locators to members.

Re: [Proposal] Gfsh Command Feature Flag Annotation

[Michael William Dodge]

I kind of like @Disabled instead.

Sarge

> On 19 Mar, 2018, at 11:58, Udo Kohlmeyer  wrote:
> 
> I wonder if this proposal could not be extended to the greater GEODE product. 
> As this feature flagging is also relevant to other parts of the system and 
> should maybe be consistently applied to all areas.
> 
> Thoughts?
> 
> 
> On 3/19/18 11:46, Patrick Rhomberg wrote:
>> Hello, All
>> 
>>   I am interested in extending annotation functionality on our gfsh
>> commands, particularly with respect to feature-flagging commands that are
>> mutually-reliant or not yet feature complete.
>>   Please review the proposal [1] at your convenience.
>> 
>> Imagination is Change.
>> ~Patrick Rhomberg
>> 
>> [1]
>> https://cwiki.apache.org/confluence/display/GEODE/Proposal+for+Gfsh+Feature+Flag
>> 
> 

Re: Default branch for geode-examples repo

[Michael William Dodge]

I vote for changing the default branch to develop.

Sarge

> On 2 Jan, 2018, at 10:05, Alexander Murmann  wrote:
> 
> I noticed that many recent commits in the geode-examples repo have been
> merged to develop, but not master. Will those be merged to master at some
> point or should we change the default branch of the repo to develop?

Re: Discussion: Native PDX Reader/Writer std::string

[Michael William Dodge]

Of those, #1 seems better than #2 and #3. Is there an elegant way to use a 
sentinel value of std::string to represent null, e.g., static const 
PdxReader::NULL_STRING = ?

Sarge

> On 19 Dec, 2017, at 13:34, Jacob Barrett  wrote:
> 
> Java String is an object and as such a member of type String can be null or
> allocated in the heap. In C++ std::string is a value and as such a member
> of type std::string can never be null and is (usually) stack allocated with
> the contents allocated (usually) on the heap. In C/C++ char* is a decayed
> char[] and a such a member of type char* can be null or a pointer to an
> allocated char[] either on the heap or the stack.
> 
> As we move away from char* representations of strings which support null to
> std::string that does not we run into an interesting problem to solve. What
> to do about null strings in a PDX object? The original PdxReader readString
> returned a char*, which could be null or allocated a char[] on the heap and
> returned the pointer (usually, though I found some places where it returned
> the address to a static constant char[] but without const qualifier, bad).
> This version of PdxReader::readString then supports null string values
> coming from Java.
> 
> Looking at options for converting to std::string we have the following:
> 
> *1) std::string PdxReader::readString(const std::string& fieldName)*
> Most natural for C++11 but does not support null. A null PDX string value
> would be converted into an empty std::string. Upon writing back to PDX the
> null value would be replaced with and empty PDX string value. This could
> impact the other domain objects derived from this same PDX type if they
> treat null and empty strings differently. For example:
> class Foo {
> std::string naturalStringMember;
> 
> ~Foo() = default;
> 
> void fromData(PdxReader reader) {
>  // RVO moved contents
>  unnaturalStringMember = readString("naturalStringMember");
>  // null value lost
> }
> };
> 
> 
> *2) std::string* PdxReader::readString(const std::string& fieldName)*
> The subtle difference there is returning raw pointer to a heap allocated
> std::string, which is very very unnatural to C++. The domain object would
> either hold the raw pointer or check for null and copy to a std::string
> value member. In either case they would have to free the allocated
> std::string when done. For example:
> class Foo {
> std::string* unnaturalStringMember;
> std::string naturalStringMember;
> 
> ~Foo() {
>  if (unnaturalStringMember) {
>delete unnaturalStringMember;
>  }
> }
> 
> void fromData(PdxReader reader) {
>  unnaturalStringMember = readString("unnaturalStringMember");
>  if (auto tmp = readString("naturalStringMember")) {
>// move contents
>naturalStringMember = std::move(*naturalStringMember);
>// delete
>delete naturalStringMember;
>  } else {
>// null value lost
>  }
> }
> };
> *YUCK!!*
> 
> *3) std::unique_ptr PdxReader::readString(const std::string&
> fieldName)*
> This shares some of the similar issues with unnaturally having a heap
> allocated std::string but solves the ownership ambiguity and deleting
> issues. For example:
> class Foo {
> std::unique_ptr unnaturalStringMember;
> std::string naturalStringMember;
> 
> ~Foo() = default;
> 
> void fromData(PdxReader reader) {
>  unnaturalStringMember = readString("unnaturalStringMember");
>  if (auto tmp = readString("naturalStringMember")) {
>// move contents and delete
>naturalStringMember = std::move(*tmp);
>  } else {
>// null value lost
>  }
> }
> };
> *Eh!*
> 
> 4) Given that C++ does not allow the same method signature only differing
> in return type we can't have all these methods on the interface unless we
> named them differently as well. For example readString and
> readNullableString. Providing both puts the power in the implementor but
> maybe we should have an opinionated approach.
> 
> std::string PdxReader::readString(const std::string& fieldName);
> std::unique_ptr PdxReader::readNullableString(const
> std::string& fieldName);
> 
> class Foo {
> std::unique_ptr nullableStringMember;
> std::string naturalStringMember;
> 
> ~Foo() = default;
> 
> void fromData(PdxReader reader) {
>  nullableStringMember = readNullabletring("nullableStringMember");
>  naturalStringMember = readString("uniqueStringMember");
> }
> };
> 
> 
> 5) Alternatively one could use out parameters, though we have been trying
> to get away from out parameters since they don't match the "functional"
> preference.
> void PdxReader::readString(const std::string& fieldName
>   std::string& value);
> void PdxReader::readString(const std::string& fieldName,
>   std::string* value);
> void PdxReader::readString(const std::string& fieldName,
>   std::unique_ptr& value);
> 
> class Foo {
> std::string* rawStringMember;
> std::unique_ptr uniqueStringMember;
> std::string naturalStringMember;
> 
> ~Foo() = 

Re: DISCUSS: Deprecating and replacing current serializable string encodings

[Michael William Dodge]

I think there is value in having a single string encoding.

Sarge

> On 1 Dec, 2017, at 17:35, Jacob Barrett  wrote:
> 
> On Fri, Dec 1, 2017 at 4:59 PM Dan Smith  wrote:
> 
>> I think I'm kinda with Mike on this one. The existing string format does
>> seem pretty gnarly. But the complexity of implementing and testing all of
>> the backwards compatibility transcoding that would be required in order to
>> move to the new proposed format seems to be way more work with much more
>> possibility for errors. Do we really expect people to be writing new
>> clients that use DataSerializable? It hasn't happened yet, and we're
>> working on a new protocol that uses protobuf right now.
>> 
> 
> Consider that any new clients written would have to implement all these
> encodings. This is going to make writing new clients using the upcoming new
> protocol laborious. The new protocol does not define object encoding, it
> strictly defines message encoding. Objects sent over the protocol will have
> to be serialized in some format, like PDX or data serializer. We could
> alway develop a better serialization format than what we have now. If we
> don't develop something new then we have to use the old. Wouldn't it be
> nice if the new clients didn't have to deal with legacy encodings?
> 
> If the issue is really the complexity of serialization from the C++ client,
>> maybe the C++ client could always write UTF-16 strings?
>> 
> 
> You can't assume that a client in one language will only be serializing
> strings for it's own consumption. We have many people using strings in PDX
> to transform between C++, .NET and Java.
> 
> The risk is high not to remove this debt. If I am developing a new Ruby
> client I am forced to deal with all 4 of these encodings. Am I really going
> to want to build a Ruby client for Geode, am I going to get these encodings
> correct? I can tell you that getting them correct may be a challenge if the
> current C++ client is any indication, it has a few incorrect assumptions in
> its encoding of ASCII and modified UTF-8.
> 
> I am fine with a compromise that deprecates but doesn't remove the old
> encodings for a few releases. This would give time for users to update. New
> clients written would not be be able to read this old data but could read
> and write new data.
> 
> 
> 
> -Jake

Re: Next release: 1.4.0

[Michael William Dodge]

What sort of update? I know that GEODE-4010 has a PR that's awaiting review and 
merge.

Sarge

> On 28 Nov, 2017, at 10:03, Swapnil Bawaskar  wrote:
> 
> I would like to volunteer as a release manager.
> Currently there are 14 issues that are marked for 1.4.0. If you are working
> on any of these, can you please update the JIRA?
> https://issues.apache.org/jira/secure/RapidBoard.jspa?rapidView=92=GEODE=planning=GEODE-3688=visible=12341842
> 
> Thanks!
> 
> On Tue, Nov 28, 2017 at 9:42 AM Anthony Baker  wrote:
> 
>> Bump.  Any volunteers?  If not, I’ll do this.
>> 
>> Anthony
>> 
>> 
>>> On Nov 22, 2017, at 1:48 PM, Anthony Baker  wrote:
>>> 
>>> We released Geode 1.3.0 at the end of October.  Our next release will be
>> 1.4.0.  Questions:
>>> 
>>> 1) Who wants to volunteer as a release manager?
>>> 2) What do we want to include in the release?
>>> 3) When do we want to do this?
>>> 
>>> IMO, let's should shoot for an early Dec release.
>>> 
>>> Anthony
>>> 
>> 
>> 

Re: Permissions To View Builds?

[Michael William Dodge]

Singular job worked. That'll teach me to transcribe an URL by hand instead of 
insisting it get emailed to me. ;P Thanks all.

Sarge

> On 7 Nov, 2017, at 14:02, Dan Smith <dsm...@pivotal.io> wrote:
> 
> I think you just have the wrong url. Try this (job, not jobs):
> 
> https://builds.apache.org/job/Geode-nightly/
> 
> On Tue, Nov 7, 2017 at 12:35 PM, Michael William Dodge <mdo...@pivotal.io>
> wrote:
> 
>> I tried to check on the nightly builds via https://builds.apache.org/
>> jobs/Geode-nightly/ <https://builds.apache.org/jobs/Geode-nightly/>. I
>> think that's the right URL but I get an error message that says "Invalid
>> login information. Please try again.". If that's the right URL, may I have
>> permissions to view the nightly builds? Thanks.
>> 
>> Sarge

Permissions To View Builds?

[Michael William Dodge]

I tried to check on the nightly builds via 
https://builds.apache.org/jobs/Geode-nightly/ 
. I think that's the right URL 
but I get an error message that says "Invalid login information. Please try 
again.". If that's the right URL, may I have permissions to view the nightly 
builds? Thanks.

Sarge

Re: Consistent results from gfsh commands

[Michael William Dodge]

+1 for consistency!

Sarge

> On 31 Oct, 2017, at 07:59, Jens Deppe  wrote:
> 
> Hi,
> 
> I've noticed that various commands, that execute on multiple members,
> return either tabulated results (one line per member - for example
> CreateRegionCommand) or a single pass/fail line. In the latter case it’s
> possible that information would get lost. For example if a command fails on
> multiple members (but not all) then the user would not have complete
> insight into where the failure occurred.
> 
> I'd like to propose that if a command executes on multiple members it
> should always return and display all the results. Alternatively, if the
> command succeeds then it should simply indicate success, however if it
> fails (even partially) it should return a full list of each member and the
> result of the command on each of the members.
> 
> Thoughts; comments?
> 
> --Jens

Re: [Discussion] Native - Should C++ client support wchar_t type?

[Michael William Dodge]

I think support for wchar_t adds unnecessary complexity and should be removed.

Sarge

> On 26 Oct, 2017, at 13:05, Jacob Barrett  wrote:
> 
> I think we should get rid of it. There are plenty of discussions out there 
> around C++ strings and how they relate to Unicode. Wide string does not mean 
> Unicode. The general consensus I have seen is to just use char and require 
> UTF-8 encoding. We have lots of confusion in our code around whether a string 
> is Unicode or ASCII. I say remove the confusion and all strings are Unicode 
> in UTF-8 encoded 8 bit chars. Also convert all char* to std::string for which 
> we already have a ticket.
> 
> -Jake
> 
> 
>> On Oct 26, 2017, at 12:15 PM, David Kimura  wrote:
>> 
>> While working on removing out parameters, we noticed code that makes
>> assumption that wchar_t is always 16 bits.
>> 
>> virtual void PdxInstance::getField(const char* fieldName, wchar_t& value)
>> const = 0;
>> 
>> void PdxInstanceImpl::getField(const char* fieldname, wchar_t& value) const
>> {
>>   auto dataInput = getDataInputForField(fieldname);
>>   uint16_t temp = dataInput->readInt16();
>>   value = static_cast(temp);
>> }
>> 
>> 
>> 
>> According to cppreference[1], this assumption is incorrect.  If that is the
>> case, should this implementation be fixed or can it be deleted altogether?
>> 
>> Thanks,
>> David
>> 
>> [1] http://en.cppreference.com/w/cpp/language/types

Re: [Discussion] Native - Dropping exceptions from Region::getAll

[Michael William Dodge]

+1 for dropping the exceptions map for symmetry with the Java API

Sarge

> On 23 Oct, 2017, at 09:32, Jacob Barrett  wrote:
> 
> As part of our decision to replace all function out variables with return
> values we ran into an interesting function with Region::getAll. That
> function had two out variables, one the map of key/value pairs and another
> map for key/exception pairs. There are no tests that actually test the
> key/exception pair nor does it match any API on the Java client. The best
> we can figure is that it was a half baked extension to the API. Converting
> to return values means that getAll must return a std::tuple and while this
> is within our decision guidelines it feels really dirty on this function.
> 
> I propose that we drop the exceptions map from the return of
> Region::getAll. Please respond in the next 48 hours with our input on this
> matter.
> 
> -Jake

Re: [Discuss] CliStrings

[Michael William Dodge]

+1 for removing it based on Dan's reasoning.

Sarge

> On 20 Oct, 2017, at 13:59, Dan Smith  wrote:
> 
> +1 for removing it.
> 
> I do think it would be nice if we add localization in the future. But
> I don't really like the idea of leaving stuff in our code just in case
> we decide to implement a feature in the future - we might not even
> want what we left in there! In this case even with localization we
> might want to move these constants into their specific command classes
> anyway - the constant might just look up the localized string instead
> of being hardcoded.
> 
> -Dan
> 
> On Fri, Oct 20, 2017 at 9:57 AM, Anilkumar Gingade  
> wrote:
>> As others mentioned this was done to support localization in the product;
>> this was driven from customer request; mainly from AMEA customers...
>> Now being open source; to build a wider community, Localization may become
>> a key requirement...Having a framework will help community to build on
>> that...
>> 
>> -Anil.
>> 
>> 
>> On Thu, Oct 19, 2017 at 4:32 PM, Mark Hanson  wrote:
>> 
>>> From my product background, maintaining a single file is often preferred
>>> for localization as mentioned by Mike, the big question, I would ask is how
>>> important is localization? If localization is important, then keeping a
>>> single or few file(s) will dramatically ease the localization process. So
>>> following Jared’s approach might make more work in the future, but it is
>>> certainly sound if there is no localization. This may also be an opportune
>>> time to review localization strategies within the code.
>>> 
>>> Thanks,
>>> Mark
 On Oct 19, 2017, at 3:28 PM, Bruce Schuchardt 
>>> wrote:
 
 +1 I thought we decided long ago to do this.  We also have
>>> LocalizedStrings.java and it looks like some folks are still adding new
>>> StringIDs to it as well.
 
 
 On 10/19/17 3:13 PM, Nick Reich wrote:
> +1 for moving those messages out of CliStrings if at all possible and
> placing them where they are used. In my experiences with those strings,
>>> I
> have rarely if ever seen them reused across classes, so they really
>>> should
> belong in the class they are used by.
> 
> On Thu, Oct 19, 2017 at 3:05 PM, Jared Stewart 
>>> wrote:
> 
>> I wanted to kick off a discussion about the usage of CliStrings.  For
>> those unfamiliar, it’s a java class that contains about ~3000 lines of
>> String constants and has a javadoc explaining that it is an attempt at
>>> i18n
>> localization.  Does anyone know if this localization is actually
>> implemented in practice?
>> 
>> If not, I would like suggest that we try to move away from this pattern
>> going forward.  We have ended up with many constants in CliStrings like
>> this:
>> CliStrings.CREATE_REGION__MSG__ONLY_ONE_OF_REGIONSHORTCUT_
>> AND_USEATTRIBUESFROM_CAN_BE_SPECIFIED
>> The constant is only used in CreateRegionCommand, so I would be
>>> happier to
>> see it as a member of CreateRegionCommand (where there would be no
>>> need for
>> the unwieldy "CREATE_REGION__MSG__” prefix) unless there is
>>> localization
>> being done which I am unaware of.
>> 
>> Thoughts?
>> 
>> Thanks,
>> Jared
 
>>> 
>>> 

Re: [VOTE] C++ standardize on return values only

[Michael William Dodge]

I have seen the uint8_t *, uint32_t tuple mostly with C-style system level 
calls (e.g., reading from a resource). Perhaps these could be allowed to remain 
as part of lowering impedance with a legacy API.

Sarge

> On 18 Oct, 2017, at 15:08, Mark Hanson  wrote:
> 
> Sorry about that last email...  to begin again
> 
> Hello All,
> 
> So in doing some work on this subject, we have come across a case that David 
> and I believed was worth raising for discussion.
> 
> As one might expect, at some point, someone is going to pass in a buffer and 
> a length into a function and expect data put into that buffer, e.g.
> inline void readBytesOnly(int8_t* buffer, uint32_t len) {
>  if (len > 0) {
>checkBufferSize(len);
>std::memcpy(buffer, m_buf, len);
>m_buf += len;
>  }
> }
> 
> That throws a kink into the whole no out variables discussion. It is 
> addressable, for this question, we want to know how the community would like 
> to move forward, would it be best just to leave an API like this  alone and 
> call it exceptional?  Would it be best for us to allocate a std::unique_ptr, 
> or less desirably a share_ptr? I could see leaving it as is, because the java 
> API does this and it give greater discretion to the caller, however, I think 
> a fine case can be made for using a unique_ptr as well.
> 
> From a higher level for this API only, should we get rid of this API and have 
> it dealt with through its more standard types? This is kind of an end run for 
> accessing generic data, if you look at it right… 
> 
> What do you think?
> 
> Thanks,
> Mark
> 
> 
> On Wed, Oct 18, 2017 at 2:49 PM, Mark Hanson  > wrote:
> Hello All,
> 
> So in doing some work on this subject, we have come across a case that David 
> and I believed was worth raising for discussion.
> 
> As one might expect, at some point, someone is going to pass in a buffer and 
> a length into a function and expect data put into that buffer, e.g.
> 
> 
> On Tue, Oct 3, 2017 at 4:26 PM, Jacob Barrett  > wrote:
> Voting on the conversation around C++ return values vs. out parameters.
> This vote is to adopt the standard of return values over the use of out
> parameters. On functions that must return more than one value to use the
> C++11 std::tuple type for future compatibility with C++17.
> 
> For example:
> 
> std::tuple foo::getAAndB() {...}
> 
> And call it with:
> 
> int a;
> std::string b;
> std::tie(a, b) = foo.getAAndB();
> 
> Alternatively the tuple can be called like:
> 
> auto r = foo.getAAndB();
> auto a = std::get<0>(r);
> auto b = std::get<1>(r);
> 
> In C++17:
> 
> auto [a, b] = foo.getAAndB();
> 
> 
> 
> Rather than:
> 
> int foo::getAAndB(std::string& b) {...}
> 
> Called like
> 
> std::string b;
> auto a = foo.getAAndB(b);
> 
> -Jake
> 
> 

Re: [VOTE] Apache Geode release - 1.3.0 RC2

[Michael William Dodge]

If we are including geode-examples, there are six PRs outstanding for 
geode-examples.

Sarge

> On 15 Oct, 2017, at 20:51, Anthony Baker  wrote:
> 
> -1
> 
> The geode-examples source should be included in this release.  
> 
> The download task in the geode-examples build also needs to be updated since 
> the distribution archive name has changed from .tar.gz to .tgz.  Are there 
> any other downstream dependencies affected by this change?
> 
> Anthony
> 
> 
>> On Oct 13, 2017, at 1:39 PM, Swapnil Bawaskar  wrote:
>> 
>> This is the second release candidate for Apache Geode, version 1.3.0. I
>> will send out a separate vote thread for geode-examples.
>> Thanks to all the community members for their contributions to this
>> release!
>> 
>> *** Please download, test and vote by Monday, October 16, 0800 hrs
>> US Pacific. ***
>> 
>> It fixes 376 issues. release notes can be found at:
>> https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12318420=12340669
>> 
>> Note that we are voting upon the source tags:  rel/v1.3.0.RC2
>> https://github.com/apache/geode/tree/rel/v1.3.0.RC2
>> 
>> Commit ID:
>> 9e076738fc2ae40f95bd179b5c1624e664a28d61
>> 
>> Source and binary files:
>> https://dist.apache.org/repos/dist/dev/geode/1.3.0.RC2
>> 
>> Maven staging repo:
>> https://repository.apache.org/content/repositories/orgapachegeode-1033
>> 
>> Geode's KEYS file containing PGP keys we use to sign the release:
>> https://github.com/apache/geode/blob/develop/KEYS
>> 
>> Release Signed with Key: pub 4096R/18F902DB 2016-04-07
>> Fingerprint: E1B1 ABE3 4753 E7BA 8097 4285 8F8F 2BCC 18F9 02DB
> 

Re: [DISCUSS] C++ Returning null values

[Michael William Dodge]

To me the question of a no-op region comes down to whether customers would need 
to do logic based on whether a region has a parent. If they do, perhaps a bool 
hasParent() along with getParent() returning a no-op region would be an elegant 
solution. If they do not, then shared_ptr is probably sufficient.

Sarge

> On 10 Oct, 2017, at 12:45, David Kimura  wrote:
> 
> Yes, but I wouldn't expect to ever need to do a type check. Admittedly, I'm
> not sure what one does with parent region, but if we no-op, return sensible
> empty values, or throw when appropriate then maybe we don't care?
> 
> I would expect it to be used something like this:
> 
> auto parent = region.getParent();
> 
> // if region didn't have parent region, this could always return false.
> // else if it did have a parent region it would behave as expected.
> parent.containsKey("a_key");
> 
> 
> If this works, I like that we don't have to special case for calling
> getParent on root region and we could return by value.
> 
> Thanks,
> David
> 
> On Tue, Oct 10, 2017 at 12:29 PM, Jacob Barrett  wrote:
> 
>> Are you thinking something like?
>> 
>> class NoRegion : public Region {...};
>> 
>> auto parent = region.getParent();
>> if (NoRegion == parent) {
>>  // no parent region
>> }
>> 
>> 
>> On Tue, Oct 10, 2017 at 11:08 AM David Kimura  wrote:
>> 
>>> I'm not sure if this is the same as the sentinel value you mentioned, but
>>> what about introducing a no-op region type and returning that?  I'm
>>> thinking a null object pattern which would no-op and then nobody should
>>> need to check if nullptr.
>>> 
>>> Thanks,
>>> David
>>> 
>>> On Tue, Oct 10, 2017 at 10:27 AM, Jacob Barrett 
>>> wrote:
>>> 
 Looking at a class like Region (Region.cpp) there are calls to get the
 parent region and sub regions, there are instances where a Region will
>>> not
 have a parent or subs. The current API returns shared_ptr that may be
 nullptr or a Region.
 
 Since we are trying to make an effort towards values over pointers
>> should
 be considered some changes here? Obviously a reference is out of the
 question because it can't be null. A value is really out of the
>> question
 too since it can't be null and making a sentinel value is not a great
 solution. Raw pointers are fine since they can be nullptr but make
 ownership ambiguous. Using shared_ptr is good since it can be nullptr
>> and
 solves the ownership problem. Another option is to embrace the
>>> forthcoming
 std::optional available as boost::optional in C++11.
 
 I am leaning towards keeping it shared_ptr since using boost::optional
 would require users compile with boost. I don't think we should have
 anything on our API that is not ours of C++11. Requiring third party
 libraries to compile against our API doesn't fly right by me.
 
 Thoughts?
 
>>> 
>> 

Re: [VOTE] C++ standardize on return values only

[Michael William Dodge]

+1 for the tuple

> On 4 Oct, 2017, at 08:01, Ernest Burghardt  wrote:
> 
> +1 Tuple
> 
> On Wed, Oct 4, 2017 at 7:58 AM, David Kimura  wrote:
> 
>> +1
>> 
>> On Tue, Oct 3, 2017 at 5:45 PM, Hitesh Khamesra
>> >> wrote:
>> 
>>> Tuple option.
>>> 
>>> Sent from Yahoo Mail on Android
>>> 
>>>  On Tue, Oct 3, 2017 at 4:27 PM, Jacob Barrett
>>> wrote:   Voting on the conversation around C++ return values vs. out
>>> parameters.
>>> This vote is to adopt the standard of return values over the use of out
>>> parameters. On functions that must return more than one value to use the
>>> C++11 std::tuple type for future compatibility with C++17.
>>> 
>>> For example:
>>> 
>>> std::tuple foo::getAAndB() {...}
>>> 
>>> And call it with:
>>> 
>>> int a;
>>> std::string b;
>>> std::tie(a, b) = foo.getAAndB();
>>> 
>>> Alternatively the tuple can be called like:
>>> 
>>> auto r = foo.getAAndB();
>>> auto a = std::get<0>(r);
>>> auto b = std::get<1>(r);
>>> 
>>> In C++17:
>>> 
>>> auto [a, b] = foo.getAAndB();
>>> 
>>> 
>>> 
>>> Rather than:
>>> 
>>> int foo::getAAndB(std::string& b) {...}
>>> 
>>> Called like
>>> 
>>> std::string b;
>>> auto a = foo.getAAndB(b);
>>> 
>>> -Jake
>>> 
>>> 
>> 

Re: New client/server protocol - seeking feedback

[Michael William Dodge]

From my days using Win32 APIs, I think fixing Foo() with FooEx() is an 
anti-pattern. But that's not to say that "version 37 fixes the parameters to 
Foo() and in no other way changes anything" is any better. I see the version as 
useful for determining the structure of the protocol, not the specifics of a 
message per se.

One of the disadvantages of using versions is that it can lead to spaghetti 
code such as cascading if statements to handle different versions of any given 
message. I worry that having the client and server negotiate which messages 
they are going to use would also be a significant addition of complexity.

Sarge

> On 2 Oct, 2017, at 11:14, Jacob Barrett  wrote:
> 
> A change to a message should just be a new message, no need to version it.
> Clients and severs could negotiate the messages they support or attempt the
> message they support and fallback to an alternative if the server rejects
> it. Consider Put and PutEx (ignore the names):
> Put ( Key, Value )
> PutEx (Key, Value, SomethingElse )
> The client could try PutEx but if rejected by older server and
> SomethingElse is not important to its operation it could try Put instead.
> Alternatively the server could be queried or a list of supported message
> IDs in which it could return only PutEx and the older client could make a
> decision easier as to whether or not it can talk to the server.
> 
> Although one could argue these are district operation that should be
> defined as independent messages anyway. Think clean OO design in your
> message design. The message should have significantly change behavior
> because of a single parameter otherwise it is really a new thing and should
> be defined as a new message.
> 
> The short answer is that version numbers make for a nightmare of
> compatibility especially when interleaving releases and maintenance
> releases. Look at our current protocol and the gaps we leave in the ordinal
> numbering to avoid this issue. Let's not make that same mistake.
> 
> 
> 
> As for interleaving requests and responses, this should be layered in the
> protocol. The top layer should only deal with serial request/response. Let
> a lower layer encapsulate the upper level in a multiplexed manor. The naive
> layer could just open additional sockets to achieve interleaving, while an
> advanced approach would create sub channels on the socket and forward to
> the appropriate upper later session. All very easily achieved with Netty.
> When the client connects it could negotiate if the server supports the
> channel method, just like we negotiate SSL or authentication. The other
> benefit to this approach is you don't have an unused field in your protocol
> for clients that don't want to implement something so complex.
> 
> 
> -Jake
> 
> 
> 
> 
> 
> On Mon, Oct 2, 2017 at 10:22 AM Michael Stolz  wrote:
> 
>> We should check that it is actually safe to add fields.
>> If it isn't we're likely to have a lot of versioning to do.
>> 
>> --
>> Mike Stolz
>> Principal Engineer, GemFire Product Lead
>> Mobile: +1-631-835-4771 <(631)%20835-4771>
>> 
>> On Mon, Sep 25, 2017 at 5:25 PM, Galen O'Sullivan 
>> wrote:
>> 
>>> Replies inline.
>>> 
>>> On Mon, Sep 25, 2017 at 12:13 PM, Udo Kohlmeyer 
>>> wrote:
>>> 
 Replies inline
 On Mon, Sep 25, 2017 at 11:21 AM, Dan Smith  wrote:
 
> This actually brings up another point I was going to ask about. I
>> don't
 see
> any version information in the protocol. How will we handle adding
>> new
> features to this protocol? Do the clients and servers negotiate which
> version of the protocol to use somehow?
> 
> I think there should be a plan in place for making changes to the
 protocol
> and supporting old clients. Given that, we shouldn't add fields that
 aren't
> actually used into the existing version of the protocol. When we
 introduce
> new features into the protocol, that's the point at which we should
>> add
 the
> fields to support that feature.
> 
 
 [UK] - Protobuf allows for the amending of messages. As soon as the
 protocol changes significantly the "magic" number will have to be
 incremented, indicating a new (non-backward compatible) version of the
 protocol. This of course has bigger problems, where Java does not allow
>>> for
 multiple versions of the same class to be loaded, so a server could run
 only 1 version of Protobuf messages at a time.
 
>>> 
>>> We have to be careful about how we extend protobuf messages, though. I'm
>>> not sure exactly what's safe to do, but it should at least be safe to add
>>> fields (assuming they don't change existing behavior -- we'll have to
>> think
>>> about this) and ignore old fields (which is how you would remove a
>>> now-unused field). It's fairly simple to add new operations without any
>>> interesting breakages - 

Re: [DISCUSS] Removal of "Submit an Issue" from Geode webpage

[Michael William Dodge]

+1 to improving the signal-to-noise ratio

> On 29 Sep, 2017, at 11:07, Jason Huynh  wrote:
> 
> GEODE-3280

Re: [DISCUSS] Using out parameters and its effects on function overload resolution

[Michael William Dodge]

I like the idea of using non-const references for in-out parameters only and 
using tuples for the cases where there are multiple out parameters. Yes, the 
return type does not participate in overload resolution but I don't think there 
would be many cases where that would be an issue. (But I haven't done any 
research so that's just a WAG.)

Sarge

> On 27 Sep, 2017, at 12:22, David Kimura  wrote:
> 
> Is there a use case in our C++ client code to ever use out parameters?
> 
> Currently code uses out parameters to return multiple values from a
> function.  [1]
> 
> virtual int64_t* PdxReader::readLongArray(const char* fieldName, int32_t&
> length) = 0;
> 
> 
> In this case, we could return a tuple instead.  I think the advantage of
> this is it doesn't force a separation between the variable declaration and
> assignment, which may lead to spaghetti code.  I think avoiding out
> parameters also leads to more well defined behavior.  What would one expect
> if they called getCqListeners with an already partially populated vector?
> [2]
> 
> virtual void CqAttributes::getCqListeners(std::vector 
> 
> As a counter argument, could function overload resolution be a valid use
> case of out parameters?  In PdxReader::readType methods, Type seems
> redundant.  As a user, why should I have to call readLongArray(int64_t[]&)
> rather than just call read(int64_t[]&)?  In this case, if we didn't use out
> parameter our signature clashes with other read methods.
> 
> Thoughts?
> 
> Thanks,
> David
> 
> [1]
> https://github.com/apache/geode-native/blob/44635ffa95926c9cffecc1dcaac02fb3012d1eef/cppcache/include/geode/PdxReader.hpp#L288
> [2]
> https://github.com/apache/geode-native/blob/44635ffa95926c9cffecc1dcaac02fb3012d1eef/cppcache/include/geode/CqAttributes.hpp#L60

Re: [Vote] Better type checking by moving PdxWrapper from a standard class to a template class.

[Michael William Dodge]

+1 for type safety

Sarge

> On 21 Sep, 2017, at 10:21, Mark Hanson  wrote:
> 
> Here is a link to my branch in my fork that has the changes on it.
> 
> https://github.com/mhansonp/geode-native/tree/wip/templatePdxWrapper
> 
> Thanks,
> Mark
> 
> On Thu, Sep 21, 2017 at 10:19 AM, Mark Hanson  wrote:
> 
>> Hi All,
>> 
>> In reviewing the PdxWrapper class it, it seemed like it would be a good
>> move to make this a template class. This will allow better type checking
>> anytime we use it.
>> 
>> An example of what is being planned is to change from
>> 
>> MyClass object;
>> PdxWrapper((void *) object,...)
>> to PdxWrapper(object, )
>> 
>> I have a chunk of code moved over that seems to show it is possible,
>> though there are some bugs that need to be addressed, so it is not a done
>> deal.
>> 
>> What do people think?
>> 
>> Thanks,
>> Mark
>> 

Re: [DISCUSS] Framework for concurrency tests

[Michael William Dodge]

+1 for unit tests for multithreaded code.

High fives to Dan.

Sarge

> On 15 Sep, 2017, at 12:08, Dan Smith  wrote:
> 
> Hi Geode devs,
> 
> I've been messing around with an open source tool called Java
> Pathfinder for writing tests of multithreaded code. Java Pathfinder is
> a special JVM which among other things tries to execute your code
> using all possible thread interleavings.
> 
> I'd like to propose two things:
> 
> 1) We introduce a framework for writing unit tests of code that is
> supposed to be thread safe. This framework should let a developer
> easily write a test with multiple things going on in parallel. The
> framework can then take that code and try to run it with different
> thread interleavings.
> 
> Here's an example of what this could look like:
> 
> @RunWith(ConcurrentTestRunner.class)
> public class AtomicIntegerTest {
> 
>  @Test
>  public void parallelIncrementReturns2(ParallelExecutor executor)
>  throws ExecutionException, InterruptedException {
>AtomicInteger atomicInteger = new AtomicInteger();
>executor.inParallel(() -> atomicInteger.incrementAndGet());
>executor.inParallel(() -> atomicInteger.incrementAndGet());
>executor.execute();
>assertEquals(2, atomicInteger.get());
>  }
> 
> 
> 2) We implement this framework initially using Java Pathfinder, but
> allow for other methods of testing the code to be plugged in for
> example just running the test in the loop. Java pathfinder is cool
> because it can run the code with different interleavings but it does
> have some serious limitations.
> 
> I've put together some code for this proposal which is available in
> this github PR:
> 
> https://github.com/apache/geode/pull/787
> 
> What do you think?
> 
> -Dan

Re: [Discuss] Investigation of C++ object return types

[Michael William Dodge]

+0 shared pointer

> On 14 Sep, 2017, at 14:09, Ernest Burghardt <eburgha...@pivotal.io> wrote:
> 
> Calling a vote for:
> 
> - Raw pointer
> - shard pointer
> 
> +1 raw Pointer, I had to look up RVO and am new to std::move(s)
> 
> On Thu, Sep 14, 2017 at 3:02 PM, Michael William Dodge <mdo...@pivotal.io>
> wrote:
> 
>> I generally dig reference-counted pointers for avoiding lifetime issues
>> with objects allocated off the heap but I can live with bare pointers, too.
>> 
>> Sarge
>> 
>>> On 13 Sep, 2017, at 16:25, Mark Hanson <mhan...@pivotal.io> wrote:
>>> 
>>> Hi All,
>>> 
>>> I favor the “pointer" approach that is identified in the code sample.
>> There is greater clarity and less bytes seemingly created and written. We
>> do sacrifice the potential ease of using an object, but in all, I think the
>> way our code is structured. It is not conducive to do a value approach,
>> from an efficiency standpoint,  because of our use of the pimpl model.
>>> 
>>> Thanks,
>>> Mark
>>> 
>>>> On Sep 12, 2017, at 11:09 AM, Jacob Barrett <jbarr...@pivotal.io>
>> wrote:
>>>> 
>>>> My biggest concern with this model is that access to the public Cache
>>>> object from other public objects results in additional allocations of a
>>>> Cache value. Think about when we are inside a Serializable object and we
>>>> access the Cache from DataOutput.
>>>> 
>>>> As value:
>>>> Serializable* MyClass::fromData(DataInput& dataInput) {
>>>> auto cache = dataOutput.getCache();
>>>> ...
>>>> }
>>>> In this the value of cache will RVO the allocation of Cache in the
>> getCache
>>>> call into the stack of this method, great. The problem is that Cache
>> must
>>>> contain a std::shared_ptr which means that each allocation
>> is 8
>>>> bytes (pointer to control block and pointer to CacheImpl) as well as
>> having
>>>> to increment the strong counter in the control block. On exit/descope,
>> the
>>>> Cache will have to decrement the control block as well.
>>>> 
>>>> Using current shared_ptr pimple model:
>>>> Serializable* MyClass::fromData(DataInput& dataInput) {
>>>> auto& cache = dataOutput.getCache();
>>>> ...
>>>> }
>>>> We only suffer the ref allocation of 4 bytes and no ref count. Since
>> this
>>>> function call can't survive past the lifespan of Cache/CacheImpl they
>> don't
>>>> need to have shared_ptr and refcounting.
>>>> 
>>>> Given that this method could be called numerous times is the overhead of
>>>> the value version going to be a significant performance issue?
>>>> 
>>>> I worry that moves and RVO is just beyond most developers. Heck I didn't
>>>> know anything about it until we started exploring it.
>>>> 
>>>> 
>>>> 
>>>> -Jake
>>>> 
>>>> 
>>>> On Tue, Sep 12, 2017 at 8:06 AM David Kimura <dkim...@pivotal.io>
>> wrote:
>>>> 
>>>>> Follow up of attached discussion after more investigation.  I created
>> an
>>>>> example of returning Cache as shared pointer versus raw value:
>>>>> 
>>>>>  https://github.com/dgkimura/geode-native-sandbox
>>>>> 
>>>>> I still like returning by value as it lets the user do what they want
>> with
>>>>> their object.
>>>>> 
>>>>>  // Here user creates object on their stack.
>>>>>  auto c = CacheFactory::createFactory().create();
>>>>> 
>>>>>  // Here user creates smart pointer in their heap.
>>>>>  auto cptr =
>>>>> std::make_shared(CacheFactory::createFactory().create());
>>>>> 
>>>>> Difficulty of implementing this is high due to circular dependencies of
>>>>> Cache/CacheImpl as well as objects hanging off CacheImpl that return
>>>>> Cache.  We must be extra careful when dealing with move/copy semantics
>> of
>>>>> Cache/CacheImpl.
>>>>> 
>>>>> Alternative, is to keep as is and only permit heap allocations from
>>>>> factory using shared pointers.
>>>>> 
>>>>> Thanks,
>>>>> David
>>>>> 
>>> 
>> 
>> 

Re: [DISCUSS] geode-native c++ exceptions

[Michael William Dodge]

+1 for avoiding multiple inheritance

> On 14 Sep, 2017, at 14:23, Ernest Burghardt <eburgha...@pivotal.io> wrote:
> 
> Sounds like the proposal currently stands to avoid the DiamondOfDeath or
> TriangleOfLove that multiple inheritance brings us
> and just have the base Exception class inherit std::exception and extend
> Exception class as appropriate within the library.
> 
> +1  and thanks for the code examples, very illistrative!
> 
> +1 to Boost stack trace as well...
> 
> 
> On Thu, Sep 14, 2017 at 10:10 AM, Jacob Barrett <jbarr...@pivotal.io> wrote:
> 
>> The problem stems from the fact that none of the std exceptions virtually
>> inherit from std::exception so you end up in the inheritance triangle of
>> love. I say we avoid the multiple inheritance issues with exceptions by
>> avoiding multiple inheritance altogether in exceptions. See this example.
>> http://coliru.stacked-crooked.com/a/2e32eb021e85
>> 
>> Basically all of our exceptions extend our Exception class which extends
>> std::exception. None extend any of the other std exceptions, like
>> bad_alloc, etc. The downside is you can't catch any std exception other
>> than std::exception but the upside is that this actually works. There were
>> also very few exceptions that actually could extend any std exceptions
>> beyond std::exception.
>> 
>> -Jake
>> 
>> 
>> On Wed, Sep 13, 2017 at 10:52 PM Jacob Barrett <jbarr...@pivotal.io>
>> wrote:
>> 
>>> Let me see if I can considerate the consensus here:
>>> 
>>> As reasonably possible throw library specific exceptions from all public
>>> methods.
>>> 
>>> As reasonably possible those exceptions should extend C++11 standard
>>> library exceptions.
>>> 
>>> All exceptions should extend less specific but logically relevant base
>>> exceptions.
>>> 
>>> A few quick examples:
>>> namespace apache {
>>> namespace geode {
>>> namespace client {
>>> 
>>> class Exception : public std::exception {...};
>>> 
>>> class IllegalArgumentException : public Exception, public
>>> std::invalid_argument {...};
>>> 
>>> class TransactionException : public Exception {...};
>>> 
>>> class RollbackException : public TransactionException {...};
>>> 
>>> }
>>> }
>>> }
>>> 
>>> Additionally, investigate using Boost Stack Track library for providing
>>> stack context in exceptions, otherwise dump the current stack tracing
>>> feature that is incomplete and very platform specific.
>>> 
>>> Does anyone have a different understanding of the consensus?
>>> 
>>> 
>>> I found some problems with this model. The IllegalArgumentException would
>>> not be caught in a catch (const std::exception& e) statement do to
>>> multiple inheritance. Another nasty is std::exception doesn't have a
>>> constructor to populate the what() value. Some examples of this problem
>>> can be seen here:
>>> http://coliru.stacked-crooked.com/a/5b6e34c7ea020fc8
>>> 
>>> 
>>> -Jake
>>> 
>>> 
>>> 
>>> 
>>> On Wed, Sep 13, 2017 at 4:37 PM Mark Hanson <mhan...@pivotal.io> wrote:
>>> 
>>>> I think that it would be best to abide by using the std::exception as
>> the
>>>> base. I think it brings with it all the language support and
>> flexibility.
>>>> There are a few penalties obviously to using std::exception as a base,
>> but
>>>> I think they are sufficiently offset by using a standard. As far as the
>>>> number of exceptions, I believe in the philosophy of using as few as
>>>> possible and using inheritance to drive specificity. The benefit is that
>>>> the code can be as simple or as complex as it can be without unnecessary
>>>> overhead e.g error => network error => tcp error. So, I may just care
>> there
>>>> is a network error and the being able to tell if something derives from
>>>> network error is perfect.
>>>> 
>>>> Thanks,
>>>> Mark
>>>> 
>>>> 
>>>>> On Sep 8, 2017, at 1:35 PM, Ernest Burghardt <eburgha...@pivotal.io>
>>>> wrote:
>>>>> 
>>>>> if we continue the merging of Jake <> Sarge comments we might find
>> that
>>>>> std::exception(s) is sufficient if the many name exceptions that
>>>> pertain to

Re: [DISCUSS] Change signature of Serializable::fromData on Geode-Native

[Michael William Dodge]

+1 for the void return type

> On 14 Sep, 2017, at 13:39, Ernest Burghardt  wrote:
> 
> +1  cleans up the public API and code using this as you can see in the
> proposed changes on Jake's fork
> 
> On Wed, Sep 13, 2017 at 3:30 PM, Jacob Barrett  wrote:
> 
>> I would like to propose a change:
>> Serializable* Serializable::formData(DataInput& in)
>> to
>> void Serializable::formData(DataInput& in)
>> 
>> The current signature allows the object being deserialized to return an
>> object other than itself. The use of this trick is only done internally for
>> a few internal system types in what appears to have been an attempt to make
>> serialization more pluggable. The downside to this approach is that it
>> leaks this ability out to the public interface. Additionally concerning is
>> that the return type is a raw pointer to Serializable but typically the
>> object was created as a std::shared_ptr, which can lead to shard_ptr errors
>> if you don't property check and swap the returned raw pointer against the
>> original shared_ptr. Lastly the return value is a pointer to the most base
>> interface providing little utility and type safety.
>> 
>> A couple of spikes investigated changing the signature to:
>> std::shared_ptr Serializable::formData(DataInput& in)
>> and:
>> template
>> std::shared_ptr Serializable::formData(DataInput& in)
>> But both approaches left some dirty things laying around. First the
>> templated version just caused all sorts of pain and failed when the value
>> was replaced on the fromData. The more generic share_ptr
>> approach uncovered a plethora of places internally that we use the
>> Serializable::fromData to deserialize objects as parts of protocol messages
>> and used as internal data where they weren't originally created as
>> shared_ptrs, so the opposite problem to what we were trying to solve.
>> 
>> The final spike investigated using void. In doing so we only had to make
>> small changes to the way PDX types were being deserialized. The void
>> signature is also more consistent with the Java DataSerializable interface.
>> By making it void the ambiguity of using or checking the return value goes
>> away.
>> 
>> You can see the proposed changes on my fork at
>> https://github.com/pivotal-jbarrett/geode-native/tree/wip/fromDataVoid.
>> 
>> Thoughts?
>> 
>> -Jake
>> 

Re: [Discuss] Investigation of C++ object return types

[Michael William Dodge]

I generally dig reference-counted pointers for avoiding lifetime issues with 
objects allocated off the heap but I can live with bare pointers, too.

Sarge

> On 13 Sep, 2017, at 16:25, Mark Hanson  wrote:
> 
> Hi All,
> 
> I favor the “pointer" approach that is identified in the code sample. There 
> is greater clarity and less bytes seemingly created and written. We do 
> sacrifice the potential ease of using an object, but in all, I think the way 
> our code is structured. It is not conducive to do a value approach,  from an 
> efficiency standpoint,  because of our use of the pimpl model. 
> 
> Thanks,
> Mark
> 
>> On Sep 12, 2017, at 11:09 AM, Jacob Barrett  wrote:
>> 
>> My biggest concern with this model is that access to the public Cache
>> object from other public objects results in additional allocations of a
>> Cache value. Think about when we are inside a Serializable object and we
>> access the Cache from DataOutput.
>> 
>> As value:
>> Serializable* MyClass::fromData(DataInput& dataInput) {
>> auto cache = dataOutput.getCache();
>> ...
>> }
>> In this the value of cache will RVO the allocation of Cache in the getCache
>> call into the stack of this method, great. The problem is that Cache must
>> contain a std::shared_ptr which means that each allocation is 8
>> bytes (pointer to control block and pointer to CacheImpl) as well as having
>> to increment the strong counter in the control block. On exit/descope, the
>> Cache will have to decrement the control block as well.
>> 
>> Using current shared_ptr pimple model:
>> Serializable* MyClass::fromData(DataInput& dataInput) {
>> auto& cache = dataOutput.getCache();
>> ...
>> }
>> We only suffer the ref allocation of 4 bytes and no ref count. Since this
>> function call can't survive past the lifespan of Cache/CacheImpl they don't
>> need to have shared_ptr and refcounting.
>> 
>> Given that this method could be called numerous times is the overhead of
>> the value version going to be a significant performance issue?
>> 
>> I worry that moves and RVO is just beyond most developers. Heck I didn't
>> know anything about it until we started exploring it.
>> 
>> 
>> 
>> -Jake
>> 
>> 
>> On Tue, Sep 12, 2017 at 8:06 AM David Kimura  wrote:
>> 
>>> Follow up of attached discussion after more investigation.  I created an
>>> example of returning Cache as shared pointer versus raw value:
>>> 
>>>   https://github.com/dgkimura/geode-native-sandbox
>>> 
>>> I still like returning by value as it lets the user do what they want with
>>> their object.
>>> 
>>>   // Here user creates object on their stack.
>>>   auto c = CacheFactory::createFactory().create();
>>> 
>>>   // Here user creates smart pointer in their heap.
>>>   auto cptr =
>>> std::make_shared(CacheFactory::createFactory().create());
>>> 
>>> Difficulty of implementing this is high due to circular dependencies of
>>> Cache/CacheImpl as well as objects hanging off CacheImpl that return
>>> Cache.  We must be extra careful when dealing with move/copy semantics of
>>> Cache/CacheImpl.
>>> 
>>> Alternative, is to keep as is and only permit heap allocations from
>>> factory using shared pointers.
>>> 
>>> Thanks,
>>> David
>>> 
> 

Re: [Discuss] Moving away from virtual CacheableStringPtr toString() for Serializable objects in debug and logging to std::string and std::wstring

[Michael William Dodge]

+1 for std::string and std::wstring.

Sarge

> On 14 Sep, 2017, at 11:10, Mark Hanson  wrote:
> 
> Hi All,
> 
> I wanted to broach the subject of moving away from moving away from 
> CacheableStringPtrs for the toString representation of Serializable. It would 
> seem desirable to move to std::string and std::wstring to use more basic 
> types that would be faster to log and the code would be simpler for a user.
> 
> Are there any opinions on this subject? 
> 
> Here is a before and after look at a chunk of code
> 
> Before
> 
> CacheableStringPtr ptr = pdxser->toString();
> if (ptr->isWideString()) {
>  printf(" query idx %d pulled object %S  :: \n", i,
> ptr->asWChar());
> } else {
>  printf(" query idx %d pulled object %s  :: \n", i,
> ptr->asChar());
> }
> 
> After
> 
> 
> if (pdxser->isWideString()) {
>   std::cout << " query idx “ << i << "pulled object ” <<  pdxser->toWString() 
> << std::endl;
> } else {
>   std::cout << " query idx “ << i << "pulled object ” <<  pdxser->toString() 
> << std::endl;
> }
> 
> 
> Thanks,
> Mark

Re: [DISCUSS] geode-native c++ exceptions

[Michael William Dodge]

I subscribe to Josh Gray's philosophy of only having another exception class if 
there is something different to be done when it's caught. For example, if the 
caller would do the exact same thing for NoPermissionsException and 
DiskFullException, just use an IOException and be done with it. I also 
subscribe to the philosophy that a library have its own exception hierarchy 
(possibly with a single class), which I think meshes with Jake's "exceptions 
exiting a library to be reasonably limited to exceptions generated by and 
relating to that library".

Sarge

> On 8 Sep, 2017, at 07:19, Jacob Barrett  wrote:
> 
> Sorry for jumping on this thread so late. This is an important issue we
> need to address.
> 
> On Thu, Aug 17, 2017 at 11:57 AM David Kimura  wrote:
> 
>> Using exceptions seems contrary to the Google C++ Style Guide we adopted,
>> which states: *"do not use C++ exceptions"* [3
>> ].  Here is
>> a
>> link [4 ] to a
>> cppcon
>> talk defending their decision.  Does it make sense to enforce this rule on
>> our code base?
>> 
> 
> I don't agree with this approach, as I always tend towards
> progress/modernization, but it was their choice to remain consistent across
> their code base. I would say if we made the same decision solely on
> consistency then we would have our own exceptions derived from a base
> exception class. This is consistent with our Java and .NET as well as
> current C++ clients.
> 
> 
>> If we decide to knowingly ignore this rule, would we want to continue to
>> propagate custom exceptions or switch to standard exceptions?  At the very
>> least, it seems to me that our custom exceptions should derive from their
>> most closely matching standard exception counterparts.  Thoughts?
>> 
> 
> I agree with your recommendation of using our exceptions but extend the
> most appropriate C++11 exceptions where applicable. I think it is good form
> for exceptions exiting a library to be reasonably limited to exceptions
> generated by and relating to that library.
> 
> Our current Exception class also brings with it stack tracing, although
> poorly supported on some platforms and disabled by default. Boost just
> recently graduated a stack track library that is supported on many
> compilers and platforms. It may be worth integrating into our exceptions as
> a replacement for the current stack tracing code.
> 
> -Jake

Re: [Discuss] What type should C++ object creation functions return?

[Michael William Dodge]

I like just returning the object as that removes the possibility of a null 
pointer.

Sarge

> On 6 Sep, 2017, at 12:27, David Kimura  wrote:
> 
> What type should C++ object creation functions return (e.g. pointer, smart
> pointer, reference, etc.)?  Several of our C++ API's return shared
> pointers.  For example in the following function signature [1]:
> 
>std::shared_ptr CacheFactory::createCacheFactory(...);
> 
> Here the only case I can see for shared pointer is to indicate ownership of
> CacheFactory.  Ideally this should probably be std::unique_ptr because
> callee shouldn't share ownership.  However, I don't see the point of using
> a pointer at all..  I suggest we return the bare object, like the following
> signature:
> 
>CacheFactory CacheFactory::createCacheFactory(...);
> 
> In C++03, this would have been a performance hit because we'd end up with
> an added call to the copy constructor.  In C++11, std::unique_ptr gives
> std::move for free and thus avoids copy-constructor call. However, most
> modern C++11 compilers already perform copy-elision here.  In fact, C++17
> standard dictates that compilers must perform RVO here.  Therefore it
> doesn't seem to me that std::shared_ptr or std::unique_ptr buys us much in
> this situation.
> 
> Thoughts?
> 
> Thanks,
> David
> 
> 
> [1] https://github.com/apache/geode-native/blob/develop/
> cppcache/include/geode/CacheFactory.hpp#L54

Re: git repo is gone

[Michael William Dodge]

Can non-committers use the ssh protocol, too?

Sarge

> On 1 Sep, 2017, at 14:37, Jens Deppe  wrote:
> 
> TL;DR - Remove references to
> https://git-wip-us.apache.org/repos/asf/geode.git and replace/add ssh://
> g...@github.com/apache/geode.git
> 
> In my environment this is what I had for remotes:
> 
> $ git remote -v
> apache https://git-wip-us.apache.org/repos/asf/geode.git (fetch)
> apache https://git-wip-us.apache.org/repos/asf/geode.git (push)
> origin http://github.com/apache/geode.git (fetch)
> origin http://github.com/apache/geode.git (push)
> 
> Then I did:
> 
> $ git remote remove apache
> $ git remote set-url origin ssh://g...@github.com/apache/geode.git
> 
> This adds github as a read/write remote.
> 
> --Jens
> 
> On Fri, Sep 1, 2017 at 2:31 PM, Jens Deppe  wrote:
> 
>> I believe it is. Just looking to see how/what we're supposed to use now.
>> 
>> --Jens
>> 
>> On Fri, Sep 1, 2017 at 2:30 PM, Bruce Schuchardt 
>> wrote:
>> 
>>> I'm not able to access the Apache repo anymore
>>> 
>>> git clone https://git-wip-us.apache.org/repos/asf/geode.git
>>> Cloning into 'geode'...
>>> fatal: repository 'https://git-wip-us.apache.org/repos/asf/geode.git/'
>>> not found
>>> 
>>> Is this part of the gitbox migration?
>>> 
>> 
>> 

Re: [DISCUSS] authorizing function execution

[Michael William Dodge]

What about an annotation for read-only functions or a subinterface off 
org.apache.geode.cache.execute.Function?

Sarge

> On 17 Aug, 2017, at 01:42, Swapnil Bawaskar  wrote:
> 
> Discuss fix for GEODE-2817
> 
> 
> Currently to execute a function, you will need "data:write" permission, but
> it really depends on what the function is doing. For example, if a function
> is just reading data, the function author might want users with DATA:READ
> permissions to execute the function. The two options mentioned in the
> ticket are:
> 
> 1) externalize SecurityService so that function author can use it in the
> function.execute code to check authorization.
> 2) add a method to function interface to tell the framework what permission
> this function needs to execute, so that the framework will check the
> permission before executing the function.
> 
> I vote for #2 because, I think, a function author will be able to easily
> discover a method on the Function interface, rather than trying to look for
> SecurityService.
> 
> I propose that we add the following new method to Function:
> 
> default public List requiredPermissions() {
>   // default DATA:WRITE
> }
> 
> In order to preserve existing behavior, the default required permission
> would be DATA:WRITE.

Re: ACTION REQUIRED: geode-native has moved

[Michael William Dodge]

Does this affect a pre-existing fork?

Sarge

> On 16 Aug, 2017, at 07:24, Jacob Barrett  wrote:
> 
> Devs,
> 
> The geode-native repo has moved to GitHub utilizing the Apache GitBox
> service.
> 
> For contributors nothing really changes, the repo is still at
> github.com/apache/geode-native.
> 
> For committers the origin is now github.com/apache/geode-native. Before you
> can get write access to the new repo you need to "link" your accounts via
> https://gitbox.apache.org/setup/. You should also change the URL of your
> origin from Apache to GitHub. You will also notice we have full control
> over pull requests, including assignments and labeling. Good times ahead.
> 
> Thanks Mark for getting this ball rolling.
> 
> -Jake

Re: []DISCUSS] Using package names to identify public API's

[Michael William Dodge]

The user shouldn't need to access any of the protobuf classes directly. I'm in 
favor of making all of the protobuf-related packages internal, including any 
classes generated from .proto files.

Sarge
 
> On 11 Aug, 2017, at 11:30, Anthony Baker  wrote:
> 
> We have policies in place for versioning [1] and backwards compatibility [2]. 
>  How do we identify which API’s need to be controlled?
> 
> In many cases we use the *.internal.* package naming format to signal API’s 
> that aren’t subject to backwards compatibility requirements.  API’s within 
> these internal packages can change and do change even within minor or patch 
> releases.  If a user creates an application that relies on an internal API, 
> it may need to be changed during an upgrade.
> 
> I’ve noticed that we haven’t been following this convention for some newer 
> changes (such as in geode-protobuf).  Should we review and modify the 
> packages names continue using the *.internal.* format?
> 
> 
> Anthony
> 
> [1] https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=57311457
> [1] 
> https://cwiki.apache.org/confluence/display/GEODE/Managing+Backward+Compatibility
> 

Re: New Committer And PMC Member: Joey McAllister

[Michael William Dodge]

Welcome, Joey!

Sarge

> On 25 Jul, 2017, at 11:58, Mark Bretl  wrote:
> 
> The Apache Geode Project Management Committee has invited Joey McAllister to
> be committer on the project and join the Geode PMC. We are pleased to
> announce he has accepted.
> 
> Please join me in welcoming Joey!
> 
> Best regards,
> 
> Mark
> On behalf of the Apache Geode PMC

Re: PING message is "Mischief Managed"

[Michael William Dodge]

+1 for "pong".

> On 13 Jul, 2017, at 13:45, Kirk Lund  wrote:
> 
> I'm really boring... I would've just had it reply "PONG" ;)
> 
> On Thu, Jul 13, 2017 at 11:57 AM, John Blum  wrote:
> 
>> Here you go...
>> 
>> https://github.com/apache/geode/blob/rel/v1.1.1/geode-
>> core/src/main/java/org/apache/geode/management/internal/web/shell/
>> RestHttpOperationInvoker.java#L151-L204
>> 
>> 
>> On Thu, Jul 13, 2017 at 10:51 AM, John Blum  wrote:
>> 
>>> Corrections below (apologies)...
>>> 
>>> On Thu, Jul 13, 2017 at 10:44 AM, John Blum  wrote:
>>> 
 Yeah, that was my doing. :)
 
 I thought it would be more fun to return a special message than the
 typical "Alive".  "Mischief Managed" comes from *Harry Potter*.
 
 This endpoint is of course benign/idempotent and was purely meant to
>> test
 the Management REST API's availability, or rather that the
>> Manage/Locator
 was still "online".
 
 Unlike JMX RMI, HTTP is stateless.  When a JMX RMI connection is made,
>> it
 is persistent and constantly "connected", where as each HTTP request to
>> the
 Management REST API opens and closes a connection.  Therefore, you have
>> no
 idea whether *Gfsh* is still connected to the Manager between requests
 unlike the JMX RMI connection.
 
 So, I run a background Thread that "polls" this endpoint every 500 ms.
 It might even test the message; I don't remember.  Once the response is
 anything other than 200 OK, then we know there is a problem and that the
 connection was most likely terminated.
 
 Therefore, it keeps the behavior of the HTTP connection between *Gfsh*
 and the Manager similar to the JMX RMI connection by returning...
 
 No longer connected to 10.99.199.10[1099].
 
 gfsh>
 
 
 On Thu, Jul 13, 2017 at 10:36 AM, Jacob Barrett 
 wrote:
 
> For the older crowed I would have rather it replied:
> 
> Ah, I see you have the machine that goes 'ping!'.
> 
> Sent from my iPhone
> 
>> On Jul 13, 2017, at 10:23 AM, Jared Stewart 
> wrote:
>> 
>> I'm young enough to recognize it as a Harry Potter reference, but I
> have no
>> idea what it's doing in our product code.
>> 
>> - Jared
>> 
>>> On Jul 13, 2017 10:14 AM, "Kirk Lund"  wrote:
>>> 
>>> Anyone know why the response to a REST service PING returns
>> "Mischief
>>> Managed!?
>>> 
>>> @RequestMapping(method = {RequestMethod.GET, RequestMethod.HEAD},
> value =
>>> "/ping")
>>> public ResponseEntity ping() {
>>> return new ResponseEntity("Mischief
>>> Managed!",
>>> HttpStatus.OK);
>>> }
>>> 
>>> /Users/klund/dev/geode [949]$ git grep
> 'Mischief
>>> Managed'
>>> geode-core/src/main/java/org/apache/geode/management/
>>> internal/web/controllers/ShellCommandsController.java:
>>>  return new ResponseEntity("Mischief
>>> Managed!",
>>> 
> 
 
 
 
 --
 -John
 john.blum10101 (skype)
 
>>> 
>>> 
>>> 
>>> --
>>> -John
>>> john.blum10101 (skype)
>>> 
>> 
>> 
>> 
>> --
>> -John
>> john.blum10101 (skype)
>> 

Re: What to do with the geode-spark-connector

[Michael William Dodge]

+1 for A

> On 24 May, 2017, at 21:30, Jared Stewart  wrote:
> 
> +1 for A
> 
> On May 24, 2017 6:48 PM, "Kirk Lund"  wrote:
> 
>> +1 for A
>> 
>> On Wed, May 24, 2017 at 5:50 PM, Jianxia Chen  wrote:
>> 
>>> I prefer option A: Move it into it's own repository, with it's own
>> release
>>> cycle.
>>> 
>>> On Wed, May 24, 2017 at 5:17 PM, Dan Smith  wrote:
>>> 
 Our geode-spark-connector needs some work. It's currently building
>>> against
 geode 1.0.0-incubating, because it has it's own separate build process.
 It's also somewhat out of date, we're building against spark 1.3. Is
>>> anyone
 actually using the spark connector?
 
 I think we need to get the spark connector out of the main geode repo
>>> since
 people are currently modifying code in the connector without even
>>> compiling
 it, since it's not linked into the gradle build.
 
 What do the geode devs think we should do with the
>> geode-spark-connector?
 
 A) Move it into it's own repository, with it's own release cycle
 B) Delete it
 C) Other??
 
 -Dan
 
>>> 
>> 

Re: Simple Java Client

[Michael William Dodge]

I like the idea of a separate code base (including repo) for separate problem 
domains. I think trying to fit a new API inside or parallel to the existing API 
has too high of a likelihood of confusion.

Sarge

> On 26 Apr, 2017, at 16:52, Bruce Schuchardt  wrote:
> 
> I don't think we should mix the old client code with a new API.  We should 
> keep the new client code separate from the server.  Maybe even in a different 
> repo, as I think Fred suggested.
> 
> Le 4/26/2017 à 3:12 PM, Kirk Lund a écrit :
>> If we want to add a new API then I suggest we create new API packages and
>> leave the "cache" package as the old deprecated API for backwards
>> compatibility.
>> 
>> The new APIs and implementation could be separated into different geode
>> modules and packages (something like what follows):
>> 
>> org.apache.geode.api -- the main rich API (geode-api module and jar)
>> org.apache.geode.api.client -- the new thin client API (geode-api-client
>> module and jar)
>> org.apache.geode.core -- the main implementation of api (geode-core module
>> and jar which also currently contains the old API)
>> 
>> On Wed, Apr 26, 2017 at 9:41 AM, William Markito Oliveira <
>> william.mark...@gmail.com> wrote:
>> 
>>> This is an awesome discussion and Jake's hitting all the right notes about
>>> why JCache is a good idea! I've fought that fight in the past and lost it
>>> but I'm happy to see it coming back...
>>> 
>>> What's really nice about Geode is that the functionalities and capabilities
>>> are all there, they're just not that well exposed, known by others or
>>> obscured by some decisions that doesn't apply anymore.
>>> 
>>> It's the same conversation about monitoring and statistics...  All the
>>> capability is there with JMX and Stats, but using an unknown custom format
>>> or tool to display that data makes it not very appealing for OSS and
>>> enterprise users that need workarounds and hacks to integrate with common
>>> monitoring tools.
>>> 
>>> Refactoring API Client APIs, documentation and implementation of a new
>>> Protocol, Reactive APIs, better integration with standard monitoring tools
>>> -  Sounds like good points for a 2.0 roadmap IMHO.
>>> 
>>> 
>>> On Wed, Apr 26, 2017 at 10:28 AM, Jacob Barrett 
>>> wrote:
>>> 
 Wes,
 
 Those are almost all administrative commands and have no place on the
 client API. They belong on an administrative API or as I'm arguing a
>>> series
 of MBeans/JMX as it is already an established standard.
 
 -Jake
 
 On Wed, Apr 26, 2017 at 8:09 AM Wes Williams 
>>> wrote:
> Now we're getting some precision. Let's talk about the "raw" Geode
> proprietary bad ass API!  Would that "raw" Geode proprietary bad ass
>>> API
> "raw"
> Geode proprietary bad ass API that we're talking about be centered
>>> around
> the commands found here:
> 
> https://github.com/apache/geode/tree/rel/v1.1.1/geode-
 core/src/main/java/org/apache/geode/management/internal/cli/commands
> Or somewhere else?
> 
> *Wes Williams | Pivotal Advisory **Data Engineer*
> 781.606.0325
> http://pivotal.io/big-data/pivotal-gemfire
> 
> On Tue, Apr 25, 2017 at 11:41 PM, Jacob Barrett 
> wrote:
> 
>> Java and its community have standards for all of these issue so why
>> re-invent the wheel. The market doesn't want proprietary anymore,
>>> they
> want
>> standards and mobility.
>> 
>> Configuration of the server should happen through MBeans. You can
>>> wrap
> that
>> in gfsh for command line, REST for remote web based admin, use
>>> JConsole
> or
>> any other number of JMX based enterprise management tools. By using
> MBeans
>> the server can easily expose new discovered services without the need
 to
>> code specific gfsh commands, REST interfaces or APIs. There is no
 reason
> my
>> SDG can't be retooled to "discover" the configuration from these
>>> MBeans
> as
>> well rather than having to be touched every time we add or change
>> something. There are tools and books already written that
>>> implementors
> can
>> consult on MBeans. There isn't anything out there on gfsh commands.
>> 
>> If we want to play in the Java community, especially J2EE (the other
 50%
> of
>> Java that isn't Spring), then we had better have a JSR-107 answer no
> matter
>> what the pain is to provide it. I can pull dozens of books of the
>>> shelf
>> that teach me how to effectively use a JCache, how many can I pull
>>> off
> the
>> shelf that teach me Geode's API? How many engineers can I get
> applications
>> form by saying "must have Geode API knowledge"? I can find people
>>> with
>> JCache knowledge though. So from in implementor's perspective having
>> standards is a must. Now I don't think the JSR-107 

Re: Simple Java Client

[Michael William Dodge]

Perhaps I'm picking nits, but I think a library that provides an API for 
interacting with Geode isn't a client. (I like to call it a driver.) The client 
is the application that someone write to use that API to interact with Geode. I 
recognize that in the past the C++ library for Geode has been called the 
"native client" but to me the term "client" implies a stand-alone application 
that has user functionality built into it.

Sarge

> On 24 Apr, 2017, at 15:03, Fred Krone  wrote:
> 
> In an effort to improve Java developer ease of use for caching with Geode I
> am looking for feedback on going forward with creating a Java client.  This
> client will allow for server-side region creation and distributed data
> caching.  This would allow for a thin client that fits with microservice
> caching patterns and also abstracts a cleaner client-server experience
> driven interface.
> 
> Initially we were going to update the Region interface but were concerned
> with breaking existing applications.  We also would like to provide Region
> creation to a client application and so propose here solving both of these
> areas with a Java client.
> 
> It would have new project repo for the client.
> 
> It would provide new Region interface for clients.  The specifics of the
> API design are too lengthy for this conversation but implementation will
> resemble JSR 107 Cache interface.
> 
> It would use the new security framework.
> 
> 
> *An example*,
> 
> The client application simply creates an instance of client and points it
> to the locator:
> 
> 
> org.apache.geode.client.Client client = Client.create(locatorHost,
> locatorPort);
> 
> 
> Client has the following methods:
> 
> package org.apache.geode.client;
> 
> 
> public interface GeodeClient {
> 
>  /**
> 
>   * creates the region on the servers, or gets the region if it exits,
> returns a PROXY region
> 
>   */
> 
>  public Region getOrCreateRegion(RegionAttributes attributes, String name);
> 
> 
>  /**
> 
>   * Returns a PROXY region if the region exists on the server
> 
>   */
> 
>  public Region getRegion(String name);
> 
> 
> 
> 
> 
> 
> MVP
> 
> The smallest set of features to test this idea, learn and iterate, and get
> the client into the communities hands for future iterations is:
> 
> 
> Create a server side Region from a client
> 
> Region interface has CRUD on par with javax.cache.Cache (from JSR 107)
> 
> Calls are asynchronous -- futures
> 
> 
> 
> Also would like feedback on which future functionality would be most useful
> from a thin client:
> 
> Function execution
> 
> Durable clients
> 
> User defined serialization
> 
> Register interest
> 
> Queries
> 
> CQ
> 
> Near side caching
> 
> Create disk stores from client
> 
> Region group membership
> 
> Client subscription load balancing
> Transactions
> 
> 
> Thanks,
> -Fred

Re: Why we shouldn't use RPC Frameworks for the New Geode Protocol

[Michael William Dodge]

MQTT is an interesting idea but from the wiki page it sounds like it depends on 
a broker. In that architecture, would the server be the broker as well as a 
publisher and subscriber? Would the locator be the broker? Or would we 
introduce a separate broker, either third-party or bespoke?

Sarge

> On 10 Apr, 2017, at 13:21, Michael Stolz  wrote:
> 
> I am wondering why we are leaning so heavily toward RPC IDL rather than
> messaging standards.
> 
> One of the big features of the client-server discussion around Geode is the
> ability to register interest and run Continuous Queries.
> Both of these behave more like messaging than RPCs.
> 
> Beyond that all that Geode really does is puts and gets and function calls.
> A put is analogous to a publish. A get is similar to a subscribe. A
> function call can be implemented as a put on a special topic that has a
> callback attached to it. In fact that's how we used to do server side
> functions before we added the function execution api.
> 
> The other thing we are constantly battling with is being able to push more
> and more data into Geode from clients faster and faster.
> That too lends itself to a messaging protocol.
> 
> In fact, I remember that last year we were already having some discussions
> about maybe developing a client based on MQTT.
> That would make GemFire a natural for the Internet-of-Things and for mobile
> devices.
> I wonder if it would be sufficient for a full-blown .Net GemFire client.
> 
> One of our goals in this thread is to be able to have clients in many
> languages.
> Well, there are at least 75 different language bindings for MQTT already
> out in the wild.
> 
> MQTT is agnostic about what format the payload is in, so we could support
> PDX if we choose to, or ProtoBufs or FlatBuffers or whatever else for
> payload serialization.
> 
> Thoughts?
> 
> 
> --
> Mike Stolz
> Principal Engineer, GemFire Product Manager
> Mobile: +1-631-835-4771
> 
> On Mon, Apr 10, 2017 at 2:39 PM, Galen M O'Sullivan 
> wrote:
> 
>> On Mon, Apr 10, 2017 at 10:47 AM, Bruce Schuchardt >> 
>> wrote:
>> 
>>> I agree that key/value serialization is a separate issue and more related
>>> to storage than communications.  The thing I'm struggling with is how to
>>> specify message metadata in an RPC IDL.  I'm thinking of things like an
>>> eventID, transaction info, security principal, etc.  The basic
>>> client/server messaging doesn't need to know the details of this
>>> information - just that it exists and maybe the ID of each piece of
>>> metadata.
>>> 
>> 
>> Is there any reason that this data couldn't be packed into, say, Thrift
>> types? It's all numbers, right?
>> 

Re: Client Commands and SecurityService

[Michael William Dodge]

+1 to modular and questioning non-constant use of static

> On 3 Apr, 2017, at 09:27, Anthony Baker  wrote:
> 
> Using singletons leads to very monolithic systems that are hard to test and 
> hard to change.  Instead we should prefer modular services like Udo proposed.
> 
> I would go further and say that we should question any non-constant use of 
> “static”.
> 
> Anthony
> 
>> On Apr 3, 2017, at 9:01 AM, Udo Kohlmeyer  wrote:
>> 
>> Correct, that would be the definition.
>> 
>> Yet, we find that our use of singletons within Geode is limiting to say that 
>> least. With the idea of wanting to be able to create/run multiple cache 
>> instance within the same JVM (especially for testing) a singleton will be 
>> problematic.
>> 
>> In addition to that, the alternative is not that hard to construct and in 
>> many cases easier to manage.
>> 
>> --Udo
>> 
>> 
>> On 4/3/17 08:57, Jinmei Liao wrote:
>>> Isn't "that instance is reused each invocation" my understanding of a
>>> "singleton"?
>>> 
>>> On Mon, Apr 3, 2017 at 11:49 AM, Udo Kohlmeyer 
>>> wrote:
>>> 
 -1 For using singletons
 
 Using a Factory pattern you can avoid having to create singletons in
 addition to caching created commands to avoid the recreation of the
 instance.
 
 The SSLConfigurationFactory is a simple example where you create an
 instance when required. Once an instance is created, that instance is
 reused each invocation.
 
 --Udo
 
 
 
 On 4/3/17 08:30, Jinmei Liao wrote:
 
> I think the client commands needs to be singleton instances even after you
> change the sequence of initialization. We don't want to have each client
> operation ends up creating a new command instance, right? That would be a
> more performance drag.
> 
> On Thu, Mar 30, 2017 at 2:14 PM, Kirk Lund  wrote:
> 
> PS: I'll be writing and using JMH benchmarks to drive these changes. I'll
>> also create new unit tests for each of these classes that don't currently
>> have unit tests.
>> 
>> On Thu, Mar 30, 2017 at 10:58 AM, Kirk Lund  wrote:
>> 
>> The client Commands now check with SecurityService even when security is
>>> not configured. This has introduced a negative performance impact.
>>> 
>>> The best way to fix something like this is to tell the Command instance
>>> when it's being constructed that is does or does not need to perform
>>> security checks.
>>> 
>>> Unfortunately, Commands are all implemented as singletons which are very
>>> eagerly instantiated during class loading of CommandInitializer (who
>>> thought that was a good idea?!).
>>> 
>>> In order to fix this performance problem, I would need to get rid of
>>> 
>> these
>> 
>>> problematic static initializer blocks that so eagerly construct the
>>> Commands so that I can put off constructing them until AFTER the Cache
>>> is
>>> initializing and specifically AFTER the Cache has determined if it is
>>> 
>> using
>> 
>>> security or not.
>>> 
>>> This means I'm going to have to do some refactoring of
>>> 
>> CommandInitializer,
>> 
>>> the Command classes, ServerConnection, AcceptorImpl, etc.
>>> 
>>> Any other approach is going to have such minimal impact on performance
>>> that I'm not even interested in doing less than this.
>>> 
>>> From a very high level, I would change the code so that the Cache owns
>>> 
>> the
>> 
>>> Server which owns the Command instances. In this way, the configuring of
>>> use of security can trickle down from Cache to each Command. I would
>>> primarily be altering static singletons, static initializers and adding
>>> constructors.
>>> 
>>> Does anyone have a problem with me changing the above classes and
>>> especially getting rid of the static initializers and singleton
>>> 
>> instances?
>> 
>>> 
> 
>>> 
>> 
> 

Re: Wait on macOS 10.12.4 and Xcode 8.3

[Michael William Dodge]

With Dave Barnes' help, we have determined that Xcode 8.3 is the problem. If 
you have already upgraded to Xcode 8.3, you can rollback to Xcode 8.2.1 to 
restore functionality.

Sarge

> On 29 Mar, 2017, at 15:25, Michael William Dodge <mdo...@pivotal.io> wrote:
> 
> With the recent upgrade to macOS 10.12.4 and Xcode 8.3, we're seeing a 
> SIGSEGV/EXC_BAD_ACCESS with pthread_mutex_lock() which is part of the 
> initialization of the ACE libraries. I haven't done enough debugging to know 
> whether it's caused by an incompatibility between ACE and the 10.12.4 
> runtimes (e.g., a struct has changed in size) or if it's an actual bug in the 
> runtimes. Mike Martell found a bug that has already been filed about Emacs 
> crashing with pthread-related problems after this update, so our current 
> theory is that there's a problem in the pthreads library from Apple.
> 
> Sarge

Re: Wait on macOS 10.12.4 and Xcode 8.3

[Michael William Dodge]

GEODE-2733

> On 29 Mar, 2017, at 19:44, Jacob Barrett <jbarr...@pivotal.io> wrote:
> 
> Can you please open a JIRA so we can track this issue, should it require an
> update to ACE or our own sources.
> 
> Thanks,
> Jake
> 
> On Wed, Mar 29, 2017 at 3:25 PM Michael William Dodge <mdo...@pivotal.io>
> wrote:
> 
>> With the recent upgrade to macOS 10.12.4 and Xcode 8.3, we're seeing a
>> SIGSEGV/EXC_BAD_ACCESS with pthread_mutex_lock() which is part of the
>> initialization of the ACE libraries. I haven't done enough debugging to
>> know whether it's caused by an incompatibility between ACE and the 10.12.4
>> runtimes (e.g., a struct has changed in size) or if it's an actual bug in
>> the runtimes. Mike Martell found a bug that has already been filed about
>> Emacs crashing with pthread-related problems after this update, so our
>> current theory is that there's a problem in the pthreads library from Apple.
>> 
>> Sarge

Wait on macOS 10.12.4 and Xcode 8.3

[Michael William Dodge]

With the recent upgrade to macOS 10.12.4 and Xcode 8.3, we're seeing a 
SIGSEGV/EXC_BAD_ACCESS with pthread_mutex_lock() which is part of the 
initialization of the ACE libraries. I haven't done enough debugging to know 
whether it's caused by an incompatibility between ACE and the 10.12.4 runtimes 
(e.g., a struct has changed in size) or if it's an actual bug in the runtimes. 
Mike Martell found a bug that has already been filed about Emacs crashing with 
pthread-related problems after this update, so our current theory is that 
there's a problem in the pthreads library from Apple.

Sarge

Re: Permission to Edit and Comment

[Michael William Dodge]

Thanks!

> On 29 Mar, 2017, at 12:16, William Markito Oliveira 
> <william.mark...@gmail.com> wrote:
> 
> You should be good to go now Sarge... :)
> 
> On Wed, Mar 29, 2017 at 1:19 PM, Michael William Dodge <mdo...@pivotal.io>
> wrote:
> 
>> May I have the necessary permissions such that I can edit and comment on
>> wiki articles under https://cwiki.apache.org/confluence/display/GEODE/ <
>> https://cwiki.apache.org/confluence/display/GEODE/> ? Thanks.
>> 
>> Sarge
> 
> 
> 
> 
> -- 
> ~/William

Permission to Edit and Comment

[Michael William Dodge]

May I have the necessary permissions such that I can edit and comment on wiki 
articles under https://cwiki.apache.org/confluence/display/GEODE/ 
 ? Thanks.

Sarge

Re: Building geode-native for release

[Michael William Dodge]

Can we not just use http://geode.apache.org/schema/cache/cache-1.0.xsd 
 which is already there?

Sarge

> On 6 Mar, 2017, at 09:56, Anthony Baker  wrote:
> 
> 2) Schema
> 
> We need to move gfcpp-cache-9.0.xsd to http://geode.apache.org/schema 
> .

Re: [GitHub] geode-native issue #44: GEODE-2578: Remove 64 KiB limit on query strings.

[Michael William Dodge]

They didn't for me before I committed the code. Let me try them again.

> On 6 Mar, 2017, at 08:03, pivotal-jbarrett  wrote:
> 
> Github user pivotal-jbarrett commented on the issue:
> 
>https://github.com/apache/geode-native/pull/44
> 
>@PivotalSarge looks like some unit tests are failing.
> 
> 
> ---
> If your project is set up for it, you can reply to this email and have your
> reply appear on GitHub as well. If your project does not have this feature
> enabled and wishes so, or if the feature is enabled but not working, please
> contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
> with INFRA.
> ---

Re: C++ Style

[Michael William Dodge]

* Variable Naming - I agree that lowercase-first camelCase is more readable and 
preferable for variable and data member names. Since class names are 
uppercase-first CamelCase, underscores should not be used. And alllowercase is 
unreadable.

* Class Data Members - Any sort of Hungarian notation, including the leading 
"m_", is more trouble than it's worth, e.g., the compiler doesn't enforce m_foo 
actually being a data member. For those who have an aversion to this->foo, foo_ 
would be much better than m_foo. But I think neither a leading "m_" or trailing 
"_" is needed.

* Constant Names - I always associated the leading "k" with Smalltalk since I 
first saw it in Objective-C but apparently it's originally from mathematics 
somehow. Just like with the rest of Hungarian notation, it doesn't mean 
anything to the compiler and I think it makes the code harder to read. It is 
unnecessary.

* Reference vs. Pointer - I'll admit I don't see the logic to their argument. 
As far as I know, compilers pass the address for both references and pointers 
so there's no difference there. From a language standpoint, however, there is 
the very important difference that references may not be null. Thus, pointers 
(both const and non-const) should be used in cases where the value may be 
entirely absent (e.g., there is no foo) and references (both const and 
non-const) used in all other cases. There may be special cases (e.g., mocks for 
Google Test) where a pointer to an interface is preferable but as a general 
rule the precondition that a reference can not be null makes them preferable.

Sarge

> On 20 Feb, 2017, at 13:48, Jacob Barrett  wrote:
> 
> A bit back we discussed and adopted the Google C++ Style Guide. As we dig
> deeper into the C++ sources we find some striking differences in some of
> the conventions that we may want to discuss and address before tackling
> them.
> 
> *Variable Naming*
> Google style dictates no *camelCase*, use either *alllowercase* or
> *variable_name.
> *Our C++ code has a mix of all three. Personally I prefer cameCase as more
> readable.
> 
> *Class Data Members*
> Google style says members must end with underscore, *my_member_*. While I
> find this preferable to the common practice in our code of *m_* prefix,
> like *m_myVariable,* I am not super fond of any decoration of member
> variables.
> 
> *Constant Names*
> Google says prefix with *k* and gives and example with *kCamelCase*. I
> think *cameCase* might be a typo but again I am not fond of any variable
> decorations.
> 
> *Reference vs. Pointer*
> Google says use pointer if you intend to modify the value being passed and
> use const references for values that are not going to be modified. It says
> do not use references except for very limited use cases. We have references
> and pointers spread inconsistently throughout the source. Worst, it is not
> consistent in the public API. We should decide on a standard and make sure
> our API adheres to it first.
> 
> Others may pop up as we go but these are the obvious ones standing out.
> 
> -Jake

Re: [GitHub] geode-native pull request #13: GEODE-2476: Replace gfcpp with geode.

[Michael William Dodge]

I've modified PR #13 to remove the vestigial executable and to elide the 
duplicate GEODE_ in the legacy include guards.

Sarge

> On 17 Feb, 2017, at 11:06, pivotal-jbarrett  wrote:
> 
> Github user pivotal-jbarrett commented on a diff in the pull request:
> 
>https://github.com/apache/geode-native/pull/13#discussion_r101825168
> 
>--- Diff: src/cppcache/include/geode/AttributesFactory.hpp ---
>@@ -1,7 +1,7 @@
> #pragma once
> 
>-#ifndef GEODE_GFCPP_ATTRIBUTESFACTORY_H_
>-#define GEODE_GFCPP_ATTRIBUTESFACTORY_H_
>+#ifndef GEODE_GEODE_ATTRIBUTESFACTORY_H_
>--- End diff --
> 
>I see an argument that `include/geode/` is the root for the public 
> headers, therefore the guards should be `GEODE_ATTRIBUTESFACTORY_H_`.
> 
> 
> ---
> If your project is set up for it, you can reply to this email and have your
> reply appear on GitHub as well. If your project does not have this feature
> enabled and wishes so, or if the feature is enabled but not working, please
> contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
> with INFRA.
> ---

Re: Requesting more JIRA permission

[Michael William Dodge]

+1

> On 15 Feb, 2017, at 16:17, Addison Huddy  wrote:
> 
> Hello Apache Geode Community,
> 
> I'm writing to request a higher level of permission on JIRA tickets in an
> effort to increase organization and clarity within our community.
> 
> As we grow our community and focus our mutual development efforts, it is
> important to stay organized. For example:
> 
>   - how we organize jira's and sub-jiras
>   - Updating the status of a ticket
>   - What we call the ticket so that it is informative
> 
> In my daily work on Geode at Pivotal Inc, I find myself wanting to organize
> things within the community, but don't have the permissions to edit tickets
> besides by own.
> 
> I fully recognize that with more ticket permissions comes a greater
> responsibility to communicate with the community and uphold our democratic
> model.  I will make sure to keep and promote these values.
> 
> *Please +1 this request if you agree to increase my JIRA permissions so I
> can edit JIRAs.*
> 
> Thanks,
> Addison

Re: Voting Process

[Michael William Dodge]

+0.9 :)

> On 15 Feb, 2017, at 14:48, Anthony Baker  wrote:
> 
>> 
>> On Feb 15, 2017, at 2:40 PM, Mark Bretl  wrote:
>> 
>> Hi,
>> 
>> Now that we are a top-level project, we need to make sure votes are counted
>> correctly, according to the Apache Voting Process [1]. For this past
>> release vote, we only had PMC votes, however, that could change in the
>> future as we would like to encourage community feedback. In the future, i
>> think we should clarify binding vs non-binding votes on the vote result
>> thread.
>> 
>> Best regards,
>> 
>> --Mark
>> 
>> [1] http://apache.org/foundation/voting.html
> 
> +1
> 
> :-)

Re: PROXY and CACHING_PROXY regions on Client

[Michael William Dodge]

I agree with Mike that whatever changes in behavior are made to the Java client 
library should also be made in the C++ and C# libraries.

Sarge

> On 15 Feb, 2017, at 08:02, Michael Stolz  wrote:
> 
> I have strong fears that if we make these wholesale changes to existing
> APIs we're going to end up breaking lots of existing code.
> 
> For instance, if we make destroyRegion propagate when it never did before,
> we may end up destroying a server side region in production that wasn't
> expected.
> 
> I will advocate for being more explicit about operations that are going to
> be performed on the server.
> 
> The other fear I have is that if we make all of these server side
> operations available to the Java client but not to the C++ and C# clients
> we will once again be making our C++ and C# users feel orphaned.
> 
> 
> --
> Mike Stolz
> Principal Engineer, GemFire Product Manager
> Mobile: +1-631-835-4771
> 
> On Wed, Feb 15, 2017 at 9:44 AM, Swapnil Bawaskar 
> wrote:
> 
>> GEODE-1887  was filed to
>> make sure that the user experience while using Geode is similar to RDBMS
>> and other data products out there. While reviewing the pull request
>>  I realized that we need to make
>> other operations propagate to the server as well. These include:
>> - invalidateRegion()
>> - destroyRegion()
>> - getSnapshotService()
>> - getEntry()
>> - keySet()
>> - values()
>> - isDestroyed()
>> - containsValueForKey()
>> - containsKey()
>> - containsValue()
>> - entrySet()
>> 
>> Also, rather than have a user "create" a PROXY region, which is just a
>> handle to a server side region, I would like to propose that
>> clientCache.getRegion("name") actually creates and returns a PROXY region
>> even if one was not created earlier/through cache.xml. So, in summary, the
>> workflow on the client would be:
>> 
>> ClientCacheFactory cacheFactory = new ClientCacheFactory();
>> cacheFactory.addPoolLocator("localhost", 10334);
>> ClientCache clientCache = cacheFactory.create();
>> 
>> Region students = clientCache.getRegion("students");
>> students.put("student1", "foo");
>> assert students.size() == 1;
>> 
>> If a client wants to have a near cache, they can still "create" a
>> CACHING_PROXY region.
>> 
>> For a CACHING_PROXY, I propose that we leave the default implementation
>> unchanged, i.e. all operations work locally on the client (except CRUD
>> operations that are always propagated to the server). In the case where the
>> client wishes to perform operations on the server, I propose that we
>> introduce a new method:
>> 
>> /**
>> * @return
>> */
>> Region serverView();
>> 
>> so that all operations on the returned view (Region) are performed on the
>> server.
>> 
>> In the longer term, we should break up Region into two interfaces, one that
>> has methods that only work on the client (like registerInterest and
>> serverView()) and other for the server.
>> 
>> Thanks!
>> Swapnil.
>> 

Re: [DISCUSS] JIRA guidelines

[Michael William Dodge]

+1

> On 14 Feb, 2017, at 20:50, William Markito Oliveira 
>  wrote:
> 
> +1 
> 
> Finally!! ;)
> 
> Sent from my iPhone
> 
>> On Feb 14, 2017, at 7:59 PM, Galen M O'Sullivan  
>> wrote:
>> 
>> +1 to the article and removing the draft label
>> 
>>> On Tue, Feb 14, 2017 at 4:05 PM, Akihiro Kitada  wrote:
>>> 
>>> I agree!
>>> 
>>> 
>>> --
>>> Akihiro Kitada  |  Staff Customer Engineer |  +81 80 3716 3736
>>> Support.Pivotal.io   |  Mon-Fri  9:00am to
>>> 5:30pm JST  |  1-877-477-2269
>>> [image: support]  [image: twitter]
>>>  [image: linkedin]
>>>  [image: facebook]
>>>  [image: google plus]
>>>  [image: youtube]
>>> 
>>> 
>>> 
>>> 2017-02-15 8:47 GMT+09:00 Dan Smith :
>>> 
 We have this draft of JIRA guidelines sitting on the wiki. I updated it
 slightly. Can we agree on these guidelines and remove the draft label? Is
 there more that needs to be here?
 
 https://cwiki.apache.org/confluence/pages/viewpage.
>>> action?pageId=57311462
 
 -Dan
 
>>> 

Re: C++ client change history

[Michael William Dodge]

I'm not sure exactly what happened as my git fu isn't that great but could it 
be as part of commit 4fa64db926f51d4b12d6e4040c703cc69a9832fe? In that commit I 
see a block under case TcrMessage::PING: being removed so that execution falls 
through to default: but I'm unsure that I've pieced the output from git diff 
together properly so that may not be a change that happened in 
TcrMessage::handleByteArrayResponse.

Sarge

> On 12 Feb, 2017, at 09:25, Avital Amity  wrote:
> 
> Hi,
> 
> I'm trying to track the history of TcrMessage.cpp but I can find it only in 
> the new client release where it moved under cppcache/src
> In particular I'm searching for the change where in function 
> TcrMessage::handleByteArrayResponse
> Where the case of PING message was merged with the case of the default message
> 
> Thanks
> Avital
> This message and the information contained herein is proprietary and 
> confidential and subject to the Amdocs policy statement,
> 
> you may review at http://www.amdocs.com/email_disclaimer.asp

Re: Automating docker builds for geode-native

[Michael William Dodge]

I think this would be a good idea as configuring a machine for building can be 
a barrier to entry for people wishing to contribute.

Sarge

> On 10 Feb, 2017, at 12:29, Anthony Baker  wrote:
> 
> The geode-native build, like most c++ projects, requires a fairly specific 
> toolchain.  Now that we have a docker build environment [1], I’d like to ask 
> INFRA to automate the creation and publishing of docker images for 
> geode-native.  This can be done by integrating GitHub / DockerHub [2].  Note 
> that the docker image would *only* be for build purposes and would not 
> contain source or binaries from geode-native.  By publishing our build 
> toolchain in a docker image:
> 
> 1) it makes contributing easier
> 2) it makes our travis-ci builds faster (currently at ~30min) 
> 3) it paves the way to create a nightly Jenkins job for geode-native
> 
> I suggest publishing this image under the apache namespace [3] as 
> geode-native-build.  Thoughts?
> 
> Anthony
> 
> [1] https://github.com/apache/geode-native/blob/develop/docker/Dockerfile
> [2] 
> https://issues.apache.org/jira/browse/INFRA-11584?jql=project%20%3D%20INFRA%20AND%20text%20~%20docker
> [3] https://hub.docker.com/u/apache/

Re: [GitHub] geode pull request #391: GEODE-2422: Switch remaining GemFire strings to Geo...

[Michael William Dodge]

I filed a separate JIRA for this as there is no geode.jar and 
KillJavaProcesses() will not work correctly.

> On 8 Feb, 2017, at 14:47, dgkimura  wrote:
> 
> Github user dgkimura commented on a diff in the pull request:
> 
>https://github.com/apache/geode/pull/391#discussion_r100191055
> 
>--- Diff: src/clicache/integration-test/CacheHelperN.cs ---
>@@ -2389,7 +2389,7 @@ public static void KillJavaProcesses()
>   string commandline = item["CommandLine"].ToString();
> 
>   Util.Log("processId:{0} name:{1}", item["ProcessId"], 
> item["Name"]);
>-  if (commandline.Contains("geode.jar"))
>+  if (commandline.Contains("gemfire.jar"))
>--- End diff --
> 
>I'm curious why this one you're switching geode with gemfire.  What makes 
> this one different?
> 
> 
> ---
> If your project is set up for it, you can reply to this email and have your
> reply appear on GitHub as well. If your project does not have this feature
> enabled and wishes so, or if the feature is enabled but not working, please
> contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
> with INFRA.
> ---

Re: Contributor Permissions in Jira

[Michael William Dodge]

It could very well be pilot error but I could swear I'm seeing some of the 
buttons (like "Resolve Issue") come and go. But it's working for now. Thanks.

Sarge

> On 7 Feb, 2017, at 14:46, Michael William Dodge <mdo...@pivotal.io> wrote:
> 
> As I'm working on the native code, may I have contributor permission in Jira 
> so that I may resolve things that I've fixed. Thanks.
> 
> Sarge

Re: Contributor Permissions in Jira

[Michael William Dodge]

PivotalSarge

> On 7 Feb, 2017, at 14:57, Dan Smith <dsm...@pivotal.io> wrote:
> 
> Hi Sarge,
> 
> What's your JIRA username?
> 
> -Dan
> 
> On Tue, Feb 7, 2017 at 2:46 PM, Michael William Dodge <mdo...@pivotal.io>
> wrote:
> 
>> As I'm working on the native code, may I have contributor permission in
>> Jira so that I may resolve things that I've fixed. Thanks.
>> 
>> Sarge

Contributor Permissions in Jira

[Michael William Dodge]

As I'm working on the native code, may I have contributor permission in Jira so 
that I may resolve things that I've fixed. Thanks.

Sarge

Re: gfsh over http & authentication

[Michael William Dodge]

A rule I've heard for UX is that anything over 200 ms is noticeable and 
anything over 2 s is slow. Unless polling every 500 ms is causing problems, it 
might be best to leave it at 500 ms as a decent compromise between efficiency 
and responsiveness.

Sarge

> On 2 Feb, 2017, at 12:04, John Blum  wrote:
> 
>> The connection probably doesn't need to poll every 500ms.
> 
> 500 ms provided a good (nearly consistent) UX for the user to know almost
> instantly that the Manager went away, like the JMX counterpart.  2s is
> arguable; 5s is probably too long as the user could already be typing
> another command that is not available.  In that case they might get another
> kind of error (don't recall for sure).  Anyway, food for thought.
> 
> If another endpoint is needed (though I cannot imagine why) perhaps `
> securePing()` would be more descriptive and still offer up an alternative
> route.
> 
> 
> 
> On Thu, Feb 2, 2017 at 11:55 AM, Kevin Duling  wrote:
> 
>> Good to know some history on it.  The connection probably doesn't need to
>> poll every 500ms.  I would think 2 seconds or even 5 seconds would be
>> sufficient in the general case.
>> 
>> If we make ping require authentication, it may resolve the issue.  But I'm
>> not sure that's the right thing to do.  We could create a 'ping2' endpoint
>> (with some better name that I cannot currently think of) that does require
>> auth for this thread to validate the connection.
>> 
>> On Thu, Feb 2, 2017 at 11:49 AM, John Blum  wrote:
>> 
>>> Back in the day, I introduced this "polling thread" to determine whether
>>> *Gfsh* was still connected, since as you say, in a HTTP "stateless"
>>> environment and in the absence of a "persistent" connection, it otherwise
>>> does not know.
>>> 
>>> So, to simulate the behavior of *Gfsh* when connected via JMX RMI, I
>> needed
>>> to poll the Manager.  That way when the Manager was no longer available,
>> it
>>> would display that *Gfsh* was no longer connected AND that the commands
>>> that "require a connection" (e.g. `list region`) were no longer
>>> available... again preserving the existing behavior in HTTP mode.
>>> 
>>> Security (basic auth) had not been implemented in *Gfsh* at that time
>> when
>>> I created the Management REST API (or rather, it is more accurate to
>> say...
>>> REST-like; it's not a true REST-ful interface to be precise, which is one
>>> reason it never was made public for users to consume, though it could
>> have
>>> been, providing we introduce the proper notion of  REST-ful resources
>>> abstractions and change the endpoints (URIs) appropriately; anyway...).
>>> 
>>> -j
>>> 
>>> 
>>> On Thu, Feb 2, 2017 at 11:08 AM, Kevin Duling 
>> wrote:
>>> 
 Yes it does, immediately on the connect.  So the behavior is different.
 
 On Thu, Feb 2, 2017 at 10:48 AM, Anthony Baker 
>>> wrote:
 
> Seems odd to me that the ‘connect’ command is where the credentials
>> are
> supplied but the failures are only realized when invoking a secure
> command.  So I would need to go back and disconnect / reconnect to
>> fix
>>> a
> password typo.
> 
> As a reference point, does ‘connect’ over JMX surface authentication
> errors?
> 
> Anthony
> 
>> On Feb 2, 2017, at 10:37 AM, Kevin Duling 
>>> wrote:
>> 
>> It's been reported in GEODE-2247 that gfsh can connect in a secured
>> environment without a username/password when using the --use-http
>>> flag.
>> When using a jmx connection, this would immediately prompt for
>> user/password.
>> 
>> In the http environment, the connection isn't any less secure.  The
> moment
>> one attempts to execute a command that an "anonymous user" cannot
> execute,
>> they will receive a failure with a message informing them that the
>>> user
> (in
>> this case anonymous) cannot execute that command.  That's all fine
>>> and
>> good, but the UX should probably be to fail instead on the
>> 'connect'
 when
>> in a secure environment.
>> 
>> Opinions?
>> 
>> The issue is that gfsh uses the 'ping' endpoint to determine
> connectivity,
>> which is not secured.  Moreover, it starts a connection poll,
>> hitting
> that
>> endpoint every 500ms to ensure the connection is still alive.  I
>>> can't
>> determine why it's doing this other than to try to wrap an
>> artificial
>> 'state' in to the stateless nature of REST.  The only advantage I
>> see
 is
>> that if I kill my server, gfsh knows right away that it's been
> disconnected
>> from it.
>> 
>> I have not yet determined whether or not the socket stays open
>>> through
> all
>> of this.  I suspect that it does or otherwise I'd see a lot of
>>> FIN_WAIT
>> entries in my netstat results.
>> 
>> One possible solution to this is to 

Re: Wrapping up: repos and versions

[Michael William Dodge]

+1 for "geode-native".

> On 31 Jan, 2017, at 10:19, Jacob Barrett  wrote:
> 
> May I suggest the repo be "geode-native" please.
> 
> On Tue, Jan 31, 2017 at 10:18 AM Anthony Baker  wrote:
> 
>> In a few recent email threads [1] [2] [3] we discussed repos and
>> versioning.  After reviewing the comments, the consensus seems to be this:
>> 
>> 1) Move the native client source into a new repo.
>> 2) Manage the geode-* repos with common versioning and releases in a
>> single community.
>> 
>> I suggest the following actions:
>> 
>> 1) Update the version in geode-examples to 1.2.0-SNAPSHOT to match the
>> geode repo.  Plan on including the geode-examples source in the 1.2.0
>> release.
>> 2) Create a geode-nativelicent repo and move the
>> next-gen-native-client-software-grant branch to the new repo on the develop
>> branch.
>> 3) Update the release wiki [4] to include instructions on creating release
>> artifacts for geode-examples and geode-nativeclient (when ready).
>> 
>> Thoughts?
>> 
>> Anthony
>> 
>> [1]
>> http://mail-archives.apache.org/mod_mbox/geode-dev/201701.mbox/%3cdb3f0e86-1d8f-4acc-8323-fe8c135b3...@pivotal.io%3e
>> [2]
>> http://mail-archives.apache.org/mod_mbox/geode-dev/201701.mbox/%3cCAA1wVxqdvaOZVPFHcDu77P=7lqbmwgk+hhzrpw4jqqhh3ve...@mail.gmail.com%3e
>> [3]
>> http://mail-archives.apache.org/mod_mbox/geode-dev/201701.mbox/%3cca+oe_e3m0vtzxxsko5ejadyjo4x1przvlxudwrwuq2ympiw...@mail.gmail.com%3e
>> [4] https://cwiki.apache.org/confluence/display/GEODE/Release+Steps

Re: Geode Native Library Filenames

[Michael William Dodge]

+1 to dropping "native"

Sarge

> On 23 Jan, 2017, at 08:48, Michael Stolz <mst...@pivotal.io> wrote:
> 
> One other thought...Maybe now is the time to drop the term "native".
> 
> --
> Mike Stolz
> Principal Engineer - Gemfire Product Manager
> Mobile: 631-835-4771
> On Jan 23, 2017 11:29 AM, "Michael William Dodge" <mdo...@pivotal.io> wrote:
> 
>> +1 for apache-geode and Apache.Geode
>> 
>>> On 22 Jan, 2017, at 19:08, Ernest Burghardt <eburgha...@pivotal.io>
>> wrote:
>>> 
>>> +1 for apache-geode (.dll, .so, .dylib) and Apache.Geode.dll
>>> 
>>> On Sun, Jan 22, 2017 at 7:18 AM, Michael Martell <mmart...@pivotal.io>
>>> wrote:
>>> 
>>>> +1 for apache-geode (.dll, .so, .dylib) and Apache.Geode.dll
>>>> 
>>>> On Fri, Jan 20, 2017 at 7:28 PM, Jacob Barrett <jbarr...@pivotal.io>
>>>> wrote:
>>>> 
>>>>> As I ready to drop the pull request to rename all the namespaces in C++
>>>> and
>>>>> .NET sources the next thing we need to decide on is the filename for
>> the
>>>>> library.
>>>>> 
>>>>> C++
>>>>> Current: gfcppcache
>>>>> gfcppcache.dll (Windows), libgfcppcache.so (*nix), libgfcppcache.dylib
>>>>> (macOS)
>>>>> Proposed: apache-geode
>>>>> apache-geode.dll, libapache-geode.so, libapache-geode.dylib
>>>>> Other libraries have embraced other styles for "branded" libraries.
>>>>> Camel case: libQtCode.so
>>>>> Underscore: libsvn_client.so, libboost_atomic-mt.so
>>>>> Dash: libxcb-compose, libcairo-gobject.so
>>>>> I prefere dash because I don't have to hit shift. ;)
>>>>> 
>>>>> .NET
>>>>> Current: GemStone.GemFire.Cache.dll
>>>>> Proposed: Apache.Geode.dll
>>>>> There is a bit more of an established norm. It should be part of the
>>>>> included namespace.
>>>>> Other option is Apache.Geode.Client.dll but I feel like there are items
>>>> in
>>>>> the Client namespace that need to be moved into another namespace as a
>>>>> sibling. So in my mind it makes more sense to have it be
>>>> Apache.Geode.dll.
>>>>> It also makes it consistent with the C++ library.
>>>>> 
>>>>> -Jake
>>>>> 
>>>> 
>> 
>> 

Re: Geode Native Library Filenames

[Michael William Dodge]

+1 for apache-geode and Apache.Geode

> On 22 Jan, 2017, at 19:08, Ernest Burghardt  wrote:
> 
> +1 for apache-geode (.dll, .so, .dylib) and Apache.Geode.dll
> 
> On Sun, Jan 22, 2017 at 7:18 AM, Michael Martell 
> wrote:
> 
>> +1 for apache-geode (.dll, .so, .dylib) and Apache.Geode.dll
>> 
>> On Fri, Jan 20, 2017 at 7:28 PM, Jacob Barrett 
>> wrote:
>> 
>>> As I ready to drop the pull request to rename all the namespaces in C++
>> and
>>> .NET sources the next thing we need to decide on is the filename for the
>>> library.
>>> 
>>> C++
>>> Current: gfcppcache
>>> gfcppcache.dll (Windows), libgfcppcache.so (*nix), libgfcppcache.dylib
>>> (macOS)
>>> Proposed: apache-geode
>>> apache-geode.dll, libapache-geode.so, libapache-geode.dylib
>>> Other libraries have embraced other styles for "branded" libraries.
>>> Camel case: libQtCode.so
>>> Underscore: libsvn_client.so, libboost_atomic-mt.so
>>> Dash: libxcb-compose, libcairo-gobject.so
>>> I prefere dash because I don't have to hit shift. ;)
>>> 
>>> .NET
>>> Current: GemStone.GemFire.Cache.dll
>>> Proposed: Apache.Geode.dll
>>> There is a bit more of an established norm. It should be part of the
>>> included namespace.
>>> Other option is Apache.Geode.Client.dll but I feel like there are items
>> in
>>> the Client namespace that need to be moved into another namespace as a
>>> sibling. So in my mind it makes more sense to have it be
>> Apache.Geode.dll.
>>> It also makes it consistent with the C++ library.
>>> 
>>> -Jake
>>> 
>> 

Re: Native client build fails linking shared library?

[Michael William Dodge]

All I can guess is that /usr/lib/libz.* is missing for some reason. Unless Jake 
has a better suggestion, I'll have to take a look at it tomorrow, in person if 
possible.

Sarge

> On 17 Jan, 2017, at 17:04, Dan Smith <dsm...@pivotal.io> wrote:
> 
> Here's the results of the VERBOSE make. I do see a -lz in the linker line. 
> 
> [ 28%] Linking CXX executable gfcppcache_unittests
> cd /home/dan/MyStuff/Code/gemfire/native/cppcache/test && /usr/bin/cmake -E 
> cmake_link_script CMakeFiles/gfcppcache_unittests.dir/link.txt --verbose=1
> /usr/bin/c++   -g   -m64 
> CMakeFiles/gfcppcache_unittests.dir/ByteArrayFixture.cpp.o 
> CMakeFiles/gfcppcache_unittests.dir/ExpirationActionTest.cpp.o 
> CMakeFiles/gfcppcache_unittests.dir/PdxLocalReaderTest.cpp.o 
> CMakeFiles/gfcppcache_unittests.dir/DataInputTest.cpp.o 
> CMakeFiles/gfcppcache_unittests.dir/DataOutputTest.cpp.o 
> CMakeFiles/gfcppcache_unittests.dir/ByteArrayTest.cpp.o 
> CMakeFiles/gfcppcache_unittests.dir/NanoTimerTest.cpp.o 
> CMakeFiles/gfcppcache_unittests.dir/AutoDeleteTest.cpp.o 
> CMakeFiles/gfcppcache_unittests.dir/SharedPtrTest.cpp.o 
> CMakeFiles/gfcppcache_unittests.dir/DiskPolicyTypeTest.cpp.o 
> CMakeFiles/gfcppcache_unittests.dir/CacheXmlParserTest.cpp.o 
> CMakeFiles/gfcppcache_unittests.dir/SharedBaseTest.cpp.o 
> CMakeFiles/gfcppcache_unittests.dir/gfcppBannerTest.cpp.o 
> CMakeFiles/gfcppcache_unittests.dir/CacheableKeysTest.cpp.o 
> CMakeFiles/gfcppcache_unittests.dir/TcrMessage_unittest.cpp.o 
> CMakeFiles/gfcppcache_unittests.dir/InterestResultPolicyTest.cpp.o 
> CMakeFiles/gfcppcache_unittests.dir/ByteArray.cpp.o  -o gfcppcache_unittests 
> -rdynamic ../src/libgfcppcache-static.a -lz 
> ../../dependencies/ACE/ACE-extern-prefix/lib/libACE.a -lpthread -ldl -lrt 
> ../../dependencies/libxml2/libxml2-extern-prefix/lib/libxml2.a 
> ../../dependencies/gtest/gtest-extern-prefix/src/gtest-extern-build//libgtest.a
>  
> ../../dependencies/gtest/gtest-extern-prefix/src/gtest-extern-build//libgtest_main.a
>  
> ../../dependencies/libxml2/libxml2-extern-prefix/lib/libxml2.a(xmlIO.o): In 
> function `xmlGzfileOpen_real':
> xmlIO.c:(.text+0xf9d): undefined reference to `gzdopen'
> 
> -Dan
> 
> On Tue, Jan 17, 2017 at 4:43 PM, Jacob Barrett <jbarr...@pivotal.io 
> <mailto:jbarr...@pivotal.io>> wrote:
> Dan,
> 
> Add VERBOSE=1 to your make command.
> 
> make -j8 VERBOSE=1
> 
> To see details of the compile commands. Some of the third party
> dependencies won't show them but give it a try.
> 
> -Jake
> 
> 
> On Tue, Jan 17, 2017 at 4:34 PM Michael William Dodge <mdo...@pivotal.io 
> <mailto:mdo...@pivotal.io>>
> wrote:
> 
> > Do you see -lz in the link line?
> >
> > Sarge
> >
> > > On 17 Jan, 2017, at 16:24, Dan Smith <dsm...@pivotal.io 
> > > <mailto:dsm...@pivotal.io>> wrote:
> > >
> > > I tried building the next-gen-native-client-software-grant branch on an
> > > ubuntu 16.04 box. It gets about halfway through and then fails with these
> > > errors. Any ideas?
> > >
> > > [ 49%] Building CXX object
> > > cppcache/src/CMakeFiles/gfcppcache.dir/PdxLocalWriter.cpp.o
> > > [ 49%] Building CXX object
> > > cppcache/src/CMakeFiles/gfcppcache.dir/CacheStatistics.cpp.o
> > > [ 49%] Linking CXX shared library libgfcppcache.so
> > > ../../dependencies/libxml2/libxml2-extern-prefix/lib/libxml2.a(xmlIO.o):
> > In
> > > function `xmlGzfileOpen_real':
> > > xmlIO.c:(.text+0xf9d): undefined reference to `gzdopen'
> > > xmlIO.c:(.text+0x1055): undefined reference to `gzopen64'
> > > ../../dependencies/libxml2/libxml2-extern-prefix/lib/libxml2.a(xmlIO.o):
> > In
> > > function `xmlGzfileOpenW':
> > > xmlIO.c:(.text+0x1158): undefined reference to `gzdopen'
> > > xmlIO.c:(.text+0x11f9): undefined reference to `gzopen64'
> > > ../../dependencies/libxml2/libxml2-extern-prefix/lib/libxml2.a(xmlIO.o):
> > In
> > > function `xmlGzfileRead':
> > > xmlIO.c:(.text+0x1240): undefined reference to `gzread'
> > > ../../dependencies/libxml2/libxml2-extern-prefix/lib/libxml2.a(xmlIO.o):
> > In
> > > function `xmlGzfileWrite':
> > >
> > > ...
> > >
> > > collect2: error: ld returned 1 exit status
> > > cppcache/src/CMakeFiles/gfcppcache.dir/build.make:6212: recipe for target
> > > 'cppcache/src/libgfcppcache.so' failed
> > > make[2]: *** [cppcache/src/libgfcppcache.so] Error 1
> > > CMakeFiles/Makefile2:936: recipe for target
> > > 'cppcache/src/CMakeFiles/gfcppcache.dir/all' failed
> > > make[1]: *** [cppcache/src/CMakeFiles/gfcppcache.dir/all] Error 2
> > > Makefile:149: recipe for target 'all' failed
> > > make: *** [all] Error 2
> > >
> > > -Dan
> >
> >
> 
> 

Re: Native client build fails linking shared library?

[Michael William Dodge]

Do you see -lz in the link line?

Sarge

> On 17 Jan, 2017, at 16:24, Dan Smith  wrote:
> 
> I tried building the next-gen-native-client-software-grant branch on an
> ubuntu 16.04 box. It gets about halfway through and then fails with these
> errors. Any ideas?
> 
> [ 49%] Building CXX object
> cppcache/src/CMakeFiles/gfcppcache.dir/PdxLocalWriter.cpp.o
> [ 49%] Building CXX object
> cppcache/src/CMakeFiles/gfcppcache.dir/CacheStatistics.cpp.o
> [ 49%] Linking CXX shared library libgfcppcache.so
> ../../dependencies/libxml2/libxml2-extern-prefix/lib/libxml2.a(xmlIO.o): In
> function `xmlGzfileOpen_real':
> xmlIO.c:(.text+0xf9d): undefined reference to `gzdopen'
> xmlIO.c:(.text+0x1055): undefined reference to `gzopen64'
> ../../dependencies/libxml2/libxml2-extern-prefix/lib/libxml2.a(xmlIO.o): In
> function `xmlGzfileOpenW':
> xmlIO.c:(.text+0x1158): undefined reference to `gzdopen'
> xmlIO.c:(.text+0x11f9): undefined reference to `gzopen64'
> ../../dependencies/libxml2/libxml2-extern-prefix/lib/libxml2.a(xmlIO.o): In
> function `xmlGzfileRead':
> xmlIO.c:(.text+0x1240): undefined reference to `gzread'
> ../../dependencies/libxml2/libxml2-extern-prefix/lib/libxml2.a(xmlIO.o): In
> function `xmlGzfileWrite':
> 
> ...
> 
> collect2: error: ld returned 1 exit status
> cppcache/src/CMakeFiles/gfcppcache.dir/build.make:6212: recipe for target
> 'cppcache/src/libgfcppcache.so' failed
> make[2]: *** [cppcache/src/libgfcppcache.so] Error 1
> CMakeFiles/Makefile2:936: recipe for target
> 'cppcache/src/CMakeFiles/gfcppcache.dir/all' failed
> make[1]: *** [cppcache/src/CMakeFiles/gfcppcache.dir/all] Error 2
> Makefile:149: recipe for target 'all' failed
> make: *** [all] Error 2
> 
> -Dan

Re: [GitHub] geode issue #341: GEODE-2306: Update native client BUILDING.md to reflect ch...

[Michael William Dodge]

It's current in src/ on next-gen-native-client-software-grant. What do you mean 
by "down to the root directory"?

Sarge

> On 17 Jan, 2017, at 13:43, pivotal-jbarrett  wrote:
> 
> Github user pivotal-jbarrett commented on the issue:
> 
>https://github.com/apache/geode/pull/341
> 
>Let's also move the BUILDING.md down to the root directory so it is easier 
> to find.
> 
> 
> ---
> If your project is set up for it, you can reply to this email and have your
> reply appear on GitHub as well. If your project does not have this feature
> enabled and wishes so, or if the feature is enabled but not working, please
> contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
> with INFRA.
> ---

Re: New Repo for Native Client

[Michael William Dodge]

To me this seems related to the question of the degree to which the Geode 
community should be homogeneous. Who are we trying to attract with the non-Java 
clients: the existing, Java-centric community or new community members from 
other platforms? If the former, gradle makes it easy for them to adopt the 
native client. If the latter, gradle is a barrier to entry.

Sarge

> On 17 Jan, 2017, at 09:34, Roman Shaposhnik  wrote:
> 
> On Mon, Jan 16, 2017 at 8:47 PM, Jacob Barrett  wrote:
>> Roman,
>> 
>> I understand what you are saying. I think that since the build process
>> between the Java Geode bits and the Native Geode bits will completely
>> different it might help to have the separate. Until someone comes up with a
>> good cross platform and cross language build tool that is commonly used in
>> the development environments for each language these builds will remain
>> different. Gradle sucks for building C++ and .NET sources and CMake sucks
>> for building Java sources. Gradle is not popular in the native project
>> world nor is CMake popular in the Java world.
> 
> Personally I feel that standardizing on Gradle in 2017 would make sense
> for C++ as well. Now, I hear what you're saying -- the popularity is an
> issue, but that's only a problem for complex builds (at the level of the 
> client)
> which our client is not. At least not really.
> 
> That said -- this comes back to my original point of thinking about 
> contributor
> community -- I could totally see folks who would otherwise have contributed
> to the unification effort not liking the switch to Gradle. So yeah --
> I hear you,
> but personally I'd err on the side of unification since there are
> greater benefits
> to be reaped.
> 
>> So making one build system to
>> cover them all would just hurt everyone. Since the experience will be
>> unique for each I feel that it justifies a separate repo but I can totally
>> see the other side of just keeping it all together.
> 
> FWIW: I think it will only hurt folks with preconceived notion of what a C++
> build should look like. In my life, I've seen way too many different
> builds systems
> come and go to worry about that ;-)
> 
> Thanks,
> Roman

Re: Native Namespace

[Michael William Dodge]

In my experience, it seems easier to have too many namespaces than it is to 
have too few. My instinct is to start with geode:: and Geode. and further 
subdivide when it becomes unwieldy and collisions arise.

Sarge

> On 16 Jan, 2017, at 21:10, Jacob Barrett  wrote:
> 
> An upcoming change we need to decide on is the C++ and .NET namespace for
> the C++ and .NET clients.
> 
> *C++*
> Current:
> *::gemfire*
> Thoughts:
> *::apache::geode::client*
> *::geode::client*
> I shy away from prefixing with *apache* since it requires extra blocks in
> C++:
> (formatted to Google C++ style guide)
> namespace apache {
> namespace geode {
> namespace client {
>  class Cache {...};
> } // namespace client
> } // namespace geode
> } // namespace apache
> vs.
> namespace geode {
> namespace client {
>  class Cache {...};
> } // namespace client
> } // namespace geode
> vs.
> namespace geode {
>  class Cache {...};
> } // namespace geode
> 
> I shy away form just *geode* because it feels too short but I am not that
> opposed to it. The question is would we likely have anything else in C++
> under the *geode* namespace that is not the client?
> 
> *.NET*
> Current:
> *GemStone.GemFire.Cache.Generic*
> Thoughts:
> *Apache.Geode.Client*
> *Geode.Client*
> I am not a fan of *Apache.Geode.Cache.Generic* because *Generic* is legacy
> form the replacement of the non-generic versions of the API and *Cache*
> feels redundant. Feels odd to be working with *Apache.Geode.Cache*.*Cache*
> objects.
> The same issues with the C++ namespaces apply here since our .NET client is
> currently C++/CLI which suffers the same namespace block issues.
> 
> It would feel good to have the namespaces be somewhat consistent between
> clients but it is not normal for namespaces in C++ or .NET to use the
> reverse domain style that Java uses. So *org::apache::geode::client /
> Org.Apache.Geode.Client* or some variant is off the table I think.
> 
> Anyone have other thoughts?
> 
> Thanks,
> Jake

Re: Native Client Directory Structure

[Michael William Dodge]

I'm not an expert at .NET development so please bear with me. In that world, is 
CLI synonymous with C#? It seems that people who want a non-Java client for 
Geode would be thinking of it in terms of programming language, which is why I 
suggested csharp. It seems like people writing Geode clients currently have 
three options for programming language: Java, C++, and C#.

Sarge

> On 16 Jan, 2017, at 08:45, Jacob Barrett <jbarr...@pivotal.io> wrote:
> 
> On Mon, Jan 16, 2017 at 8:35 AM Michael William Dodge <mdo...@pivotal.io>
> wrote:
> 
>> Given that the current source structure uses cpp and cli prefixes for the
>> C++ and .NET clients, respectively, what about using cpp and cli for the
>> directories, allowing any new .NET development to go into a separate
>> directory (perhaps cs or csharp) without any additional moves?
>> 
> 
> Sarge,
> 
> Given that .NET/CLI is language independent, calling a forthcoming pure CLI
> client after it's potential language would not be appropriate.
> 
> If the concerns is more moves than leaving everything under a
> "geode-native" directory might make most sense. Later we could add
> "geode-cli" when the new pure CLI client is developed. Then slowly phase
> out the CLI inside the "geode-native".
> 
> -Jake

Re: Native Client Directory Structure

[Michael William Dodge]

Given that the current source structure uses cpp and cli prefixes for the C++ 
and .NET clients, respectively, what about using cpp and cli for the 
directories, allowing any new .NET development to go into a separate directory 
(perhaps cs or csharp) without any additional moves?

Sarge

> On 16 Jan, 2017, at 08:26, Jacob Barrett  wrote:
> 
> One of the first things necessary to get NC merged into the the develop
> branch is understanding where it will go under the current geode project
> structure.
> 
> The quick and obvious solution is adding a 'geode-nativeclient` subproject
> and relocating all the NC sources into that directory.
> 
> Given that NC consists of two semi-distinct clients, C++ and .NET, it may
> also make sense to organize more of a hierarchy. Consider:
> geode-client/
>geode++
>geode.net
> (or some other more creative names)
> Keep in mind that today the .NET client is very tightly coupled with the
> C++ client, so you can't build .NET without first building C++.
> 
> My suggestion would be to do the quick and easy now and as we continue to
> refine and refactor and hopefully write the .NET in pure CLI we make that
> move them. Perhaps by that time there will be a pure Java client to include
> in that structure.
> 
> Thoughts?
> 
> 
> -Jake

Permission to Create JIRA Issues

[Michael William Dodge]

I'd like to create a JIRA issue regarding a problem I've discovered. May I 
please have permission added for username PivotalSarge? Thanks.

Sarge