Fwd: [DISCUSS] Storage-class memory ecosystem program

[Roman Shaposhnik]

FYI


-- Forwarded message --
From: Gang(Gary) Wang 
Date: Tue, Oct 24, 2017 at 10:10 AM
Subject: Re: [DISCUSS] Storage-class memory ecosystem program
To: gene...@incubator.apache.org
Cc: d...@mnemonic.incubator.apache.org


Add *Hive/LLAP and RocketMQ*

   - *Ignite* represented by Denis Magda
   - *Arrow *represented by Wes McKinney
   - *Hbase *represented by Anoop John
   - *Crail* represented by *Patrick Stuedi*
   - *Hive/LLAP *represented by *Gopal Vijayaraghavan*
   - *RocketMQ  *represented by *Von Gosling*
   - *ORC *represented by* Owen O'Malley*
   - *Mnemonic *represented by* Gary*

With above projects, we could cover Storage-class memory oriented *Distributed
Database, KV Store, Columnar Structured Dataset, Distributed Data Store,
Columnar Storage, **Live Long And Process, Distributed Messaging and
Streaming, **Durable Object Model, Durable Computing Model* for new
generation high-performance applications, e.g. data querying, processing,
and analytics.

On Mon, Oct 23, 2017 at 9:32 PM, Von Gosling  wrote:

> I'm happy to represent Apache RocketMQ on this  storage collaboration
> effort. This is a very key in low latency messaging.
>
> Best Regards,
> Von Gosling
>
>
>
> > 在 2017年10月20日，02:55，Gang(Gary) Wang  写道：
> >
> > Hi all,
> >
> > We can expect more and more projects will take the huge potential
> > advantages of storage-class memory for data processing and analytics
> > because silicon companies are able to produce high capacity non-volatile
> > memory on a large scale, this hardware technology will fundamentally
> change
> > the way to construct high performance applications similar to what
> happened
> > when replacing tape with disk technology since the 1980s. so if
> possible, I
> > advocate establishing an Apache working group to enhance the
> collaboration
> > and synergies mentioned by Patrick Stuedi for storage-class memory
> > technology-oriented projects.
> >
> > Best.
> > Gary.
>
>

Gitbox enables the full GitHub workflow

[Roman Shaposhnik]

Hi!

it has just come to my attention that Gitbox at ASF
has been enabling full GitHub workflow (with being
able to click Merge this PR button, etc.) for quite
some time.

This basically allows a project to have GH as a R/W
repo as opposed to R/O mirror of what we all currnently
have: https://gitbox.apache.org/repos/asf

Personally I'm not sure I like GH workflow all that much,
but if there's interest -- you can opt-in into Gitbox. Once
you do -- your source of truth moves to GH. You can't
have it both ways with git-wip-us.apache.org and Gitbox.

Thanks,
Roman.

Re: Jenkins karma (was Re: Blacklist asf904 for Geode nightly job)

[Roman Shaposhnik]

On Mon, Apr 10, 2017 at 9:46 AM, Anthony Baker  wrote:
> Anyone?

This should be done by Mark (current Chair).

Mark, can you add Anthony to the Jenkins LDAP group?

Thanks,
Roman.

Re: Request for edit permissions

[Roman Shaposhnik]

Done!

On Thu, Mar 30, 2017 at 9:08 AM, Brian Baynes <bbay...@pivotal.io> wrote:
> Hi, Roman.
>
> Sure, understood -- that info was by way of introduction, not a claim to
> anything :)
> Thanks for making it clear.
>
> Both IDs are bbaynes.
>
> Thanks for helping me get set up!
>
> Bran
>
> On Thu, Mar 30, 2017 at 8:50 AM, Roman Shaposhnik <ro...@shaposhnik.org>
> wrote:
>
>> Hi Brian!
>>
>> On Thu, Mar 30, 2017 at 8:32 AM, Brian Baynes <bbay...@pivotal.io> wrote:
>> > Hello,
>> >
>> > I'm a new PM at Pivotal working with the Communications team.
>>
>> Quick note on the Apache way of doing things. While we appreciate
>> knowing your relationship with Pivotal please realize that neither your
>> title nor your affiliation with this particular employer have any bearings
>> on what kind of access you get with the project. We're all contributing
>> to Apache Geode as individual contributors and a lot of times we don't
>> even know the employment affiliation of others.
>>
>> This is a good thing. This is how Apache Way works.
>>
>> > I'd like to request edit access for wikis and Jira tickets for Geode.
>> For
>> > example, I'd like to edit the description for GEODE-2580, a project our
>> > team is working on.
>>
>> Now, for both of these -- we welcome all levels of contributions - giving
>> you
>> an access is not a problem.
>>
>> What's your ASF wiki and JIRA ID?
>>
>> Thanks,
>> Roman.
>>

Re: Requesting edit permissions for Geode tickets

[Roman Shaposhnik]

Kirk, I've bumped your karma to the admin.

Thanks,
Roman.

On Thu, Mar 30, 2017 at 10:20 AM, Kirk Lund  wrote:
> I gave you permissions for Jira. Someone else still needs to give you karma
> on the Wiki.
>
> -Kirk
>
> On Thu, Mar 30, 2017 at 9:22 AM, Fred Krone  wrote:
>
>> Hi,
>>
>> Could I get editing permissions for Geode tickets?  I can only edit mine
>> and I'd like to reword some descriptions etc on other tickets.  I'd also
>> like to edit some wikis if needed.
>>
>> Both ASF and JIRA IDs are fkrone.
>>
>> Thanks,
>>
>> -Fred
>>

Re: Requesting edit permissions for Geode tickets

[Roman Shaposhnik]

On Thu, Mar 30, 2017 at 9:22 AM, Fred Krone  wrote:
> Hi,
>
> Could I get editing permissions for Geode tickets?  I can only edit mine
> and I'd like to reword some descriptions etc on other tickets.  I'd also
> like to edit some wikis if needed.
>
> Both ASF and JIRA IDs are fkrone.

Here's what I'm getting from cwiki:
   User fkrone could not be found

Thanks,
Roman.

Re: Request for edit permissions

[Roman Shaposhnik]

Hi Brian!

On Thu, Mar 30, 2017 at 8:32 AM, Brian Baynes  wrote:
> Hello,
>
> I'm a new PM at Pivotal working with the Communications team.

Quick note on the Apache way of doing things. While we appreciate
knowing your relationship with Pivotal please realize that neither your
title nor your affiliation with this particular employer have any bearings
on what kind of access you get with the project. We're all contributing
to Apache Geode as individual contributors and a lot of times we don't
even know the employment affiliation of others.

This is a good thing. This is how Apache Way works.

> I'd like to request edit access for wikis and Jira tickets for Geode.  For
> example, I'd like to edit the description for GEODE-2580, a project our
> team is working on.

Now, for both of these -- we welcome all levels of contributions - giving you
an access is not a problem.

What's your ASF wiki and JIRA ID?

Thanks,
Roman.

Re: [VOTE] Apache Geode release - v1.1.1 RC1

[Roman Shaposhnik]

On Mon, Mar 27, 2017 at 12:13 PM, Anthony Baker  wrote:
> The release/1.1.1 branch was made from master which shows several 
> documentation-related changes after the 1.1.0 release:

I understand that, but I think in this particular case a patch release
should be as close to 1.1.0 as possible.
If there's a desire to get these changes out ASAP (although I really
don't quite see why) -- you can always roll 1.1.2.

Thanks,
Roman.

Re: JIRA versions and multiple repos

[Roman Shaposhnik]

Btw, I've just confirmed that NiFi considers those subprojects
(AKA separate communities).

In fact ASF board asked them for that very clarification recently

Thanks,
Roman.

On Thu, Jan 19, 2017 at 2:27 PM, Roman Shaposhnik <ro...@shaposhnik.org> wrote:
> On Thu, Jan 19, 2017 at 2:23 PM, Anthony Baker <aba...@pivotal.io> wrote:
>>
>>> On Jan 19, 2017, at 1:01 PM, Roman Shaposhnik <ro...@shaposhnik.org> wrote:
>>>
>>> On Thu, Jan 19, 2017 at 12:54 PM, Anthony Baker <aba...@pivotal.io 
>>> <mailto:aba...@pivotal.io>> wrote:
>>>>
>>>>> On Jan 19, 2017, at 11:53 AM, Roman Shaposhnik <ro...@shaposhnik.org> 
>>>>> wrote:
>>>>>
>>>>> On Thu, Jan 19, 2017 at 11:21 AM, Dan Smith <dsm...@pivotal.io> wrote:
>>>>>> I wonder if we're trying to overcomplicate things there. I don't see why
>>>>>> the geode-examples wouldn't use the same release schedule and version
>>>>>> number as geode.
>>>>>>
>>>>>> The C++ and .NET clients are also somewhat tied to the version of geode
>>>>>> that they support. As long as we can stick to a regular release cadence, 
>>>>>> It
>>>>>> seems like those clients couldn't also follow the same release schedule 
>>>>>> and
>>>>>> version numbers.
>>>>>
>>>>> Huge +1 to the above!
>>>>>
>>>>> Thanks,
>>>>> Roman.
>>>>
>>>>
>>>> Here’s a few examples of ASF projects with multiple repos for reference:
>>>>
>>>> - ActiveMQ
>>>>https://github.com/apache?utf8=✓=activemq==
>>>>https://issues.apache.org/jira/secure/BrowseProjects.jspa#11160
>>>> - Nifi
>>>>https://github.com/apache?utf8=✓=nifi==
>>>>https://issues.apache.org/jira/secure/BrowseProjects.jspa#13460
>>>>
>>>> I agree that semi-coordinated releases from a single project community make
>>>> sense—these are not independent things.  Using lock-step versioning means
>>>> we release everything together, even for patch releases right?  And I’m
>>>> assuming we would be doing separate release VOTE threads per repo.
>>>
>>> An interesting thing to note is that despite multiple repos they still 
>>> release
>>> a single source artifact:
>>>   https://www.apache.org/dist/activemq/5.13.5/ 
>>> <https://www.apache.org/dist/activemq/5.13.5/>
>>>   https://www.apache.org/dist/nifi/1.1.1/ 
>>> <https://www.apache.org/dist/nifi/1.1.1/>
>>>
>>> Thanks,
>>> Roman.
>>
>> FWIW, it looks to me like these projects are doing releases from each of 
>> their repos:
>>
>> https://www.apache.org/dist/activemq/activemq-apollo/1.7.1/
>> https://www.apache.org/dist/activemq/activemq-artemis/1.5.1/
>> https://www.apache.org/dist/activemq/activemq-cpp/3.9.3/
>
> I'm actually not sure what the status of these is sine I can't seem to find
> those on the official download page:
> http://activemq.apache.org/download.html
>
>> https://www.apache.org/dist/nifi/minifi/0.1.0/
>> https://www.apache.org/dist/nifi/nifi-minifi-cpp/0.1.0/
>> https://www.apache.org/dist/nifi/nifi-nar-maven-plugin-1.1.0/
>
> Ditt for NiFi. In fact, as far as minifi is concerned -- I'm pretty
> sure it is still
> considered to be 'do not try this at home'.
>
>> Which is not to say that is how Geode should operate, but I’m just looking 
>> for precedent and prior art :-)
>
> But that's my point -- I don't think it constitutes prior art. In fact
> the only prior art I'm
> aware of would be Apache Commons (which actually does explicitly split
> the community).
>
> Thanks,
> Roman.

Re: JIRA versions and multiple repos

[Roman Shaposhnik]

On Thu, Jan 19, 2017 at 2:23 PM, Anthony Baker <aba...@pivotal.io> wrote:
>
>> On Jan 19, 2017, at 1:01 PM, Roman Shaposhnik <ro...@shaposhnik.org> wrote:
>>
>> On Thu, Jan 19, 2017 at 12:54 PM, Anthony Baker <aba...@pivotal.io 
>> <mailto:aba...@pivotal.io>> wrote:
>>>
>>>> On Jan 19, 2017, at 11:53 AM, Roman Shaposhnik <ro...@shaposhnik.org> 
>>>> wrote:
>>>>
>>>> On Thu, Jan 19, 2017 at 11:21 AM, Dan Smith <dsm...@pivotal.io> wrote:
>>>>> I wonder if we're trying to overcomplicate things there. I don't see why
>>>>> the geode-examples wouldn't use the same release schedule and version
>>>>> number as geode.
>>>>>
>>>>> The C++ and .NET clients are also somewhat tied to the version of geode
>>>>> that they support. As long as we can stick to a regular release cadence, 
>>>>> It
>>>>> seems like those clients couldn't also follow the same release schedule 
>>>>> and
>>>>> version numbers.
>>>>
>>>> Huge +1 to the above!
>>>>
>>>> Thanks,
>>>> Roman.
>>>
>>>
>>> Here’s a few examples of ASF projects with multiple repos for reference:
>>>
>>> - ActiveMQ
>>>https://github.com/apache?utf8=✓=activemq==
>>>https://issues.apache.org/jira/secure/BrowseProjects.jspa#11160
>>> - Nifi
>>>https://github.com/apache?utf8=✓=nifi==
>>>https://issues.apache.org/jira/secure/BrowseProjects.jspa#13460
>>>
>>> I agree that semi-coordinated releases from a single project community make
>>> sense—these are not independent things.  Using lock-step versioning means
>>> we release everything together, even for patch releases right?  And I’m
>>> assuming we would be doing separate release VOTE threads per repo.
>>
>> An interesting thing to note is that despite multiple repos they still 
>> release
>> a single source artifact:
>>   https://www.apache.org/dist/activemq/5.13.5/ 
>> <https://www.apache.org/dist/activemq/5.13.5/>
>>   https://www.apache.org/dist/nifi/1.1.1/ 
>> <https://www.apache.org/dist/nifi/1.1.1/>
>>
>> Thanks,
>> Roman.
>
> FWIW, it looks to me like these projects are doing releases from each of 
> their repos:
>
> https://www.apache.org/dist/activemq/activemq-apollo/1.7.1/
> https://www.apache.org/dist/activemq/activemq-artemis/1.5.1/
> https://www.apache.org/dist/activemq/activemq-cpp/3.9.3/

I'm actually not sure what the status of these is sine I can't seem to find
those on the official download page:
http://activemq.apache.org/download.html

> https://www.apache.org/dist/nifi/minifi/0.1.0/
> https://www.apache.org/dist/nifi/nifi-minifi-cpp/0.1.0/
> https://www.apache.org/dist/nifi/nifi-nar-maven-plugin-1.1.0/

Ditt for NiFi. In fact, as far as minifi is concerned -- I'm pretty
sure it is still
considered to be 'do not try this at home'.

> Which is not to say that is how Geode should operate, but I’m just looking 
> for precedent and prior art :-)

But that's my point -- I don't think it constitutes prior art. In fact
the only prior art I'm
aware of would be Apache Commons (which actually does explicitly split
the community).

Thanks,
Roman.

Re: JIRA versions and multiple repos

[Roman Shaposhnik]

On Thu, Jan 19, 2017 at 12:54 PM, Anthony Baker <aba...@pivotal.io> wrote:
>
>> On Jan 19, 2017, at 11:53 AM, Roman Shaposhnik <ro...@shaposhnik.org> wrote:
>>
>> On Thu, Jan 19, 2017 at 11:21 AM, Dan Smith <dsm...@pivotal.io> wrote:
>>> I wonder if we're trying to overcomplicate things there. I don't see why
>>> the geode-examples wouldn't use the same release schedule and version
>>> number as geode.
>>>
>>> The C++ and .NET clients are also somewhat tied to the version of geode
>>> that they support. As long as we can stick to a regular release cadence, It
>>> seems like those clients couldn't also follow the same release schedule and
>>> version numbers.
>>
>> Huge +1 to the above!
>>
>> Thanks,
>> Roman.
>
>
> Here’s a few examples of ASF projects with multiple repos for reference:
>
> - ActiveMQ
> https://github.com/apache?utf8=✓=activemq==
> https://issues.apache.org/jira/secure/BrowseProjects.jspa#11160
> - Nifi
> https://github.com/apache?utf8=✓=nifi==
> https://issues.apache.org/jira/secure/BrowseProjects.jspa#13460
>
> I agree that semi-coordinated releases from a single project community make
> sense—these are not independent things.  Using lock-step versioning means
> we release everything together, even for patch releases right?  And I’m
> assuming we would be doing separate release VOTE threads per repo.

An interesting thing to note is that despite multiple repos they still release
a single source artifact:
   https://www.apache.org/dist/activemq/5.13.5/
   https://www.apache.org/dist/nifi/1.1.1/

Thanks,
Roman.

Re: JIRA versions and multiple repos

[Roman Shaposhnik]

On Thu, Jan 19, 2017 at 11:21 AM, Dan Smith  wrote:
> I wonder if we're trying to overcomplicate things there. I don't see why
> the geode-examples wouldn't use the same release schedule and version
> number as geode.
>
> The C++ and .NET clients are also somewhat tied to the version of geode
> that they support. As long as we can stick to a regular release cadence, It
> seems like those clients couldn't also follow the same release schedule and
> version numbers.

Huge +1 to the above!

Thanks,
Roman.

Re: JIRA versions and multiple repos

[Roman Shaposhnik]

What you're talking about makes sense -- but it is mechanics. What
I'm talking about is a community composition. E.g. if we truly believe
that communities of developers around, lets say core Java Geode
and Native Client are non-overlapping (or there's very little overlap)
we need to recognize this fact by running them as separate ASF
projects. The mechanics will flow from the answering the community
composition question.

Thanks,
Roman.

On Thu, Jan 19, 2017 at 9:20 AM, John Blum <jb...@pivotal.io> wrote:
> Not sure if ASF has, or uses the same concept, but this could easily be
> handled with a GitHub "organization" encompassing 1 or more repos (for
> example... https://github.com/reactor).
>
> Of course, you could organize the source, and in particular, the Geode
> "modules" anyway you like, for example, as 1 repo.  It's just more
> common/natural to use separate repos for independently releasable
> artifacts.  In that way, the community does not seem as fragmented, rather
> organized into teams around particular concerns (aka modules/features).
> Native Client is perhaps the best example of this since it encompasses
> different tools and different languages but with a common "purpose".
>
> More food for thought.
>
> -j
>
>
> On Thu, Jan 19, 2017 at 9:12 AM, Roman Shaposhnik <ro...@shaposhnik.org>
> wrote:
>
>> On Thu, Jan 19, 2017 at 7:55 AM, Anthony Baker <aba...@pivotal.io> wrote:
>> > Currently our JIRA versions look like this:
>> >
>> > 1.0.0-incubating.M1
>> > 1.0.0-incubating.M2
>> > 1.0.0-incubating.M3
>> > 1.1.0
>> >
>> > That works great for the geode repo.  However, what about the
>> geode-examples repo?  I would like to set a ‘Fix version’ that matches the
>> version in [1].  Since the repos can release independently of each other, I
>> think we need a way to completely disambiguate versions like
>> ‘geode-examples-0.1’.  We could also ask for a JIRA project for each repo.
>> Thoughts?
>> >
>> > More stuff:
>> >
>> > - GEODE-2318 didn’t get updated with commit logs from geode-examples.
>> Anyone know how to fix this?
>> > - Travis-CI is now running on geode-examples.  If you notice problems
>> with PR’s or email notifications let me know.
>>
>> This is the slippery slope I was alluding to. If the repos, releases,
>> etc. are asynchronous
>> in the eyes of ASF it strongly suggests that communities are
>> asynchronous as well. Which
>> means you're really two separate ASF projects. Which may, very well,
>> be the case I just
>> wanted to point it out.
>>
>> Thanks,
>> Roman.
>>
>
>
>
> --
> -John
> john.blum10101 (skype)

Re: New Repo for Native Client

[Roman Shaposhnik]

On Mon, Jan 16, 2017 at 8:47 PM, Jacob Barrett  wrote:
> Roman,
>
> I understand what you are saying. I think that since the build process
> between the Java Geode bits and the Native Geode bits will completely
> different it might help to have the separate. Until someone comes up with a
> good cross platform and cross language build tool that is commonly used in
> the development environments for each language these builds will remain
> different. Gradle sucks for building C++ and .NET sources and CMake sucks
> for building Java sources. Gradle is not popular in the native project
> world nor is CMake popular in the Java world.

Personally I feel that standardizing on Gradle in 2017 would make sense
for C++ as well. Now, I hear what you're saying -- the popularity is an
issue, but that's only a problem for complex builds (at the level of the client)
which our client is not. At least not really.

That said -- this comes back to my original point of thinking about contributor
community -- I could totally see folks who would otherwise have contributed
to the unification effort not liking the switch to Gradle. So yeah --
I hear you,
but personally I'd err on the side of unification since there are
greater benefits
to be reaped.

> So making one build system to
> cover them all would just hurt everyone. Since the experience will be
> unique for each I feel that it justifies a separate repo but I can totally
> see the other side of just keeping it all together.

FWIW: I think it will only hurt folks with preconceived notion of what a C++
build should look like. In my life, I've seen way too many different
builds systems
come and go to worry about that ;-)

Thanks,
Roman

Re: New Repo for Native Client

[Roman Shaposhnik]

As was pointed out by Anthony, the way to go for Native builds is
Docker containers.

Thanks,
Roman.

On Tue, Jan 17, 2017 at 7:27 AM, Michael William Dodge
<mdo...@pivotal.io> wrote:
> I know a guy (sadly not part of the Geode community) who has used Jenkins 
> with gcc. I could probably use him as a resource if we need some extra 
> expertise on that.
>
> Sarge
>
>> On 17 Jan, 2017, at 06:39, Anthony Baker <aba...@pivotal.io> wrote:
>>
>> There was some work done earlier to run the Jenkins jobs in a Docker 
>> container.  We’re not currently doing that, but I think that’s your best bet 
>> to get a stable environment.  See 
>> https://issues.apache.org/jira/browse/GEODE-60.
>>
>> Anthony
>>
>>> On Jan 16, 2017, at 8:51 PM, Jacob Barrett <jbarr...@pivotal.io> wrote:
>>>
>>> Roman or Mark,
>>>
>>> Reading the list of build tools on the Jenkins slaves it sounds like these
>>> boxes are geared solely towards Java compilation. Is there a build system
>>> or slave for building native bits?
>>>
>>> We will need GCC 4.9 or newer (C++11), CMake, Doxygen, and a few other
>>> tools.
>>>
>>> Thanks,
>>> Jake
>>>
>>>
>>> On Mon, Jan 16, 2017 at 8:47 PM Jacob Barrett <jbarr...@pivotal.io> wrote:
>>>
>>>> Roman,
>>>>
>>>> I understand what you are saying. I think that since the build process
>>>> between the Java Geode bits and the Native Geode bits will completely
>>>> different it might help to have the separate. Until someone comes up with a
>>>> good cross platform and cross language build tool that is commonly used in
>>>> the development environments for each language these builds will remain
>>>> different. Gradle sucks for building C++ and .NET sources and CMake sucks
>>>> for building Java sources. Gradle is not popular in the native project
>>>> world nor is CMake popular in the Java world. So making one build system to
>>>> cover them all would just hurt everyone. Since the experience will be
>>>> unique for each I feel that it justifies a separate repo but I can totally
>>>> see the other side of just keeping it all together.
>>>>
>>>> I too am worried about being isolated but I think as long as it is just
>>>> the repo we should be fine.
>>>>
>>>> Thanks,
>>>> jake
>>>>
>>>>
>>>> On Mon, Jan 16, 2017 at 4:14 PM Roman Shaposhnik <ro...@shaposhnik.org>
>>>> wrote:
>>>>
>>>> Here's my own, personal minority report: I think that a separate repo
>>>> will complicate your build and release process and will fracture your
>>>> nascent community. That said...
>>>>
>>>> On Mon, Jan 16, 2017 at 3:58 PM, Jacob Barrett <jbarr...@pivotal.io>
>>>> wrote:
>>>>> Mark,
>>>>>
>>>>> Looks like we have lots of votes for your separate repo idea. What do we
>>>>> need to do to get that going?
>>>>
>>>> This is a self-managing thing. Here's the tool:
>>>>   https://reporeq.apache.org/
>>>>
>>>>> On that note too, do you know who we need to ping to get a build going?
>>>>
>>>> Did I mention complications to build and release process? ;-)
>>>>
>>>> At any rate -- there's nobody to ping -- it'll be you Jacob (or whoever
>>>> else is signing up to hack on the Native client). Mark can give you
>>>> Jenkins karma tho:
>>>>   https://wiki.apache.org/general/Jenkins#How_do_I_get_an_account
>>>>
>>>>> I would suggest we target Linux first since it is the easiest. The tools
>>>>> necessary can be found in the src/BUILDING.md file.
>>>>
>>>> That's very much up to whoever is doing the actual work, but it sounds
>>>> reasonable.
>>>>
>>>> Thanks,
>>>> Roman.
>>>>
>>>>
>>
>

Re: JIRA components for native clients

[Roman Shaposhnik]

We do have an existing "native client" component. What should we do with that?

Thanks,
Roman.

On Mon, Jan 16, 2017 at 12:00 PM, Jacob Barrett  wrote:
> Would someone with some karma please add some components to the JIRA for
> native client.
>
> At a minimum "C++ client" and ".NET client" components would be helpful.
>
> Thanks,
> Jake

Re: Visualize Geode Metrics/Stats with Grafana Dashboards

[Roman Shaposhnik]

On Fri, Jan 13, 2017 at 10:10 AM, Michael Stolz  wrote:
> Geode doesn't have a time-series capability yet, but I have a proposal that
> I'm planning to submit to add it.

Looking forward to it!

Thanks,
Roman.

Re: Visualize Geode Metrics/Stats with Grafana Dashboards

[Roman Shaposhnik]

First of all -- this looks awesome! You're the man!

On Wed, Jan 11, 2017 at 3:42 AM, Christian Tzolov  wrote:
> At the moment the tool uses InfluxDB as a time-series DB.

[...]

> Another even more interesting angle is to make Geode itself a Grafana
> compliant datasource (http://docs.grafana.org/plugins/datasources,
> https://grafana.net/plugins). So

Anybody can comment on this? This really gets me super excited, but I don't
know whether Geode for the TS Database would make sense or not.

> Do you think it would be worth bringing part of this work under Geode
> project umbrella?

I'd love to have it! At least as a contrib to begin with. Now having
it, of course,
means that we all collectively signing up to maintaining it with each release,
but I think it is well worth it!

Thanks,
Roman.

Re: copy files to servers

[Roman Shaposhnik]

Btw, I'm sure a comparison of capabilities with Ignite will come up at
some point. So here's what
they do in this department (which I personally find really cool):
   http://apacheignite.gridgain.org/v1.0/docs/zero-deployment

Thanks,
Roman.

On Fri, Jan 6, 2017 at 12:11 PM, Anthony Baker  wrote:
> Hmmm, I agree with Udo.  I’d like to push a new version of my application 
> with a single idempotent command.  The server should be smart enough to 
> figure out what's in my bundle and understand how to deploy it including any 
> dependencies (because who writes dependency-free code?).
>
> I do want some lifecycle hooks to alloc/free resources.  This seems 
> conceptually similar to the “war” model which is pretty familiar to most Java 
> devs.
>
> Anthony
>
>> On Jan 6, 2017, at 11:37 AM, Udo Kohlmeyer  wrote:
>>
>> In some ways that is a great idea but sometimes too explicit... Do we 
>> expect them to have fine grained jars?
>> Also how do we handle dependencies as a single util class might be used 
>> by both a cache-listener and a partition listener... is the expectation that 
>> we update the dependent util class for one but not the other
>>
>> It's a very grey area
>>
>> On 1/6/17 11:19, John Blum wrote:
>>> How about...
>>>
>>> * deploy function
>>> * deploy cache-listener
>>> * deploy cache-loader
>>> * deploy cache-loader
>>> * deploy resource (jar, xml, properties, etc)
>>> * etc.
>>>
>>> Might as was make it explicit.  For instance, I may have a JAR file I just
>>> deployed (uploaded) that contains Functions, Listeners, Loaders, Writers,
>>> etc but I only want to deploy functions.
>>>
>>> Having 1 uber "deploy" command with many options gets cumbersome.
>>>
>>> It is a simple matter to introduce multiple command but have those commands
>>> share similar logic.  This would also enable different workflows for
>>> different commands in a more non-convoluted, maintainable way.
>>>
>>> These could be matched with corresponding `undeploy` commands.
>>>
>>> Food for thought,
>>>
>>> John
>>>
>>>
>>>
>>> On Fri, Jan 6, 2017 at 11:11 AM, Kirk Lund  wrote:
>>>
 With appropriate constraints, a copy file type command could be secure.

 1) don't use Apache Geode without security AND make the command require
 authorization permissions
 2) limit the target directory to a directory under the working directory of
 the remote server
 3) rename it to "deploy resource" so people don't expect it to copy to an
 arbitrary target directory on the remote machine

 Back to "deploy jar":

 The deploy command is only for deploying Apache Geode callbacks (Function,
 CacheListener, etc). "deploy resource" such Spring jars or Spring xml files
 or anything similar does not overlap with "deploy jar". There is continued
 confusion over what "deploy jar" is or does. I propose we rename it to
 "deploy functions" or "deploy callbacks" or something along those lines to
 end the confusion.

 On Fri, Jan 6, 2017 at 8:13 AM, Michael Stolz  wrote:

> So maybe a generic copy command is too insecure, I agree.
> What we should do is think about exactly what files we think we are
 trying
> to deploy.
>
>1. I believe that there is a need to deploy dependency jars into the
>system classpath.
>2. I believe that there is also a desire to be able to deploy Spring
>Data Geode xml configuration files.
>3. There is also an issue with attempting to deploy Spring Data Geode
>functions that we should try to work out.
>
> Anything else that anyone is aware of?
>
>
>
>
> --
> Mike Stolz
> Principal Engineer, GemFire Product Manager
> Mobile: 631-835-4771
>
> On Fri, Jan 6, 2017 at 10:59 AM, Jacob Barrett 
> wrote:
>
>> Agree with Anthony. A copy command would either duplicate what deploy
> does
>> by only putting files within as specific location in the server's
> directory
>> or creating a security nightmare if allowed to write anywhere on the
> host.
>>
>> On Fri, Jan 6, 2017 at 7:56 AM Anthony Baker 
 wrote:
>>> I think there are lots of great OS orchestration and automation
 tools.
>>> I’m not sure I understand the need for `gfsh cp`.  If I could easily
> grab
>>> the member hostnames from `gfsh list members` and pipe them into
 mpssh
>> (for
>>> example) that would do the job.
>>>
>>> I *do* like the idea of an improved `gfsh deploy` that supports hot
>> deploy
>>> and reconfiguration.
>>>
>>> Anthony
>>>
 On Jan 6, 2017, at 2:38 AM, Swapnil Bawaskar >> wrote:
 Some application may need to copy files to all the servers. These
> files
 could either be data files or they 

[jira] [Commented] (GEODE-1960) Provide protection against malicious developer jars deployed as functions

[Roman Shaposhnik (JIRA)]

[ 
https://issues.apache.org/jira/browse/GEODE-1960?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15806436#comment-15806436
 ] 

Roman Shaposhnik commented on GEODE-1960:
-

[~dhardman] just so I understand, are you talking about a concept similar to 
what, lets say Apache Tomcan is doing with its security manager? E.g. 
http://tomcat.apache.org/tomcat-7.0-doc/security-manager-howto.html

> Provide protection against malicious developer jars deployed as functions
> -
>
> Key: GEODE-1960
> URL: https://issues.apache.org/jira/browse/GEODE-1960
> Project: Geode
>  Issue Type: Improvement
>  Components: functions, security
>Reporter: Diane Hardman
>Priority: Minor
>  Labels: ClassLoader, DeployCommand, deploy
>
> Provide protection for functions that might contain malicious calls like 
> System.exit().



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Re: Unable to post to reviewboard

[Roman Shaposhnik]

I suggest reaching out to ASF INFRA:
https://www.apache.org/dev/infra-contact#misdirected

Thanks,
Roman.

On Fri, Dec 9, 2016 at 7:45 AM, Jens Deppe  wrote:
> I'm having trouble posting to reviewboard and am getting this error (debug
> output):
>
> ± jd+am |feature/GEODE-2201 ✗| → rbt post -d --repository geode
> --tracking-branch=origin/develop --target-groups=geode --server
> https://reviews.apache.org
 RBTools 0.7.5
 Python 2.6.9 (unknown, Jul 30 2016, 18:31:01)
> [GCC 4.2.1 Compatible Apple LLVM 8.0.0 (clang-800.0.34)]
 Running on Darwin-16.1.0-x86_64-i386-64bit
 Home = /Users/pivotal
 Current directory = /Users/pivotal/workspace/gemfire/open
 Checking for a Subversion repository...
 Running: svn --non-interactive info
 Command exited with rc 1: ['svn', '--non-interactive', u'info']
> svn: E155007: '/Users/pivotal/workspace/gemfire/open' is not a working copy
> ---
 Checking for a Git repository...
 Running: git rev-parse --git-dir
 Running: git config core.bare
 Running: git rev-parse --show-toplevel
 Running: git symbolic-ref -q HEAD
 Running: git config --get branch.feature/GEODE-2201.merge
 Running: git config --get branch.feature/GEODE-2201.remote
 Running: git config --get remote.origin.url
 repository info: Path: https://git-wip-us.apache.org/repos/asf/geode.git,
> Base path: , Supports changesets: False
 Making HTTP GET request to https://reviews.apache.org/api/
 Running: git rev-parse refs/heads/feature/GEODE-2201
 Running: git merge-base d4276136337d2c9bee7336bafa281f2301af2a40
> origin/develop
 Running: git rev-parse 1d951e334334393c2052307988a21c1cd1f11985
 Running: git status --porcelain --untracked-files=no
 Running: git rev-parse --git-dir
 Running: git version
 Running: git -c core.quotepath=false -c diff.noprefix=false diff
> --no-color --full-index --ignore-submodules -M --no-ext-diff
> 1d951e334334393c2052307988a21c1cd1f11985..d4276136337d2c9bee7336bafa281f2301af2a40
 Making HTTP GET request to
> https://reviews.apache.org/api/validation/diffs/
 Cached response for HTTP GET
> https://reviews.apache.org/api/validation/diffs/ expired and was modified
 Making HTTP POST request to
> https://reviews.apache.org/api/validation/diffs/
 Got API Error 224 (HTTP code 400): The specified diff file could not be
> parsed.
 Error data: {u'stat': u'fail', u'reason': u'error: unable to find
> 91465f08f7a483cab0164f5d84dc8b0b8bdf780c\nfatal: git cat-file
> 91465f08f7a483cab0164f5d84dc8b0b8bdf780c: bad file\n', u'err': {u'msg':
> u'The specified diff file could not be parsed.', u'code': 224}}
> Traceback (most recent call last):
>   File "/usr/local/bin/rbt", line 9, in 
> load_entry_point('RBTools==0.7.5.dev0', 'console_scripts', 'rbt')()
>   File "/Library/Python/2.6/site-packages/rbtools/commands/main.py", line
> 133, in main
> command.run_from_argv([RB_MAIN, command_name] + args)
>   File "/Library/Python/2.6/site-packages/rbtools/commands/__init__.py",
> line 622, in run_from_argv
> exit_code = self.main(*args) or 0
>   File "/Library/Python/2.6/site-packages/rbtools/commands/post.py", line
> 754, in main
> (msg_prefix, e))
> rbtools.commands.CommandError: Error validating diff
>
> The specified diff file could not be parsed. (HTTP 400, API Error 224)
>
>
> However, in my repo the blob in question
> (91465f08f7a483cab0164f5d84dc8b0b8bdf780c) does exist and was introduced in
> commit 67dafd8 'GEODE-2112: UITests actually take screenshots on failure'.
>
> --Jens

Re: Nightly geode build artifacts

[Roman Shaposhnik]

On Wed, Dec 7, 2016 at 10:23 AM, Kirk Lund  wrote:
> Close but not quite. The user would need the product tree that would be
> under geode-assembly/build/install/apache-geode. They would then use
> geode-assembly/build/install/apache-geode/bin/gfsh to start and stop
> locators and servers.

Then just make sure the assembly gets deployed to Maven as a regular
zip artifact. Pretty easy to do with Gradle.

Thanks,
Roman.

Re: Nightly geode build artifacts

[Roman Shaposhnik]

On Wed, Dec 7, 2016 at 9:34 AM, Kirk Lund  wrote:
> Does the nightly build produce some sort of binary that users could
> download and use? I'm thinking about a specific user that hits a
> problematic bug in 1.0.0 but it's fixed on develop for 1.1.0. If the
> nightly build produces a downloadable binary then they could try that out
> without having to clone the repo and build from sources.

Is this what you're looking for:
 https://repository.apache.org/content/groups/snapshots/org/apache/geode/
?

Thanks,
Roman.

Re: TLP Transition Changes Coming

[Roman Shaposhnik]

On Tue, Dec 6, 2016 at 11:43 AM, Mark Bretl  wrote:
> Roman,
>
> Looks a ticket then. Do you still want to be the 'Lead' for the project? Or
> should we update to PMC Chair since we are a TLP now.

I think PMC Chair makes much more sense.

Thanks,
Roman.

Re: TLP Transition Changes Coming

[Roman Shaposhnik]

Mark, you're one of the JIRA admins for the project -- if you can't do
it -- it means you need to file an INFRA ticket.

Thanks,
Roman.

On Tue, Dec 6, 2016 at 10:37 AM, Mark Bretl <asf.mbr...@gmail.com> wrote:
> Roman,
>
> It looks like JIRA notifications are still going to
> iss...@geode.incubator.apache.org and Geode is still listed as Incubator,
> would you be able to update it or is that a task for INFRA?
>
> --Mark
>
> On Mon, Dec 5, 2016 at 5:11 PM, Mark Bretl <asf.mbr...@gmail.com> wrote:
>
>> TLP Tasks are completed.
>>
>> New Apache Git URL: https://git.apache.org/geode.git
>> New GitHub Mirror URL: https://github.com/apache/geode
>>
>> Easiest way to migrate to the new URL is to edit your Git config. Looks
>> like the GitHub Mirror might take until later tonight, Pacifc Standard Time.
>>
>> If you run into any issues, be sure to email the dev list.
>>
>> --Mark
>>
>>
>>
>> On Mon, Dec 5, 2016 at 10:39 AM, Mark Bretl <asf.mbr...@gmail.com> wrote:
>>
>>> As stated last week, INFRA will be working on the TLP tasks today and I
>>> have been notified they will be starting within the hour.
>>>
>>> Best Regards,
>>>
>>> --Mark
>>>
>>>
>>>
>>> On Fri, Dec 2, 2016 at 12:44 PM, Joey McAllister <jmcallis...@pivotal.io>
>>> wrote:
>>>
>>>> +1
>>>>
>>>> On Fri, Dec 2, 2016 at 12:12 PM Kenneth Howe <kh...@pivotal.io> wrote:
>>>>
>>>> > +1
>>>> > > On Dec 2, 2016, at 12:10 PM, Roman Shaposhnik <ro...@shaposhnik.org>
>>>> > wrote:
>>>> > >
>>>> > > On Fri, Dec 2, 2016 at 11:53 AM, Dave Barnes <dbar...@pivotal.io>
>>>> wrote:
>>>> > >> As I recall, the committer list on the incubator page was far
>>>> behind the
>>>> > >> list on the product's web page (under "Community").
>>>> > >
>>>> > > I believe I updated it to be a pretty close match right before the
>>>> > discussion
>>>> > > on graduation. At any rate -- I think the answer here is the union
>>>> of two
>>>> > > lists ;-)
>>>> > >
>>>> > > Thanks,
>>>> > > Roman.
>>>> >
>>>> >
>>>>
>>>
>>>
>>

Re: Top-Level Project Tasks Completed

[Roman Shaposhnik]

Congrats!

Thanks,
Roman.

On Mon, Dec 5, 2016 at 5:11 PM, Mark Bretl  wrote:
> Hi Everyone,
>
> The Apache INFRA team has completed the Top-Level Project for Apache Geode.
>
> Summary of Changes:
> - Website is https://geode.apache.org/
> - Mailing lists (dev, users, private) now have suffix of '@geode.apache.org'
> - GitHub URL is now https://github.com/apache/geode (It has error of 404
> now, but should resolve in about 7 hours)
>
> If you have any issues, please email the Dev list at 'dev@geode.apache.org'
> so we can investigate.
>
> Thanks to everyone for their patience. Now, let's move forward with growing
> this community and completing another release!
>
> Best Regards,
>
> Mark Bretl
> V.P. Apache Geode

Re: TLP Transition Changes Coming

[Roman Shaposhnik]

On Fri, Dec 2, 2016 at 12:27 AM, Mark Bretl  wrote:
> John, Redirects are in place for the incubator website once hosting is
> moved.
>
> Anthony,
>
> Lots of questions and I think have most of the answers...
>
> - Status of items in [1]:
> -- All items are completed except for the remaining INFRA transition items.
> -- Our reporting schedule after the initial three months is February, May,
> August, November.
> -- Once the private mail list has been confirmed as migrated, then all
> non-PMC members will be removed from the list if they have not been already
> - Geode Committer List [2]: It is correct as it was submitted by the
> resolution (as only PMC members were defined), however, yes I do agree that
> even though people were not added to the PMC they should have retained
> committer status.
> - Common Tasks [3]: I already created INFRA-12938 as the Common Task
> ticket. I added those items from your list which I did not have to that
> ticket.
> -- Note: I do not see a Geode Sonar project on https://analysis.apache.org,
> so we will have to request one later.
>
> How would people like to handle re-adding the initial committers? Should it
> be done on an as-needed basis? However, I hope we would not need to vote
> them into committers again for Geode.
>
> Does anyone have any history or experience with this issue?

Committers and PMCs are two completely different things. ASF only
concerns itself
with the PMC composition -- the rest it up to your project to define.

Theoretically you could have a re-qualification for committers, but my
recommendation
will simply be to bulk add all of the existing ones to the LDAP group.

Thanks,
Roman.

Re: [ANNOUNCE] Geode Is Now A Top-Level Apache Project

[Roman Shaposhnik]

Major congratulations to the Geode community!

The final steps to wrap up the graduation process
are listed here:
   http://incubator.apache.org/guides/graduation.html#life-after-graduation
please make sure to go through them.

Also, I highly recommend all of the PMC (but especially
Mark) attend the next ASF board meeting as guests. I think
that would really help understand the level of reporting that
the board will require from the PMC.

Thanks,
Roman.

On Mon, Nov 21, 2016 at 7:48 PM, Mark Bretl  wrote:
> Hello Everyone,
>
> It is a great honor to announce the Apache Geode project has graduated to a
> Top-Level Project (TLP)
>
> The official announcement is on the Apache Blog at:
> https://blogs.apache.org/foundation/entry/the_apache_software_foundation_announces102
>
> I would like to thank everyone for their contributions, especially our
> mentors during the incubation process, to getting our community to this
> milestone.
>
> As VP of Apache Geode, I am here to serve the community. Please let me know
> if you have issues or need anything.
>
> More on the technical details of the transition from Incubator to TLP
> project in emails to come.
>
> Looking forward to continuing to grow the community,
>
> Mark Bretl
> V.P. Apache Geode