Re: [DISCUSS] METRON-433: Documentation update

[Anand Subramanian]

Thank you, Casey.

Please see comment here:
https://github.com/apache/incubator-metron/pull/260#issuecomment-248315513

Regards,
Anand

From: Casey Stella mailto:ceste...@gmail.com>>
Reply-To: 
"user@metron.incubator.apache.org<mailto:user@metron.incubator.apache.org>" 
mailto:user@metron.incubator.apache.org>>
Date: Tuesday, September 20, 2016 at 7:48 PM
To: "d...@metron.incubator.apache.org<mailto:d...@metron.incubator.apache.org>" 
mailto:d...@metron.incubator.apache.org>>
Cc: "user@metron.incubator.apache.org<mailto:user@metron.incubator.apache.org>" 
mailto:user@metron.incubator.apache.org>>
Subject: Re: [DISCUSS] METRON-433: Documentation update

Oh, it appears that attachments get stripped off sending to the mailing lists.  
Maybe you can make the comment on the PR 
(https://github.com/apache/incubator-metron/pull/260) and link to it here?

On Tue, Sep 20, 2016 at 10:10 AM, Anand Subramanian 
mailto:asubraman...@hortonworks.com>> wrote:
Would the Storm topology visualization diagram work for the parser, enrichment 
and indexing illustration?

Attached is a sample image for the parser topology. I can work on getting rid 
of the numbers beneath the names.

Cheers,
Anand

From: Casey Stella mailto:ceste...@gmail.com>>
Reply-To: 
"user@metron.incubator.apache.org<mailto:user@metron.incubator.apache.org>" 
mailto:user@metron.incubator.apache.org>>
Date: Tuesday, September 20, 2016 at 7:03 PM
To: "d...@metron.incubator.apache.org<mailto:d...@metron.incubator.apache.org>" 
mailto:d...@metron.incubator.apache.org>>
Cc: "user@metron.incubator.apache.org<mailto:user@metron.incubator.apache.org>" 
mailto:user@metron.incubator.apache.org>>
Subject: Re: [DISCUSS] METRON-433: Documentation update

I do think that's a good idea.  Unfortunately, I made those images in 
powerpoint and I don't know how to get stencils with the storm symbols on them. 
 Does anyone want to volunteer to transition the architecture docs over to use 
storm symbols?

Best,

Casey

On Tue, Sep 20, 2016 at 8:30 AM, zeo...@gmail.com<mailto:zeo...@gmail.com> 
mailto:zeo...@gmail.com>> wrote:
Do you think it would be worthwhile to revise the enrichment and parser
illustrations to use storm component symbols instead of generic symbols?
That may add additional clarity to people who aren't familiar with all of
the different components and how they work together.  Still running through
the updated docs but so far it's great stuff.

Jon

On Mon, Sep 19, 2016 at 6:18 PM Kevin Waterson 
mailto:kevin.water...@gmail.com>>
wrote:

> I have a revised document for installation that I would like to
> contribute. (ubuntu).
> What I have not found is an easily accessible of contributing. This may be
> due to my time poor life where I simply do not have the time to be
> searching for such a link.
> Where can I find community contributions link?
>
> Kev
>
>
>
> On Tue, Sep 20, 2016 at 8:06 AM, Casey Stella 
> mailto:ceste...@gmail.com>> wrote:
>
>> Hi all,
>>
>> I noticed that our public facing docs were really getting stale.  This is
>> a normal thing in a fast moving project, but I imagine it's really hard for
>> the community to get engaged with a project that has such details caught up
>> inside of code and unit tests.  As such, I thought it is reasonable to
>> spend a little time ensuring the docs are better organized and clear, so I
>> spent the day focusing on that:
>>
>
>>1. they were nonexistent in a lot of places like model as a service
>>which are new
>>
>>
>>1. they were stale (mostly around stellar functions)
>>
>>
>>1. they were incomplete (parsers take configs, we don't document
>>that..we now do)
>>
>>
>>1. they were not clear. I added some clarification and pictures of
>>the topologies
>>
>>
>>1. they were in the wrong place (e.g. enrichment config documentation
>>was in metron-common instead of metron-enrichment)
>>
>>
>>1. there wasn't a starting place from the top level Readme into the
>>scary subdepths of the individual projects
>>
>> I also gave a small fully worked example for some of the commonly
>> confusing tasks, like doing a stellar enrichment or deploying a MaaS
>> model.  I wrote them up, but they were caught up inside of PR comments,
>> this just brought them into the docs.
>>
>
>> You can find the PR here
>> <https://github.com/apache/incubator-metron/pull/260>. It's already
>> gotten a +1, but I'd love to see some discussion around what ELSE we can do
>> to make it easier to get community contributions.  After all, there would
>> be no Metron without its community.
>>
>> Best,
>>
>> Casey
>>
>
>
>
> --
> --
> "Democracy is two wolves and a lamb voting on what to have for lunch.
> Liberty is a well-armed lamb contesting the vote."
>
--

Jon

Re: [DISCUSS] METRON-433: Documentation update

[Casey Stella]

One quick update, I also added a section in metron-parsers,
metron-enrichment and metron-indexing detailing the performance knobs to
tweak.  I am not giving direct suggestions as to the numbers, but just the
available knobs available to tweak in each context, how to tweak them and
some hint as to what they might do.  I notice a lot of questions that I get
from people actually deploying Metron is around that, so I thought it
worthwhile to bring out in the docs.

On Mon, Sep 19, 2016 at 6:06 PM, Casey Stella  wrote:

> Hi all,
>
> I noticed that our public facing docs were really getting stale.  This is
> a normal thing in a fast moving project, but I imagine it's really hard for
> the community to get engaged with a project that has such details caught up
> inside of code and unit tests.  As such, I thought it is reasonable to
> spend a little time ensuring the docs are better organized and clear, so I
> spent the day focusing on that:
>
>1. they were nonexistent in a lot of places like model as a service
>which are new
>2. they were stale (mostly around stellar functions)
>3. they were incomplete (parsers take configs, we don't document
>that..we now do)
>4. they were not clear. I added some clarification and pictures of the
>topologies
>5. they were in the wrong place (e.g. enrichment config documentation
>was in metron-common instead of metron-enrichment)
>6. there wasn't a starting place from the top level Readme into the
>scary subdepths of the individual projects
>
> I also gave a small fully worked example for some of the commonly
> confusing tasks, like doing a stellar enrichment or deploying a MaaS
> model.  I wrote them up, but they were caught up inside of PR comments,
> this just brought them into the docs.
>
> You can find the PR here
> . It's already
> gotten a +1, but I'd love to see some discussion around what ELSE we can do
> to make it easier to get community contributions.  After all, there would
> be no Metron without its community.
>
> Best,
>
> Casey
>

Re: [DISCUSS] METRON-433: Documentation update

[Casey Stella]

Oh, it appears that attachments get stripped off sending to the mailing
lists.  Maybe you can make the comment on the PR (
https://github.com/apache/incubator-metron/pull/260) and link to it here?

On Tue, Sep 20, 2016 at 10:10 AM, Anand Subramanian <
asubraman...@hortonworks.com> wrote:

> Would the Storm topology visualization diagram work for the parser,
> enrichment and indexing illustration?
>
> Attached is a sample image for the parser topology. I can work on getting
> rid of the numbers beneath the names.
>
> Cheers,
> Anand
>
> From: Casey Stella 
> Reply-To: "user@metron.incubator.apache.org" <
> user@metron.incubator.apache.org>
> Date: Tuesday, September 20, 2016 at 7:03 PM
> To: "d...@metron.incubator.apache.org" 
> Cc: "user@metron.incubator.apache.org" 
> Subject: Re: [DISCUSS] METRON-433: Documentation update
>
> I do think that's a good idea.  Unfortunately, I made those images in
> powerpoint and I don't know how to get stencils with the storm symbols on
> them.  Does anyone want to volunteer to transition the architecture docs
> over to use storm symbols?
>
> Best,
>
> Casey
>
> On Tue, Sep 20, 2016 at 8:30 AM, zeo...@gmail.com 
> wrote:
>
>> Do you think it would be worthwhile to revise the enrichment and parser
>> illustrations to use storm component symbols instead of generic symbols?
>> That may add additional clarity to people who aren't familiar with all of
>> the different components and how they work together.  Still running
>> through
>> the updated docs but so far it's great stuff.
>>
>> Jon
>>
>> On Mon, Sep 19, 2016 at 6:18 PM Kevin Waterson 
>> wrote:
>>
>> > I have a revised document for installation that I would like to
>> > contribute. (ubuntu).
>> > What I have not found is an easily accessible of contributing. This may
>> be
>> > due to my time poor life where I simply do not have the time to be
>> > searching for such a link.
>> > Where can I find community contributions link?
>> >
>> > Kev
>> >
>> >
>> >
>> > On Tue, Sep 20, 2016 at 8:06 AM, Casey Stella 
>> wrote:
>> >
>> >> Hi all,
>> >>
>> >> I noticed that our public facing docs were really getting stale.  This
>> is
>> >> a normal thing in a fast moving project, but I imagine it's really
>> hard for
>> >> the community to get engaged with a project that has such details
>> caught up
>> >> inside of code and unit tests.  As such, I thought it is reasonable to
>> >> spend a little time ensuring the docs are better organized and clear,
>> so I
>> >> spent the day focusing on that:
>> >>
>> >
>> >>1. they were nonexistent in a lot of places like model as a service
>> >>which are new
>> >>
>> >>
>> >>1. they were stale (mostly around stellar functions)
>> >>
>> >>
>> >>1. they were incomplete (parsers take configs, we don't document
>> >>that..we now do)
>> >>
>> >>
>> >>1. they were not clear. I added some clarification and pictures of
>> >>the topologies
>> >>
>> >>
>> >>1. they were in the wrong place (e.g. enrichment config
>> documentation
>> >>was in metron-common instead of metron-enrichment)
>> >>
>> >>
>> >>1. there wasn't a starting place from the top level Readme into the
>> >>scary subdepths of the individual projects
>> >>
>> >> I also gave a small fully worked example for some of the commonly
>> >> confusing tasks, like doing a stellar enrichment or deploying a MaaS
>> >> model.  I wrote them up, but they were caught up inside of PR comments,
>> >> this just brought them into the docs.
>> >>
>> >
>> >> You can find the PR here
>> >> <https://github.com/apache/incubator-metron/pull/260>. It's already
>> >> gotten a +1, but I'd love to see some discussion around what ELSE we
>> can do
>> >> to make it easier to get community contributions.  After all, there
>> would
>> >> be no Metron without its community.
>> >>
>> >> Best,
>> >>
>> >> Casey
>> >>
>> >
>> >
>> >
>> > --
>> > --
>> > "Democracy is two wolves and a lamb voting on what to have for lunch.
>> > Liberty is a well-armed lamb contesting the vote."
>> >
>> --
>>
>> Jon
>>
>
>

Re: [DISCUSS] METRON-433: Documentation update

[Anand Subramanian]

Would the Storm topology visualization diagram work for the parser, enrichment 
and indexing illustration?

Attached is a sample image for the parser topology. I can work on getting rid 
of the numbers beneath the names.

Cheers,
Anand

From: Casey Stella mailto:ceste...@gmail.com>>
Reply-To: 
"user@metron.incubator.apache.org<mailto:user@metron.incubator.apache.org>" 
mailto:user@metron.incubator.apache.org>>
Date: Tuesday, September 20, 2016 at 7:03 PM
To: "d...@metron.incubator.apache.org<mailto:d...@metron.incubator.apache.org>" 
mailto:d...@metron.incubator.apache.org>>
Cc: "user@metron.incubator.apache.org<mailto:user@metron.incubator.apache.org>" 
mailto:user@metron.incubator.apache.org>>
Subject: Re: [DISCUSS] METRON-433: Documentation update

I do think that's a good idea.  Unfortunately, I made those images in 
powerpoint and I don't know how to get stencils with the storm symbols on them. 
 Does anyone want to volunteer to transition the architecture docs over to use 
storm symbols?

Best,

Casey

On Tue, Sep 20, 2016 at 8:30 AM, zeo...@gmail.com<mailto:zeo...@gmail.com> 
mailto:zeo...@gmail.com>> wrote:
Do you think it would be worthwhile to revise the enrichment and parser
illustrations to use storm component symbols instead of generic symbols?
That may add additional clarity to people who aren't familiar with all of
the different components and how they work together.  Still running through
the updated docs but so far it's great stuff.

Jon

On Mon, Sep 19, 2016 at 6:18 PM Kevin Waterson 
mailto:kevin.water...@gmail.com>>
wrote:

> I have a revised document for installation that I would like to
> contribute. (ubuntu).
> What I have not found is an easily accessible of contributing. This may be
> due to my time poor life where I simply do not have the time to be
> searching for such a link.
> Where can I find community contributions link?
>
> Kev
>
>
>
> On Tue, Sep 20, 2016 at 8:06 AM, Casey Stella 
> mailto:ceste...@gmail.com>> wrote:
>
>> Hi all,
>>
>> I noticed that our public facing docs were really getting stale.  This is
>> a normal thing in a fast moving project, but I imagine it's really hard for
>> the community to get engaged with a project that has such details caught up
>> inside of code and unit tests.  As such, I thought it is reasonable to
>> spend a little time ensuring the docs are better organized and clear, so I
>> spent the day focusing on that:
>>
>
>>1. they were nonexistent in a lot of places like model as a service
>>which are new
>>
>>
>>1. they were stale (mostly around stellar functions)
>>
>>
>>1. they were incomplete (parsers take configs, we don't document
>>that..we now do)
>>
>>
>>1. they were not clear. I added some clarification and pictures of
>>the topologies
>>
>>
>>1. they were in the wrong place (e.g. enrichment config documentation
>>was in metron-common instead of metron-enrichment)
>>
>>
>>1. there wasn't a starting place from the top level Readme into the
>>scary subdepths of the individual projects
>>
>> I also gave a small fully worked example for some of the commonly
>> confusing tasks, like doing a stellar enrichment or deploying a MaaS
>> model.  I wrote them up, but they were caught up inside of PR comments,
>> this just brought them into the docs.
>>
>
>> You can find the PR here
>> <https://github.com/apache/incubator-metron/pull/260>. It's already
>> gotten a +1, but I'd love to see some discussion around what ELSE we can do
>> to make it easier to get community contributions.  After all, there would
>> be no Metron without its community.
>>
>> Best,
>>
>> Casey
>>
>
>
>
> --
> --
> "Democracy is two wolves and a lamb voting on what to have for lunch.
> Liberty is a well-armed lamb contesting the vote."
>
--

Jon

Re: [DISCUSS] METRON-433: Documentation update

[Casey Stella]

I do think that's a good idea.  Unfortunately, I made those images in
powerpoint and I don't know how to get stencils with the storm symbols on
them.  Does anyone want to volunteer to transition the architecture docs
over to use storm symbols?

Best,

Casey

On Tue, Sep 20, 2016 at 8:30 AM, zeo...@gmail.com  wrote:

> Do you think it would be worthwhile to revise the enrichment and parser
> illustrations to use storm component symbols instead of generic symbols?
> That may add additional clarity to people who aren't familiar with all of
> the different components and how they work together.  Still running through
> the updated docs but so far it's great stuff.
>
> Jon
>
> On Mon, Sep 19, 2016 at 6:18 PM Kevin Waterson 
> wrote:
>
> > I have a revised document for installation that I would like to
> > contribute. (ubuntu).
> > What I have not found is an easily accessible of contributing. This may
> be
> > due to my time poor life where I simply do not have the time to be
> > searching for such a link.
> > Where can I find community contributions link?
> >
> > Kev
> >
> >
> >
> > On Tue, Sep 20, 2016 at 8:06 AM, Casey Stella 
> wrote:
> >
> >> Hi all,
> >>
> >> I noticed that our public facing docs were really getting stale.  This
> is
> >> a normal thing in a fast moving project, but I imagine it's really hard
> for
> >> the community to get engaged with a project that has such details
> caught up
> >> inside of code and unit tests.  As such, I thought it is reasonable to
> >> spend a little time ensuring the docs are better organized and clear,
> so I
> >> spent the day focusing on that:
> >>
> >
> >>1. they were nonexistent in a lot of places like model as a service
> >>which are new
> >>
> >>
> >>1. they were stale (mostly around stellar functions)
> >>
> >>
> >>1. they were incomplete (parsers take configs, we don't document
> >>that..we now do)
> >>
> >>
> >>1. they were not clear. I added some clarification and pictures of
> >>the topologies
> >>
> >>
> >>1. they were in the wrong place (e.g. enrichment config documentation
> >>was in metron-common instead of metron-enrichment)
> >>
> >>
> >>1. there wasn't a starting place from the top level Readme into the
> >>scary subdepths of the individual projects
> >>
> >> I also gave a small fully worked example for some of the commonly
> >> confusing tasks, like doing a stellar enrichment or deploying a MaaS
> >> model.  I wrote them up, but they were caught up inside of PR comments,
> >> this just brought them into the docs.
> >>
> >
> >> You can find the PR here
> >> . It's already
> >> gotten a +1, but I'd love to see some discussion around what ELSE we
> can do
> >> to make it easier to get community contributions.  After all, there
> would
> >> be no Metron without its community.
> >>
> >> Best,
> >>
> >> Casey
> >>
> >
> >
> >
> > --
> > --
> > "Democracy is two wolves and a lamb voting on what to have for lunch.
> > Liberty is a well-armed lamb contesting the vote."
> >
> --
>
> Jon
>

Re: [DISCUSS] METRON-433: Documentation update

[zeo...@gmail.com]

Do you think it would be worthwhile to revise the enrichment and parser
illustrations to use storm component symbols instead of generic symbols?
That may add additional clarity to people who aren't familiar with all of
the different components and how they work together.  Still running through
the updated docs but so far it's great stuff.

Jon

On Mon, Sep 19, 2016 at 6:18 PM Kevin Waterson 
wrote:

> I have a revised document for installation that I would like to
> contribute. (ubuntu).
> What I have not found is an easily accessible of contributing. This may be
> due to my time poor life where I simply do not have the time to be
> searching for such a link.
> Where can I find community contributions link?
>
> Kev
>
>
>
> On Tue, Sep 20, 2016 at 8:06 AM, Casey Stella  wrote:
>
>> Hi all,
>>
>> I noticed that our public facing docs were really getting stale.  This is
>> a normal thing in a fast moving project, but I imagine it's really hard for
>> the community to get engaged with a project that has such details caught up
>> inside of code and unit tests.  As such, I thought it is reasonable to
>> spend a little time ensuring the docs are better organized and clear, so I
>> spent the day focusing on that:
>>
>
>>1. they were nonexistent in a lot of places like model as a service
>>which are new
>>
>>
>>1. they were stale (mostly around stellar functions)
>>
>>
>>1. they were incomplete (parsers take configs, we don't document
>>that..we now do)
>>
>>
>>1. they were not clear. I added some clarification and pictures of
>>the topologies
>>
>>
>>1. they were in the wrong place (e.g. enrichment config documentation
>>was in metron-common instead of metron-enrichment)
>>
>>
>>1. there wasn't a starting place from the top level Readme into the
>>scary subdepths of the individual projects
>>
>> I also gave a small fully worked example for some of the commonly
>> confusing tasks, like doing a stellar enrichment or deploying a MaaS
>> model.  I wrote them up, but they were caught up inside of PR comments,
>> this just brought them into the docs.
>>
>
>> You can find the PR here
>> . It's already
>> gotten a +1, but I'd love to see some discussion around what ELSE we can do
>> to make it easier to get community contributions.  After all, there would
>> be no Metron without its community.
>>
>> Best,
>>
>> Casey
>>
>
>
>
> --
> --
> "Democracy is two wolves and a lamb voting on what to have for lunch.
> Liberty is a well-armed lamb contesting the vote."
>
-- 

Jon

Re: [DISCUSS] METRON-433: Documentation update

[Kevin Waterson]

I have a revised document for installation that I would like to contribute.
(ubuntu).
What I have not found is an easily accessible of contributing. This may be
due to my time poor life where I simply do not have the time to be
searching for such a link.
Where can I find community contributions link?

Kev



On Tue, Sep 20, 2016 at 8:06 AM, Casey Stella  wrote:

> Hi all,
>
> I noticed that our public facing docs were really getting stale.  This is
> a normal thing in a fast moving project, but I imagine it's really hard for
> the community to get engaged with a project that has such details caught up
> inside of code and unit tests.  As such, I thought it is reasonable to
> spend a little time ensuring the docs are better organized and clear, so I
> spent the day focusing on that:
>
>1. they were nonexistent in a lot of places like model as a service
>which are new
>2. they were stale (mostly around stellar functions)
>3. they were incomplete (parsers take configs, we don't document
>that..we now do)
>4. they were not clear. I added some clarification and pictures of the
>topologies
>5. they were in the wrong place (e.g. enrichment config documentation
>was in metron-common instead of metron-enrichment)
>6. there wasn't a starting place from the top level Readme into the
>scary subdepths of the individual projects
>
> I also gave a small fully worked example for some of the commonly
> confusing tasks, like doing a stellar enrichment or deploying a MaaS
> model.  I wrote them up, but they were caught up inside of PR comments,
> this just brought them into the docs.
>
> You can find the PR here
> . It's already
> gotten a +1, but I'd love to see some discussion around what ELSE we can do
> to make it easier to get community contributions.  After all, there would
> be no Metron without its community.
>
> Best,
>
> Casey
>



-- 
-- 
"Democracy is two wolves and a lamb voting on what to have for lunch.
Liberty is a well-armed lamb contesting the vote."