Re: [DISCUSS] Slack Channel Use

[Casey Stella]

Agreed, the benefit of the mailing list is that it’s searchable by ponymail
and the major search engines.
On Mon, Oct 22, 2018 at 17:18 Nick Allen  wrote:

> I don't know that it is the same kind of searchable.  Is it being indexed
> by the major search engines?  I have never used a search engine and
> uncovered the answer to my problem in a Slack archive.
>
> On Mon, Oct 22, 2018 at 5:05 PM Otto Fowler 
> wrote:
>
> > According to Greg Stein, an infra admin on the NiFi slack, the ASF slack
> > that metron is in IS the standard plan, not the free one and is
> searchable
> > past 10,000 messages.
> >
> >
> >
> > On October 22, 2018 at 15:35:51, Michael Miklavcic (
> > michael.miklav...@gmail.com) wrote:
> >
> > ...From an archival and broader reach point of view, I do think there's
> > something to be said about using the mailing list. It's also easier to
> link
> > to Q/A threads from the mailing list archives and do searches...
> >
> >
> https://lists.apache.org/thread.html/1aa85bc13d41e04a1f85c3100c2b803abe35d79b54062bbeaab83ace@%3Cdev.metron.apache.org%3E
> >
> > How very Inception.
> >
> >
> > On Mon, Oct 22, 2018 at 1:32 PM Michael Miklavcic <
> > michael.miklav...@gmail.com> wrote:
> >
> > > I just want to point out that we currently have 32 members in the
> Metron
> > > Slack channel which I personally think is a great sign. This is good
> from
> > a
> > > community perspective and helps foster interactive sessions where
> > required.
> > > From an archival and broader reach point of view, I do think there's
> > > something to be said about using the mailing list. It's also easier to
> > link
> > > to Q/A threads from the mailing list archives and do searches. As
> such, I
> > > would also go along with Nick's suggestion and urge members to prefer
> the
> > > user/dev list where possible.
> > >
> > > On Mon, Oct 22, 2018 at 10:51 AM Justin Leet 
> > > wrote:
> > >
> > >> If we want to push more discussion to the dev list, my obvious follow
> up
> > >> question then is "What are we hoping to get out of Slack/irc/other
> > >> interactive medium?". What discussion would we even want on there, if
> we
> > >> can't have decisions and don't want usage/support?
> > >>
> > >> On Mon, Oct 22, 2018 at 12:44 PM Casey Stella 
> > wrote:
> > >>
> > >> > I am of 2 minds, but I tend to agree. On the one hand, it's
> definitely
> > >> the
> > >> > preference that we use the mailing lists for the reasons you stated
> > (and
> > >> > also because not everyone has access to slack generally). On the
> other
> > >> > hand, I think an interactive medium like Slack has a lot of
> advantages
> > >> in
> > >> > terms of user satisfaction. Ultimately, though, we may satisfy 1
> user
> > >> at
> > >> > the cost of not persisting the discussion and satisfying many users.
> > >> >
> > >> > I'll go along with a specific preference to drive more discussion to
> > the
> > >> > mailing list.
> > >> >
> > >> > Casey
> > >> >
> > >> > On Mon, Oct 22, 2018 at 12:18 PM Nick Allen 
> > wrote:
> > >> >
> > >> > > It seems that we are seeing a lot of Metron usage and support
> > >> questions
> > >> > on
> > >> > > the Slack Channel.
> > >> > > These are questions that previously would have been directed to
> the
> > >> User
> > >> > or
> > >> > > Dev mailing lists. Since this is occurring in the Slack Channel,
> the
> > >> > > conversations are not archived.
> > >> > >
> > >> > > In my opinion, this is not good for the Metron community. Having
> > this
> > >> > > persisted in a discoverable form (like a mailing list archive) not
> > >> only
> > >> > > helps support current users, but also helps *potential* users
> > >> understand
> > >> > > how Metron is being used.
> > >> > >
> > >> > > Does anyone else agree or disagree? At a minimum, I feel we need
> to
> > >> do
> > >> > > something to direct these conversations back to the mailing list.
> > >> > >
> > >> >
> > >>
> > >
> >
>

Re: [DISCUSS] Slack Channel Use

[Nick Allen]

I don't know that it is the same kind of searchable.  Is it being indexed
by the major search engines?  I have never used a search engine and
uncovered the answer to my problem in a Slack archive.

On Mon, Oct 22, 2018 at 5:05 PM Otto Fowler  wrote:

> According to Greg Stein, an infra admin on the NiFi slack, the ASF slack
> that metron is in IS the standard plan, not the free one and is searchable
> past 10,000 messages.
>
>
>
> On October 22, 2018 at 15:35:51, Michael Miklavcic (
> michael.miklav...@gmail.com) wrote:
>
> ...From an archival and broader reach point of view, I do think there's
> something to be said about using the mailing list. It's also easier to link
> to Q/A threads from the mailing list archives and do searches...
>
> https://lists.apache.org/thread.html/1aa85bc13d41e04a1f85c3100c2b803abe35d79b54062bbeaab83ace@%3Cdev.metron.apache.org%3E
>
> How very Inception.
>
>
> On Mon, Oct 22, 2018 at 1:32 PM Michael Miklavcic <
> michael.miklav...@gmail.com> wrote:
>
> > I just want to point out that we currently have 32 members in the Metron
> > Slack channel which I personally think is a great sign. This is good from
> a
> > community perspective and helps foster interactive sessions where
> required.
> > From an archival and broader reach point of view, I do think there's
> > something to be said about using the mailing list. It's also easier to
> link
> > to Q/A threads from the mailing list archives and do searches. As such, I
> > would also go along with Nick's suggestion and urge members to prefer the
> > user/dev list where possible.
> >
> > On Mon, Oct 22, 2018 at 10:51 AM Justin Leet 
> > wrote:
> >
> >> If we want to push more discussion to the dev list, my obvious follow up
> >> question then is "What are we hoping to get out of Slack/irc/other
> >> interactive medium?". What discussion would we even want on there, if we
> >> can't have decisions and don't want usage/support?
> >>
> >> On Mon, Oct 22, 2018 at 12:44 PM Casey Stella 
> wrote:
> >>
> >> > I am of 2 minds, but I tend to agree. On the one hand, it's definitely
> >> the
> >> > preference that we use the mailing lists for the reasons you stated
> (and
> >> > also because not everyone has access to slack generally). On the other
> >> > hand, I think an interactive medium like Slack has a lot of advantages
> >> in
> >> > terms of user satisfaction. Ultimately, though, we may satisfy 1 user
> >> at
> >> > the cost of not persisting the discussion and satisfying many users.
> >> >
> >> > I'll go along with a specific preference to drive more discussion to
> the
> >> > mailing list.
> >> >
> >> > Casey
> >> >
> >> > On Mon, Oct 22, 2018 at 12:18 PM Nick Allen 
> wrote:
> >> >
> >> > > It seems that we are seeing a lot of Metron usage and support
> >> questions
> >> > on
> >> > > the Slack Channel.
> >> > > These are questions that previously would have been directed to the
> >> User
> >> > or
> >> > > Dev mailing lists. Since this is occurring in the Slack Channel, the
> >> > > conversations are not archived.
> >> > >
> >> > > In my opinion, this is not good for the Metron community. Having
> this
> >> > > persisted in a discoverable form (like a mailing list archive) not
> >> only
> >> > > helps support current users, but also helps *potential* users
> >> understand
> >> > > how Metron is being used.
> >> > >
> >> > > Does anyone else agree or disagree? At a minimum, I feel we need to
> >> do
> >> > > something to direct these conversations back to the mailing list.
> >> > >
> >> >
> >>
> >
>

Re: [DISCUSS] Slack Channel Use

[Otto Fowler]

According to Greg Stein, an infra admin on the NiFi slack, the ASF slack
that metron is in IS the standard plan, not the free one and is searchable
past 10,000 messages.



On October 22, 2018 at 15:35:51, Michael Miklavcic (
michael.miklav...@gmail.com) wrote:

...From an archival and broader reach point of view, I do think there's
something to be said about using the mailing list. It's also easier to link
to Q/A threads from the mailing list archives and do searches...
https://lists.apache.org/thread.html/1aa85bc13d41e04a1f85c3100c2b803abe35d79b54062bbeaab83ace@%3Cdev.metron.apache.org%3E

How very Inception.


On Mon, Oct 22, 2018 at 1:32 PM Michael Miklavcic <
michael.miklav...@gmail.com> wrote:

> I just want to point out that we currently have 32 members in the Metron
> Slack channel which I personally think is a great sign. This is good from
a
> community perspective and helps foster interactive sessions where
required.
> From an archival and broader reach point of view, I do think there's
> something to be said about using the mailing list. It's also easier to
link
> to Q/A threads from the mailing list archives and do searches. As such, I
> would also go along with Nick's suggestion and urge members to prefer the
> user/dev list where possible.
>
> On Mon, Oct 22, 2018 at 10:51 AM Justin Leet 
> wrote:
>
>> If we want to push more discussion to the dev list, my obvious follow up
>> question then is "What are we hoping to get out of Slack/irc/other
>> interactive medium?". What discussion would we even want on there, if we
>> can't have decisions and don't want usage/support?
>>
>> On Mon, Oct 22, 2018 at 12:44 PM Casey Stella 
wrote:
>>
>> > I am of 2 minds, but I tend to agree. On the one hand, it's definitely
>> the
>> > preference that we use the mailing lists for the reasons you stated
(and
>> > also because not everyone has access to slack generally). On the other
>> > hand, I think an interactive medium like Slack has a lot of advantages
>> in
>> > terms of user satisfaction. Ultimately, though, we may satisfy 1 user
>> at
>> > the cost of not persisting the discussion and satisfying many users.
>> >
>> > I'll go along with a specific preference to drive more discussion to
the
>> > mailing list.
>> >
>> > Casey
>> >
>> > On Mon, Oct 22, 2018 at 12:18 PM Nick Allen 
wrote:
>> >
>> > > It seems that we are seeing a lot of Metron usage and support
>> questions
>> > on
>> > > the Slack Channel.
>> > > These are questions that previously would have been directed to the
>> User
>> > or
>> > > Dev mailing lists. Since this is occurring in the Slack Channel, the
>> > > conversations are not archived.
>> > >
>> > > In my opinion, this is not good for the Metron community. Having
this
>> > > persisted in a discoverable form (like a mailing list archive) not
>> only
>> > > helps support current users, but also helps *potential* users
>> understand
>> > > how Metron is being used.
>> > >
>> > > Does anyone else agree or disagree? At a minimum, I feel we need to
>> do
>> > > something to direct these conversations back to the mailing list.
>> > >
>> >
>>
>

Re: Invite to Slack Channel

[Michael Miklavcic]

Sent

On Mon, Oct 22, 2018 at 1:00 PM vpiserc...@gmail.com 
wrote:

> Hi,
>
> can anyone invite to the metron slack channel?
>
> Thanks,
> vito piserchia
> On 10/22/18 3:31 PM, zeo...@gmail.com wrote:
> > Invite sent
> >
> > On Mon, Oct 22, 2018 at 9:26 AM Muhammed Irshad 
> wrote:
> >
> >> Some one get me also the slack channel link ?
> >> Thanks,
> >> Muhammed Irshad
> >> Q*Burst*
> >> www.qburst.com
> >>
> >>
> >> On Wed, Oct 17, 2018 at 7:33 PM Michael Miklavcic <
> >> michael.miklav...@gmail.com> wrote:
> >>
> >>> Sent
> >>>
> >>> On Wed, Oct 17, 2018 at 7:23 AM Tibor Meller 
> >>> wrote:
> >>>
>  Hi Guys,
>  Can you add me to the apache metron slack chanel?
> 
>  Thanks,
> 
>  On Thu, Oct 4, 2018 at 1:14 PM Otto Fowler 
>  wrote:
> 
> > Done
> >
> >
> > On October 4, 2018 at 05:35:06, Tamás Fodor (ftamas.m...@gmail.com)
>  wrote:
> >
> > Hello,
> >
> > Michael, can you add me as well?
> >
> > Thank you in advance!
> >
> > Tamas
> >
> > On Wed, Oct 3, 2018 at 4:27 PM Michael Miklavcic <
> > michael.miklav...@gmail.com> wrote:
> >
> >> Sent
> >>
> >> On Wed, Oct 3, 2018 at 8:17 AM Shane Ardell <
> >>> shane.m.ard...@gmail.com>
> >> wrote:
> >>
> >>> Hello everyone,
> >>>
> >>> Is it possible for someone to send me an invite to the Metron
> >> Slack
> >>> channel?
> >>>
> >>> Regards,
> >>> Shane
> >>>
> >>
> >
> 
> >>>
> >>
>
>
> --
>  Vito Piserchia
>  Security and Software Engineer
>
>  : vito.piserchia[at]dreamlab.net
>  : 4915 8835 2C18 9CAE F14F 2314 613D 51C5 106B 83EA
>  : https://dreamlab.net
>  : +41 31 398 66 66
>  : +41 31 398 66 69
>  -
>
>  DreamLab Technologies AG
>  Monbijoustrasse 36
>  3011 Bern, Switzerland
>
> -
>
>  This e-mail may contain confidential and/or privileged information.
>  If you are not the intended recipient (or have received this e-mail
>  in error) please notify the sender immediately and destroy this
>  e-mail. Any unauthorised copying, disclosure or distribution of the
>  material in this e-mail is strictly forbidden.
>
> -
>

Re: [DISCUSS] Slack Channel Use

[Michael Miklavcic]

...From an archival and broader reach point of view, I do think there's
something to be said about using the mailing list. It's also easier to link
to Q/A threads from the mailing list archives and do searches...
https://lists.apache.org/thread.html/1aa85bc13d41e04a1f85c3100c2b803abe35d79b54062bbeaab83ace@%3Cdev.metron.apache.org%3E

How very Inception.


On Mon, Oct 22, 2018 at 1:32 PM Michael Miklavcic <
michael.miklav...@gmail.com> wrote:

> I just want to point out that we currently have 32 members in the Metron
> Slack channel which I personally think is a great sign. This is good from a
> community perspective and helps foster interactive sessions where required.
> From an archival and broader reach point of view, I do think there's
> something to be said about using the mailing list. It's also easier to link
> to Q/A threads from the mailing list archives and do searches. As such, I
> would also go along with Nick's suggestion and urge members to prefer the
> user/dev list where possible.
>
> On Mon, Oct 22, 2018 at 10:51 AM Justin Leet 
> wrote:
>
>> If we want to push more discussion to the dev list, my obvious follow up
>> question then is "What are we hoping to get out of Slack/irc/other
>> interactive medium?".  What discussion would we even want on there, if we
>> can't have decisions and don't want usage/support?
>>
>> On Mon, Oct 22, 2018 at 12:44 PM Casey Stella  wrote:
>>
>> > I am of 2 minds, but I tend to agree.  On the one hand, it's definitely
>> the
>> > preference that we use the mailing lists for the reasons you stated (and
>> > also because not everyone has access to slack generally).  On the other
>> > hand, I think an interactive medium like Slack has a lot of advantages
>> in
>> > terms of user satisfaction.  Ultimately, though, we may satisfy 1 user
>> at
>> > the cost of not persisting the discussion and satisfying many users.
>> >
>> > I'll go along with a specific preference to drive more discussion to the
>> > mailing list.
>> >
>> > Casey
>> >
>> > On Mon, Oct 22, 2018 at 12:18 PM Nick Allen  wrote:
>> >
>> > > It seems that we are seeing a lot of Metron usage and support
>> questions
>> > on
>> > > the Slack Channel.
>> > > These are questions that previously would have been directed to the
>> User
>> > or
>> > > Dev mailing lists.  Since this is occurring in the Slack Channel, the
>> > > conversations are not archived.
>> > >
>> > > In my opinion, this is not good for the Metron community.  Having this
>> > > persisted in a discoverable form (like a mailing list archive) not
>> only
>> > > helps support current users, but also helps *potential* users
>> understand
>> > > how Metron is being used.
>> > >
>> > > Does anyone else agree or disagree?  At a minimum, I feel we need to
>> do
>> > > something to direct these conversations back to the mailing list.
>> > >
>> >
>>
>

Re: Invite to Slack Channel

[vpiserc...@gmail.com]

Hi,

can anyone invite to the metron slack channel?

Thanks,
vito piserchia
On 10/22/18 3:31 PM, zeo...@gmail.com wrote:
> Invite sent
> 
> On Mon, Oct 22, 2018 at 9:26 AM Muhammed Irshad  wrote:
> 
>> Some one get me also the slack channel link ?
>> Thanks,
>> Muhammed Irshad
>> Q*Burst*
>> www.qburst.com
>>
>>
>> On Wed, Oct 17, 2018 at 7:33 PM Michael Miklavcic <
>> michael.miklav...@gmail.com> wrote:
>>
>>> Sent
>>>
>>> On Wed, Oct 17, 2018 at 7:23 AM Tibor Meller 
>>> wrote:
>>>
 Hi Guys,
 Can you add me to the apache metron slack chanel?

 Thanks,

 On Thu, Oct 4, 2018 at 1:14 PM Otto Fowler 
 wrote:

> Done
>
>
> On October 4, 2018 at 05:35:06, Tamás Fodor (ftamas.m...@gmail.com)
 wrote:
>
> Hello,
>
> Michael, can you add me as well?
>
> Thank you in advance!
>
> Tamas
>
> On Wed, Oct 3, 2018 at 4:27 PM Michael Miklavcic <
> michael.miklav...@gmail.com> wrote:
>
>> Sent
>>
>> On Wed, Oct 3, 2018 at 8:17 AM Shane Ardell <
>>> shane.m.ard...@gmail.com>
>> wrote:
>>
>>> Hello everyone,
>>>
>>> Is it possible for someone to send me an invite to the Metron
>> Slack
>>> channel?
>>>
>>> Regards,
>>> Shane
>>>
>>
>

>>>
>>


-- 
 Vito Piserchia
 Security and Software Engineer

 : vito.piserchia[at]dreamlab.net
 : 4915 8835 2C18 9CAE F14F 2314 613D 51C5 106B 83EA
 : https://dreamlab.net
 : +41 31 398 66 66
 : +41 31 398 66 69
 -

 DreamLab Technologies AG
 Monbijoustrasse 36
 3011 Bern, Switzerland

-

 This e-mail may contain confidential and/or privileged information.
 If you are not the intended recipient (or have received this e-mail
 in error) please notify the sender immediately and destroy this
 e-mail. Any unauthorised copying, disclosure or distribution of the
 material in this e-mail is strictly forbidden.

-

Re: [DISCUSS] Slack Channel Use

[Justin Leet]

If we want to push more discussion to the dev list, my obvious follow up
question then is "What are we hoping to get out of Slack/irc/other
interactive medium?".  What discussion would we even want on there, if we
can't have decisions and don't want usage/support?

On Mon, Oct 22, 2018 at 12:44 PM Casey Stella  wrote:

> I am of 2 minds, but I tend to agree.  On the one hand, it's definitely the
> preference that we use the mailing lists for the reasons you stated (and
> also because not everyone has access to slack generally).  On the other
> hand, I think an interactive medium like Slack has a lot of advantages in
> terms of user satisfaction.  Ultimately, though, we may satisfy 1 user at
> the cost of not persisting the discussion and satisfying many users.
>
> I'll go along with a specific preference to drive more discussion to the
> mailing list.
>
> Casey
>
> On Mon, Oct 22, 2018 at 12:18 PM Nick Allen  wrote:
>
> > It seems that we are seeing a lot of Metron usage and support questions
> on
> > the Slack Channel.
> > These are questions that previously would have been directed to the User
> or
> > Dev mailing lists.  Since this is occurring in the Slack Channel, the
> > conversations are not archived.
> >
> > In my opinion, this is not good for the Metron community.  Having this
> > persisted in a discoverable form (like a mailing list archive) not only
> > helps support current users, but also helps *potential* users understand
> > how Metron is being used.
> >
> > Does anyone else agree or disagree?  At a minimum, I feel we need to do
> > something to direct these conversations back to the mailing list.
> >
>

Re: [DISCUSS] Slack Channel Use

[Casey Stella]

I am of 2 minds, but I tend to agree.  On the one hand, it's definitely the
preference that we use the mailing lists for the reasons you stated (and
also because not everyone has access to slack generally).  On the other
hand, I think an interactive medium like Slack has a lot of advantages in
terms of user satisfaction.  Ultimately, though, we may satisfy 1 user at
the cost of not persisting the discussion and satisfying many users.

I'll go along with a specific preference to drive more discussion to the
mailing list.

Casey

On Mon, Oct 22, 2018 at 12:18 PM Nick Allen  wrote:

> It seems that we are seeing a lot of Metron usage and support questions on
> the Slack Channel.
> These are questions that previously would have been directed to the User or
> Dev mailing lists.  Since this is occurring in the Slack Channel, the
> conversations are not archived.
>
> In my opinion, this is not good for the Metron community.  Having this
> persisted in a discoverable form (like a mailing list archive) not only
> helps support current users, but also helps *potential* users understand
> how Metron is being used.
>
> Does anyone else agree or disagree?  At a minimum, I feel we need to do
> something to direct these conversations back to the mailing list.
>

Re: [DISCUSS] Slack Channel Use

[Nick Allen]

The help given on the Channel is great.  It is just much more scalable on
an archived mailing list.  Hence the Apache Foundation's preference for
mailing lists.




On Mon, Oct 22, 2018 at 12:27 PM Otto Fowler 
wrote:

> These questions also occurred on the IRC channel.  The difference is that
> there are more than Jon and I answering now.
>
>
> On October 22, 2018 at 12:18:08, Nick Allen (n...@nickallen.org) wrote:
>
> It seems that we are seeing a lot of Metron usage and support questions on
> the Slack Channel.
> These are questions that previously would have been directed to the User
> or
> Dev mailing lists. Since this is occurring in the Slack Channel, the
> conversations are not archived.
>
> In my opinion, this is not good for the Metron community. Having this
> persisted in a discoverable form (like a mailing list archive) not only
> helps support current users, but also helps *potential* users understand
> how Metron is being used.
>
> Does anyone else agree or disagree? At a minimum, I feel we need to do
> something to direct these conversations back to the mailing list.
>
>

Re: [DISCUSS] Slack Channel Use

[Otto Fowler]

These questions also occurred on the IRC channel.  The difference is that
there are more than Jon and I answering now.


On October 22, 2018 at 12:18:08, Nick Allen (n...@nickallen.org) wrote:

It seems that we are seeing a lot of Metron usage and support questions on
the Slack Channel.
These are questions that previously would have been directed to the User or
Dev mailing lists. Since this is occurring in the Slack Channel, the
conversations are not archived.

In my opinion, this is not good for the Metron community. Having this
persisted in a discoverable form (like a mailing list archive) not only
helps support current users, but also helps *potential* users understand
how Metron is being used.

Does anyone else agree or disagree? At a minimum, I feel we need to do
something to direct these conversations back to the mailing list.

[DISCUSS] Slack Channel Use

[Nick Allen]

It seems that we are seeing a lot of Metron usage and support questions on
the Slack Channel.
These are questions that previously would have been directed to the User or
Dev mailing lists.  Since this is occurring in the Slack Channel, the
conversations are not archived.

In my opinion, this is not good for the Metron community.  Having this
persisted in a discoverable form (like a mailing list archive) not only
helps support current users, but also helps *potential* users understand
how Metron is being used.

Does anyone else agree or disagree?  At a minimum, I feel we need to do
something to direct these conversations back to the mailing list.

Re: HCP in Cloud infrastructures such as AWS , GCP, AZURE

[deepak kumar]

Thanks Carolyn.
Is there any defined reference architecture to refer to?

Thanks
Deepak

On Mon, Oct 22, 2018 at 8:23 PM Carolyn Duby  wrote:

>
> Hive 3.0 works well with block stores.  You can either add it to your
> Metron cluster or spin up an ephemeral cluster with Cloudbreak:
>
> 1. Metron streams into HDFS in JSON.
> 2. Compact daily with Spark into ORC format and store in block store (S3,
> ADLS, etc).
> 3. Query ORC in block store using external Hive 3.0 tables in HDP 3 using
> LLAP.
> 4. If querying externally from block store is too slow, try adding more
> LLAP cache or load data into HDFS prior to analysis.
>
> If you are using the Metron Alerts UI, you will need solr which works well
> only on fast disk.   To keep costs down, reduce the context stored in Solr
> using the following techniques:
> 1. Only index the fields you might search on.
> 2. Reduce the formats you store in Solr to only those you will want to see
> in the Alerts UI.
> 3. Reduce the length of time you store data in Solr.
>
> Thanks
> Carolyn Duby
> Solutions Engineer, Northeast
> cd...@hortonworks.com
> +1.508.965.0584
>
> Join my team!
> Enterprise Account Manager – Boston - http://grnh.se/wepchv1
> Solutions Engineer – Boston - http://grnh.se/8gbxy41
> Need Answers? Try https://community.hortonworks.com <
> https://community.hortonworks.com/answers/index.html>
>
>
>
>
>
>
>
>
> On 10/19/18, 7:18 AM, "deepak kumar"  wrote:
>
> >Hi All
> >I have a quick question around HCP deployments in cloud infra such as AWS.
> >I am planning to run persistent cluster for all event streaming and
> >processing.
> >And then run transient cluster such as AWS EMR to run batch loads on the
> >data ingested from persistent cluster.
> >Have anyone tried this model ?
> >Since data volume is going to be humongous ,cloud is charging lot of money
> >for data io and storage.
> >Keeping this in mind , what could be the best cloud deployment of hcp
> >components assuming there is going to be ingest rate of 10TB per day .
> >
> >Thanks in advance.
> >
> >
> >Regards,
> >Deepak
>

Re: HCP in Cloud infrastructures such as AWS , GCP, AZURE

[Carolyn Duby]

Hive 3.0 works well with block stores.  You can either add it to your Metron 
cluster or spin up an ephemeral cluster with Cloudbreak:

1. Metron streams into HDFS in JSON.
2. Compact daily with Spark into ORC format and store in block store (S3, ADLS, 
etc).
3. Query ORC in block store using external Hive 3.0 tables in HDP 3 using LLAP.
4. If querying externally from block store is too slow, try adding more LLAP 
cache or load data into HDFS prior to analysis.

If you are using the Metron Alerts UI, you will need solr which works well only 
on fast disk.   To keep costs down, reduce the context stored in Solr using the 
following techniques:
1. Only index the fields you might search on.
2. Reduce the formats you store in Solr to only those you will want to see in 
the Alerts UI.
3. Reduce the length of time you store data in Solr.

Thanks
Carolyn Duby
Solutions Engineer, Northeast
cd...@hortonworks.com
+1.508.965.0584

Join my team!
Enterprise Account Manager – Boston - http://grnh.se/wepchv1
Solutions Engineer – Boston - http://grnh.se/8gbxy41
Need Answers? Try https://community.hortonworks.com 









On 10/19/18, 7:18 AM, "deepak kumar"  wrote:

>Hi All
>I have a quick question around HCP deployments in cloud infra such as AWS.
>I am planning to run persistent cluster for all event streaming and
>processing.
>And then run transient cluster such as AWS EMR to run batch loads on the
>data ingested from persistent cluster.
>Have anyone tried this model ?
>Since data volume is going to be humongous ,cloud is charging lot of money
>for data io and storage.
>Keeping this in mind , what could be the best cloud deployment of hcp
>components assuming there is going to be ingest rate of 10TB per day .
>
>Thanks in advance.
>
>
>Regards,
>Deepak

Re: Invite to Slack Channel

[zeo...@gmail.com]

Invite sent

On Mon, Oct 22, 2018 at 9:26 AM Muhammed Irshad  wrote:

> Some one get me also the slack channel link ?
> Thanks,
> Muhammed Irshad
> Q*Burst*
> www.qburst.com
>
>
> On Wed, Oct 17, 2018 at 7:33 PM Michael Miklavcic <
> michael.miklav...@gmail.com> wrote:
>
> > Sent
> >
> > On Wed, Oct 17, 2018 at 7:23 AM Tibor Meller 
> > wrote:
> >
> > > Hi Guys,
> > > Can you add me to the apache metron slack chanel?
> > >
> > > Thanks,
> > >
> > > On Thu, Oct 4, 2018 at 1:14 PM Otto Fowler 
> > > wrote:
> > >
> > > > Done
> > > >
> > > >
> > > > On October 4, 2018 at 05:35:06, Tamás Fodor (ftamas.m...@gmail.com)
> > > wrote:
> > > >
> > > > Hello,
> > > >
> > > > Michael, can you add me as well?
> > > >
> > > > Thank you in advance!
> > > >
> > > > Tamas
> > > >
> > > > On Wed, Oct 3, 2018 at 4:27 PM Michael Miklavcic <
> > > > michael.miklav...@gmail.com> wrote:
> > > >
> > > > > Sent
> > > > >
> > > > > On Wed, Oct 3, 2018 at 8:17 AM Shane Ardell <
> > shane.m.ard...@gmail.com>
> > > > > wrote:
> > > > >
> > > > > > Hello everyone,
> > > > > >
> > > > > > Is it possible for someone to send me an invite to the Metron
> Slack
> > > > > > channel?
> > > > > >
> > > > > > Regards,
> > > > > > Shane
> > > > > >
> > > > >
> > > >
> > >
> >
>
-- 

Jon Zeolla

Re: Invite to Slack Channel

[Muhammed Irshad]

Some one get me also the slack channel link ?
Thanks,
Muhammed Irshad
Q*Burst*
www.qburst.com


On Wed, Oct 17, 2018 at 7:33 PM Michael Miklavcic <
michael.miklav...@gmail.com> wrote:

> Sent
>
> On Wed, Oct 17, 2018 at 7:23 AM Tibor Meller 
> wrote:
>
> > Hi Guys,
> > Can you add me to the apache metron slack chanel?
> >
> > Thanks,
> >
> > On Thu, Oct 4, 2018 at 1:14 PM Otto Fowler 
> > wrote:
> >
> > > Done
> > >
> > >
> > > On October 4, 2018 at 05:35:06, Tamás Fodor (ftamas.m...@gmail.com)
> > wrote:
> > >
> > > Hello,
> > >
> > > Michael, can you add me as well?
> > >
> > > Thank you in advance!
> > >
> > > Tamas
> > >
> > > On Wed, Oct 3, 2018 at 4:27 PM Michael Miklavcic <
> > > michael.miklav...@gmail.com> wrote:
> > >
> > > > Sent
> > > >
> > > > On Wed, Oct 3, 2018 at 8:17 AM Shane Ardell <
> shane.m.ard...@gmail.com>
> > > > wrote:
> > > >
> > > > > Hello everyone,
> > > > >
> > > > > Is it possible for someone to send me an invite to the Metron Slack
> > > > > channel?
> > > > >
> > > > > Regards,
> > > > > Shane
> > > > >
> > > >
> > >
> >
>