Re: metron-elasticsearch integration tests failing after merging in master
I usually create a JIRA when I see an intermittent failure like this. But I do not see a record of this particular issue before. https://jira.apache.org/jira/issues/?jql=text%20~%20%22Intermittent%22%20AND%20project%20%3D%20Metron%20AND%20resolution%20%3D%20Unresolved On Wed, Oct 24, 2018 at 12:35 PM Michael Miklavcic < michael.miklav...@gmail.com> wrote: > That's a new one on me. Does this run locally for you? > > Failed tests: > ElasticsearchUpdateIntegrationTest>UpdateIntegrationTest.test:132 Data > store is not updated!. Actual: 0 > > > > On Wed, Oct 24, 2018 at 10:10 AM Otto Fowler > wrote: > > > https://travis-ci.org/ottobackwards/metron/jobs/445723343 > > Anyone having ES test problems? Anyone shed any light on this. > > >
Re: metron-elasticsearch integration tests failing after merging in master
That's a new one on me. Does this run locally for you? Failed tests: ElasticsearchUpdateIntegrationTest>UpdateIntegrationTest.test:132 Data store is not updated!. Actual: 0 On Wed, Oct 24, 2018 at 10:10 AM Otto Fowler wrote: > https://travis-ci.org/ottobackwards/metron/jobs/445723343 > Anyone having ES test problems? Anyone shed any light on this. >
metron-elasticsearch integration tests failing after merging in master
https://travis-ci.org/ottobackwards/metron/jobs/445723343 Anyone having ES test problems? Anyone shed any light on this.
Re: [DISCUSS] Slack Channel Use
Yeah, I'm also surprised by that comment about the mailing list activity. Our dev/user list discussions are by far more active than they've ever been. Just have a look at the list of DISCUSS threads that have come up in the past few months and it's clear that not only participation has increased, but diversity of topic and participant. On Wed, Oct 24, 2018 at 8:08 AM Casey Stella wrote: > Not for nothing, but at least according to the last board report that I > submitted, the user@ traffic is up 100% and the dev list traffic is flat > as > compared to last quarter. That's not to say that we couldn't stand more > discussion on the lists, but a lot of the dev discussion happens on github > and JIRA and I'm happy to see an uptick in user traffic. > > On Wed, Oct 24, 2018 at 10:05 AM Otto Fowler > wrote: > > > I wouldn’t be so quick to related the slack discussion with perceived > > activity on the list. > > That is more do to the other things that are bigger issues. > > > > > > On October 24, 2018 at 07:15:30, Nick Allen (n...@nickallen.org) wrote: > > > > > I have heard recently people thought Metron is sort of dead just > because > > the mailing list is not so active anymore! > > > > That is exactly my concern. > > > > > > On Wed, Oct 24, 2018, 2:49 AM Ali Nazemian > wrote: > > > > > I kind of expect to have Slack for more dev related discussions rather > > than > > > user QA. I guess it is quite common to expect mailing list to be used > for > > > the purpose of knowledge sharing to make sure it will be accessible by > > > other users as well. Of course, it is a trade-off that most of the > other > > > Apache projects decided to accept the risk of keeping user related > > > discussions out of Slack/IRC. However, it sometimes happens to see the > > > mixture of questions coming to Slack. I have heard recently people > > thought > > > Metron is sort of dead just because the mailing list is not so active > > > anymore! > > > > > > Cheers, > > > Ali > > > > > > On Tue, Oct 23, 2018 at 8:23 AM Casey Stella > wrote: > > > > > > > Agreed, the benefit of the mailing list is that it’s searchable by > > > ponymail > > > > and the major search engines. > > > > On Mon, Oct 22, 2018 at 17:18 Nick Allen wrote: > > > > > > > > > I don't know that it is the same kind of searchable. Is it being > > > indexed > > > > > by the major search engines? I have never used a search engine and > > > > > uncovered the answer to my problem in a Slack archive. > > > > > > > > > > On Mon, Oct 22, 2018 at 5:05 PM Otto Fowler < > ottobackwa...@gmail.com > > > > > > > > wrote: > > > > > > > > > > > According to Greg Stein, an infra admin on the NiFi slack, the > ASF > > > > slack > > > > > > that metron is in IS the standard plan, not the free one and is > > > > > searchable > > > > > > past 10,000 messages. > > > > > > > > > > > > > > > > > > > > > > > > On October 22, 2018 at 15:35:51, Michael Miklavcic ( > > > > > > michael.miklav...@gmail.com) wrote: > > > > > > > > > > > > ...From an archival and broader reach point of view, I do think > > > there's > > > > > > something to be said about using the mailing list. It's also > easier > > > to > > > > > link > > > > > > to Q/A threads from the mailing list archives and do searches... > > > > > > > > > > > > > > > > > > > > > > > > > > > > > https://lists.apache.org/thread.html/1aa85bc13d41e04a1f85c3100c2b803abe35d79b54062bbeaab83ace@%3Cdev.metron.apache.org%3E > > > > > > > > > > > > How very Inception. > > > > > > > > > > > > > > > > > > On Mon, Oct 22, 2018 at 1:32 PM Michael Miklavcic < > > > > > > michael.miklav...@gmail.com> wrote: > > > > > > > > > > > > > I just want to point out that we currently have 32 members in > the > > > > > Metron > > > > > > > Slack channel which I personally think is a great sign. This is > > > good > > > > > from > > > > > > a > > > > > > > community perspective and helps foster interactive sessions > where > > > > > > required. > > > > > > > From an archival and broader reach point of view, I do think > > > there's > > > > > > > something to be said about using the mailing list. It's also > > easier > > > > to > > > > > > link > > > > > > > to Q/A threads from the mailing list archives and do searches. > As > > > > > such, I > > > > > > > would also go along with Nick's suggestion and urge members to > > > prefer > > > > > the > > > > > > > user/dev list where possible. > > > > > > > > > > > > > > On Mon, Oct 22, 2018 at 10:51 AM Justin Leet < > > > justinjl...@gmail.com> > > > > > > > wrote: > > > > > > > > > > > > > >> If we want to push more discussion to the dev list, my obvious > > > > follow > > > > > up > > > > > > >> question then is "What are we hoping to get out of > > Slack/irc/other > > > > > > >> interactive medium?". What discussion would we even want on > > there, > > > > if > > > > > we > > > > > > >> can't have decisions and don't want usage/support? > > > > > > >> > > > > > > >> On Mon, Oct 22, 2018 at 12:44 PM Casey Stella < >
Re: [DISCUSS] Slack Channel Use
quick clarification, I said "a lot of dev discussion happens on github and JIRA". I want to make sure I didn't mean to imply that larger decisions were being made outside of the appropriate place, the dev list. On Wed, Oct 24, 2018 at 10:08 AM Casey Stella wrote: > Not for nothing, but at least according to the last board report that I > submitted, the user@ traffic is up 100% and the dev list traffic is flat > as compared to last quarter. That's not to say that we couldn't stand more > discussion on the lists, but a lot of the dev discussion happens on github > and JIRA and I'm happy to see an uptick in user traffic. > > On Wed, Oct 24, 2018 at 10:05 AM Otto Fowler > wrote: > >> I wouldn’t be so quick to related the slack discussion with perceived >> activity on the list. >> That is more do to the other things that are bigger issues. >> >> >> On October 24, 2018 at 07:15:30, Nick Allen (n...@nickallen.org) wrote: >> >> > I have heard recently people thought Metron is sort of dead just because >> the mailing list is not so active anymore! >> >> That is exactly my concern. >> >> >> On Wed, Oct 24, 2018, 2:49 AM Ali Nazemian wrote: >> >> > I kind of expect to have Slack for more dev related discussions rather >> than >> > user QA. I guess it is quite common to expect mailing list to be used >> for >> > the purpose of knowledge sharing to make sure it will be accessible by >> > other users as well. Of course, it is a trade-off that most of the other >> > Apache projects decided to accept the risk of keeping user related >> > discussions out of Slack/IRC. However, it sometimes happens to see the >> > mixture of questions coming to Slack. I have heard recently people >> thought >> > Metron is sort of dead just because the mailing list is not so active >> > anymore! >> > >> > Cheers, >> > Ali >> > >> > On Tue, Oct 23, 2018 at 8:23 AM Casey Stella >> wrote: >> > >> > > Agreed, the benefit of the mailing list is that it’s searchable by >> > ponymail >> > > and the major search engines. >> > > On Mon, Oct 22, 2018 at 17:18 Nick Allen wrote: >> > > >> > > > I don't know that it is the same kind of searchable. Is it being >> > indexed >> > > > by the major search engines? I have never used a search engine and >> > > > uncovered the answer to my problem in a Slack archive. >> > > > >> > > > On Mon, Oct 22, 2018 at 5:05 PM Otto Fowler < >> ottobackwa...@gmail.com> >> > > > wrote: >> > > > >> > > > > According to Greg Stein, an infra admin on the NiFi slack, the ASF >> > > slack >> > > > > that metron is in IS the standard plan, not the free one and is >> > > > searchable >> > > > > past 10,000 messages. >> > > > > >> > > > > >> > > > > >> > > > > On October 22, 2018 at 15:35:51, Michael Miklavcic ( >> > > > > michael.miklav...@gmail.com) wrote: >> > > > > >> > > > > ...From an archival and broader reach point of view, I do think >> > there's >> > > > > something to be said about using the mailing list. It's also >> easier >> > to >> > > > link >> > > > > to Q/A threads from the mailing list archives and do searches... >> > > > > >> > > > > >> > > > >> > > >> > >> >> https://lists.apache.org/thread.html/1aa85bc13d41e04a1f85c3100c2b803abe35d79b54062bbeaab83ace@%3Cdev.metron.apache.org%3E >> > > > > >> > > > > How very Inception. >> > > > > >> > > > > >> > > > > On Mon, Oct 22, 2018 at 1:32 PM Michael Miklavcic < >> > > > > michael.miklav...@gmail.com> wrote: >> > > > > >> > > > > > I just want to point out that we currently have 32 members in >> the >> > > > Metron >> > > > > > Slack channel which I personally think is a great sign. This is >> > good >> > > > from >> > > > > a >> > > > > > community perspective and helps foster interactive sessions >> where >> > > > > required. >> > > > > > From an archival and broader reach point of view, I do think >> > there's >> > > > > > something to be said about using the mailing list. It's also >> easier >> > > to >> > > > > link >> > > > > > to Q/A threads from the mailing list archives and do searches. >> As >> > > > such, I >> > > > > > would also go along with Nick's suggestion and urge members to >> > prefer >> > > > the >> > > > > > user/dev list where possible. >> > > > > > >> > > > > > On Mon, Oct 22, 2018 at 10:51 AM Justin Leet < >> > justinjl...@gmail.com> >> > > > > > wrote: >> > > > > > >> > > > > >> If we want to push more discussion to the dev list, my obvious >> > > follow >> > > > up >> > > > > >> question then is "What are we hoping to get out of >> Slack/irc/other >> > > > > >> interactive medium?". What discussion would we even want on >> there, >> > > if >> > > > we >> > > > > >> can't have decisions and don't want usage/support? >> > > > > >> >> > > > > >> On Mon, Oct 22, 2018 at 12:44 PM Casey Stella < >> ceste...@gmail.com >> > > >> > > > > wrote: >> > > > > >> >> > > > > >> > I am of 2 minds, but I tend to agree. On the one hand, it's >> > > > definitely >> > > > > >> the >> > > > > >> > preference that we use the mailing lists for the reasons
Re: [DISCUSS] Slack Channel Use
Not for nothing, but at least according to the last board report that I submitted, the user@ traffic is up 100% and the dev list traffic is flat as compared to last quarter. That's not to say that we couldn't stand more discussion on the lists, but a lot of the dev discussion happens on github and JIRA and I'm happy to see an uptick in user traffic. On Wed, Oct 24, 2018 at 10:05 AM Otto Fowler wrote: > I wouldn’t be so quick to related the slack discussion with perceived > activity on the list. > That is more do to the other things that are bigger issues. > > > On October 24, 2018 at 07:15:30, Nick Allen (n...@nickallen.org) wrote: > > > I have heard recently people thought Metron is sort of dead just because > the mailing list is not so active anymore! > > That is exactly my concern. > > > On Wed, Oct 24, 2018, 2:49 AM Ali Nazemian wrote: > > > I kind of expect to have Slack for more dev related discussions rather > than > > user QA. I guess it is quite common to expect mailing list to be used for > > the purpose of knowledge sharing to make sure it will be accessible by > > other users as well. Of course, it is a trade-off that most of the other > > Apache projects decided to accept the risk of keeping user related > > discussions out of Slack/IRC. However, it sometimes happens to see the > > mixture of questions coming to Slack. I have heard recently people > thought > > Metron is sort of dead just because the mailing list is not so active > > anymore! > > > > Cheers, > > Ali > > > > On Tue, Oct 23, 2018 at 8:23 AM Casey Stella wrote: > > > > > Agreed, the benefit of the mailing list is that it’s searchable by > > ponymail > > > and the major search engines. > > > On Mon, Oct 22, 2018 at 17:18 Nick Allen wrote: > > > > > > > I don't know that it is the same kind of searchable. Is it being > > indexed > > > > by the major search engines? I have never used a search engine and > > > > uncovered the answer to my problem in a Slack archive. > > > > > > > > On Mon, Oct 22, 2018 at 5:05 PM Otto Fowler > > > > > wrote: > > > > > > > > > According to Greg Stein, an infra admin on the NiFi slack, the ASF > > > slack > > > > > that metron is in IS the standard plan, not the free one and is > > > > searchable > > > > > past 10,000 messages. > > > > > > > > > > > > > > > > > > > > On October 22, 2018 at 15:35:51, Michael Miklavcic ( > > > > > michael.miklav...@gmail.com) wrote: > > > > > > > > > > ...From an archival and broader reach point of view, I do think > > there's > > > > > something to be said about using the mailing list. It's also easier > > to > > > > link > > > > > to Q/A threads from the mailing list archives and do searches... > > > > > > > > > > > > > > > > > > > > > https://lists.apache.org/thread.html/1aa85bc13d41e04a1f85c3100c2b803abe35d79b54062bbeaab83ace@%3Cdev.metron.apache.org%3E > > > > > > > > > > How very Inception. > > > > > > > > > > > > > > > On Mon, Oct 22, 2018 at 1:32 PM Michael Miklavcic < > > > > > michael.miklav...@gmail.com> wrote: > > > > > > > > > > > I just want to point out that we currently have 32 members in the > > > > Metron > > > > > > Slack channel which I personally think is a great sign. This is > > good > > > > from > > > > > a > > > > > > community perspective and helps foster interactive sessions where > > > > > required. > > > > > > From an archival and broader reach point of view, I do think > > there's > > > > > > something to be said about using the mailing list. It's also > easier > > > to > > > > > link > > > > > > to Q/A threads from the mailing list archives and do searches. As > > > > such, I > > > > > > would also go along with Nick's suggestion and urge members to > > prefer > > > > the > > > > > > user/dev list where possible. > > > > > > > > > > > > On Mon, Oct 22, 2018 at 10:51 AM Justin Leet < > > justinjl...@gmail.com> > > > > > > wrote: > > > > > > > > > > > >> If we want to push more discussion to the dev list, my obvious > > > follow > > > > up > > > > > >> question then is "What are we hoping to get out of > Slack/irc/other > > > > > >> interactive medium?". What discussion would we even want on > there, > > > if > > > > we > > > > > >> can't have decisions and don't want usage/support? > > > > > >> > > > > > >> On Mon, Oct 22, 2018 at 12:44 PM Casey Stella < > ceste...@gmail.com > > > > > > > > wrote: > > > > > >> > > > > > >> > I am of 2 minds, but I tend to agree. On the one hand, it's > > > > definitely > > > > > >> the > > > > > >> > preference that we use the mailing lists for the reasons you > > > stated > > > > > (and > > > > > >> > also because not everyone has access to slack generally). On > the > > > > other > > > > > >> > hand, I think an interactive medium like Slack has a lot of > > > > advantages > > > > > >> in > > > > > >> > terms of user satisfaction. Ultimately, though, we may satisfy > 1 > > > > user > > > > > >> at > > > > > >> > the cost of not persisting the discussion and satisfying many > > > users. > > > > > >>
Re: [DISCUSS] Slack Channel Use
I wouldn’t be so quick to related the slack discussion with perceived activity on the list. That is more do to the other things that are bigger issues. On October 24, 2018 at 07:15:30, Nick Allen (n...@nickallen.org) wrote: > I have heard recently people thought Metron is sort of dead just because the mailing list is not so active anymore! That is exactly my concern. On Wed, Oct 24, 2018, 2:49 AM Ali Nazemian wrote: > I kind of expect to have Slack for more dev related discussions rather than > user QA. I guess it is quite common to expect mailing list to be used for > the purpose of knowledge sharing to make sure it will be accessible by > other users as well. Of course, it is a trade-off that most of the other > Apache projects decided to accept the risk of keeping user related > discussions out of Slack/IRC. However, it sometimes happens to see the > mixture of questions coming to Slack. I have heard recently people thought > Metron is sort of dead just because the mailing list is not so active > anymore! > > Cheers, > Ali > > On Tue, Oct 23, 2018 at 8:23 AM Casey Stella wrote: > > > Agreed, the benefit of the mailing list is that it’s searchable by > ponymail > > and the major search engines. > > On Mon, Oct 22, 2018 at 17:18 Nick Allen wrote: > > > > > I don't know that it is the same kind of searchable. Is it being > indexed > > > by the major search engines? I have never used a search engine and > > > uncovered the answer to my problem in a Slack archive. > > > > > > On Mon, Oct 22, 2018 at 5:05 PM Otto Fowler > > > wrote: > > > > > > > According to Greg Stein, an infra admin on the NiFi slack, the ASF > > slack > > > > that metron is in IS the standard plan, not the free one and is > > > searchable > > > > past 10,000 messages. > > > > > > > > > > > > > > > > On October 22, 2018 at 15:35:51, Michael Miklavcic ( > > > > michael.miklav...@gmail.com) wrote: > > > > > > > > ...From an archival and broader reach point of view, I do think > there's > > > > something to be said about using the mailing list. It's also easier > to > > > link > > > > to Q/A threads from the mailing list archives and do searches... > > > > > > > > > > > > > > https://lists.apache.org/thread.html/1aa85bc13d41e04a1f85c3100c2b803abe35d79b54062bbeaab83ace@%3Cdev.metron.apache.org%3E > > > > > > > > How very Inception. > > > > > > > > > > > > On Mon, Oct 22, 2018 at 1:32 PM Michael Miklavcic < > > > > michael.miklav...@gmail.com> wrote: > > > > > > > > > I just want to point out that we currently have 32 members in the > > > Metron > > > > > Slack channel which I personally think is a great sign. This is > good > > > from > > > > a > > > > > community perspective and helps foster interactive sessions where > > > > required. > > > > > From an archival and broader reach point of view, I do think > there's > > > > > something to be said about using the mailing list. It's also easier > > to > > > > link > > > > > to Q/A threads from the mailing list archives and do searches. As > > > such, I > > > > > would also go along with Nick's suggestion and urge members to > prefer > > > the > > > > > user/dev list where possible. > > > > > > > > > > On Mon, Oct 22, 2018 at 10:51 AM Justin Leet < > justinjl...@gmail.com> > > > > > wrote: > > > > > > > > > >> If we want to push more discussion to the dev list, my obvious > > follow > > > up > > > > >> question then is "What are we hoping to get out of Slack/irc/other > > > > >> interactive medium?". What discussion would we even want on there, > > if > > > we > > > > >> can't have decisions and don't want usage/support? > > > > >> > > > > >> On Mon, Oct 22, 2018 at 12:44 PM Casey Stella > > > > > wrote: > > > > >> > > > > >> > I am of 2 minds, but I tend to agree. On the one hand, it's > > > definitely > > > > >> the > > > > >> > preference that we use the mailing lists for the reasons you > > stated > > > > (and > > > > >> > also because not everyone has access to slack generally). On the > > > other > > > > >> > hand, I think an interactive medium like Slack has a lot of > > > advantages > > > > >> in > > > > >> > terms of user satisfaction. Ultimately, though, we may satisfy 1 > > > user > > > > >> at > > > > >> > the cost of not persisting the discussion and satisfying many > > users. > > > > >> > > > > > >> > I'll go along with a specific preference to drive more > discussion > > to > > > > the > > > > >> > mailing list. > > > > >> > > > > > >> > Casey > > > > >> > > > > > >> > On Mon, Oct 22, 2018 at 12:18 PM Nick Allen > > > > > wrote: > > > > >> > > > > > >> > > It seems that we are seeing a lot of Metron usage and support > > > > >> questions > > > > >> > on > > > > >> > > the Slack Channel. > > > > >> > > These are questions that previously would have been directed > to > > > the > > > > >> User > > > > >> > or > > > > >> > > Dev mailing lists. Since this is occurring in the Slack > Channel, > > > the > > > > >> > > conversations are not archived. > > > > >> > > > > > >
Re: [DISCUSS] Slack Channel Use
> I have heard recently people thought Metron is sort of dead just because the mailing list is not so active anymore! That is exactly my concern. On Wed, Oct 24, 2018, 2:49 AM Ali Nazemian wrote: > I kind of expect to have Slack for more dev related discussions rather than > user QA. I guess it is quite common to expect mailing list to be used for > the purpose of knowledge sharing to make sure it will be accessible by > other users as well. Of course, it is a trade-off that most of the other > Apache projects decided to accept the risk of keeping user related > discussions out of Slack/IRC. However, it sometimes happens to see the > mixture of questions coming to Slack. I have heard recently people thought > Metron is sort of dead just because the mailing list is not so active > anymore! > > Cheers, > Ali > > On Tue, Oct 23, 2018 at 8:23 AM Casey Stella wrote: > > > Agreed, the benefit of the mailing list is that it’s searchable by > ponymail > > and the major search engines. > > On Mon, Oct 22, 2018 at 17:18 Nick Allen wrote: > > > > > I don't know that it is the same kind of searchable. Is it being > indexed > > > by the major search engines? I have never used a search engine and > > > uncovered the answer to my problem in a Slack archive. > > > > > > On Mon, Oct 22, 2018 at 5:05 PM Otto Fowler > > > wrote: > > > > > > > According to Greg Stein, an infra admin on the NiFi slack, the ASF > > slack > > > > that metron is in IS the standard plan, not the free one and is > > > searchable > > > > past 10,000 messages. > > > > > > > > > > > > > > > > On October 22, 2018 at 15:35:51, Michael Miklavcic ( > > > > michael.miklav...@gmail.com) wrote: > > > > > > > > ...From an archival and broader reach point of view, I do think > there's > > > > something to be said about using the mailing list. It's also easier > to > > > link > > > > to Q/A threads from the mailing list archives and do searches... > > > > > > > > > > > > > > https://lists.apache.org/thread.html/1aa85bc13d41e04a1f85c3100c2b803abe35d79b54062bbeaab83ace@%3Cdev.metron.apache.org%3E > > > > > > > > How very Inception. > > > > > > > > > > > > On Mon, Oct 22, 2018 at 1:32 PM Michael Miklavcic < > > > > michael.miklav...@gmail.com> wrote: > > > > > > > > > I just want to point out that we currently have 32 members in the > > > Metron > > > > > Slack channel which I personally think is a great sign. This is > good > > > from > > > > a > > > > > community perspective and helps foster interactive sessions where > > > > required. > > > > > From an archival and broader reach point of view, I do think > there's > > > > > something to be said about using the mailing list. It's also easier > > to > > > > link > > > > > to Q/A threads from the mailing list archives and do searches. As > > > such, I > > > > > would also go along with Nick's suggestion and urge members to > prefer > > > the > > > > > user/dev list where possible. > > > > > > > > > > On Mon, Oct 22, 2018 at 10:51 AM Justin Leet < > justinjl...@gmail.com> > > > > > wrote: > > > > > > > > > >> If we want to push more discussion to the dev list, my obvious > > follow > > > up > > > > >> question then is "What are we hoping to get out of Slack/irc/other > > > > >> interactive medium?". What discussion would we even want on there, > > if > > > we > > > > >> can't have decisions and don't want usage/support? > > > > >> > > > > >> On Mon, Oct 22, 2018 at 12:44 PM Casey Stella > > > > > wrote: > > > > >> > > > > >> > I am of 2 minds, but I tend to agree. On the one hand, it's > > > definitely > > > > >> the > > > > >> > preference that we use the mailing lists for the reasons you > > stated > > > > (and > > > > >> > also because not everyone has access to slack generally). On the > > > other > > > > >> > hand, I think an interactive medium like Slack has a lot of > > > advantages > > > > >> in > > > > >> > terms of user satisfaction. Ultimately, though, we may satisfy 1 > > > user > > > > >> at > > > > >> > the cost of not persisting the discussion and satisfying many > > users. > > > > >> > > > > > >> > I'll go along with a specific preference to drive more > discussion > > to > > > > the > > > > >> > mailing list. > > > > >> > > > > > >> > Casey > > > > >> > > > > > >> > On Mon, Oct 22, 2018 at 12:18 PM Nick Allen > > > > > wrote: > > > > >> > > > > > >> > > It seems that we are seeing a lot of Metron usage and support > > > > >> questions > > > > >> > on > > > > >> > > the Slack Channel. > > > > >> > > These are questions that previously would have been directed > to > > > the > > > > >> User > > > > >> > or > > > > >> > > Dev mailing lists. Since this is occurring in the Slack > Channel, > > > the > > > > >> > > conversations are not archived. > > > > >> > > > > > > >> > > In my opinion, this is not good for the Metron community. > Having > > > > this > > > > >> > > persisted in a discoverable form (like a mailing list archive) > > not > > > > >> only > > > > >> > > helps support current
Re: [DISCUSS] Slack Channel Use
I kind of expect to have Slack for more dev related discussions rather than user QA. I guess it is quite common to expect mailing list to be used for the purpose of knowledge sharing to make sure it will be accessible by other users as well. Of course, it is a trade-off that most of the other Apache projects decided to accept the risk of keeping user related discussions out of Slack/IRC. However, it sometimes happens to see the mixture of questions coming to Slack. I have heard recently people thought Metron is sort of dead just because the mailing list is not so active anymore! Cheers, Ali On Tue, Oct 23, 2018 at 8:23 AM Casey Stella wrote: > Agreed, the benefit of the mailing list is that it’s searchable by ponymail > and the major search engines. > On Mon, Oct 22, 2018 at 17:18 Nick Allen wrote: > > > I don't know that it is the same kind of searchable. Is it being indexed > > by the major search engines? I have never used a search engine and > > uncovered the answer to my problem in a Slack archive. > > > > On Mon, Oct 22, 2018 at 5:05 PM Otto Fowler > > wrote: > > > > > According to Greg Stein, an infra admin on the NiFi slack, the ASF > slack > > > that metron is in IS the standard plan, not the free one and is > > searchable > > > past 10,000 messages. > > > > > > > > > > > > On October 22, 2018 at 15:35:51, Michael Miklavcic ( > > > michael.miklav...@gmail.com) wrote: > > > > > > ...From an archival and broader reach point of view, I do think there's > > > something to be said about using the mailing list. It's also easier to > > link > > > to Q/A threads from the mailing list archives and do searches... > > > > > > > > > https://lists.apache.org/thread.html/1aa85bc13d41e04a1f85c3100c2b803abe35d79b54062bbeaab83ace@%3Cdev.metron.apache.org%3E > > > > > > How very Inception. > > > > > > > > > On Mon, Oct 22, 2018 at 1:32 PM Michael Miklavcic < > > > michael.miklav...@gmail.com> wrote: > > > > > > > I just want to point out that we currently have 32 members in the > > Metron > > > > Slack channel which I personally think is a great sign. This is good > > from > > > a > > > > community perspective and helps foster interactive sessions where > > > required. > > > > From an archival and broader reach point of view, I do think there's > > > > something to be said about using the mailing list. It's also easier > to > > > link > > > > to Q/A threads from the mailing list archives and do searches. As > > such, I > > > > would also go along with Nick's suggestion and urge members to prefer > > the > > > > user/dev list where possible. > > > > > > > > On Mon, Oct 22, 2018 at 10:51 AM Justin Leet > > > > wrote: > > > > > > > >> If we want to push more discussion to the dev list, my obvious > follow > > up > > > >> question then is "What are we hoping to get out of Slack/irc/other > > > >> interactive medium?". What discussion would we even want on there, > if > > we > > > >> can't have decisions and don't want usage/support? > > > >> > > > >> On Mon, Oct 22, 2018 at 12:44 PM Casey Stella > > > wrote: > > > >> > > > >> > I am of 2 minds, but I tend to agree. On the one hand, it's > > definitely > > > >> the > > > >> > preference that we use the mailing lists for the reasons you > stated > > > (and > > > >> > also because not everyone has access to slack generally). On the > > other > > > >> > hand, I think an interactive medium like Slack has a lot of > > advantages > > > >> in > > > >> > terms of user satisfaction. Ultimately, though, we may satisfy 1 > > user > > > >> at > > > >> > the cost of not persisting the discussion and satisfying many > users. > > > >> > > > > >> > I'll go along with a specific preference to drive more discussion > to > > > the > > > >> > mailing list. > > > >> > > > > >> > Casey > > > >> > > > > >> > On Mon, Oct 22, 2018 at 12:18 PM Nick Allen > > > wrote: > > > >> > > > > >> > > It seems that we are seeing a lot of Metron usage and support > > > >> questions > > > >> > on > > > >> > > the Slack Channel. > > > >> > > These are questions that previously would have been directed to > > the > > > >> User > > > >> > or > > > >> > > Dev mailing lists. Since this is occurring in the Slack Channel, > > the > > > >> > > conversations are not archived. > > > >> > > > > > >> > > In my opinion, this is not good for the Metron community. Having > > > this > > > >> > > persisted in a discoverable form (like a mailing list archive) > not > > > >> only > > > >> > > helps support current users, but also helps *potential* users > > > >> understand > > > >> > > how Metron is being used. > > > >> > > > > > >> > > Does anyone else agree or disagree? At a minimum, I feel we need > > to > > > >> do > > > >> > > something to direct these conversations back to the mailing > list. > > > >> > > > > > >> > > > > >> > > > > > > > > > > -- A.Nazemian
Re: HCP in Cloud infrastructures such as AWS , GCP, AZURE
Depending on the model of security, you may have some challenges with the Ranger integration with your cloud storage especially if you are thinking of using TDE for the encryption at rest. Otherwise, using Metron in that way should be quite feasible. However, you may face some performance issues depending on what the required SLA is, but the cost saving will most probably convince you to go with the decoupling storage from compute. Cheers, Ali On Tue, Oct 23, 2018 at 2:57 AM deepak kumar wrote: > Thanks Carolyn. > Is there any defined reference architecture to refer to? > > Thanks > Deepak > > On Mon, Oct 22, 2018 at 8:23 PM Carolyn Duby > wrote: > > > > > Hive 3.0 works well with block stores. You can either add it to your > > Metron cluster or spin up an ephemeral cluster with Cloudbreak: > > > > 1. Metron streams into HDFS in JSON. > > 2. Compact daily with Spark into ORC format and store in block store (S3, > > ADLS, etc). > > 3. Query ORC in block store using external Hive 3.0 tables in HDP 3 using > > LLAP. > > 4. If querying externally from block store is too slow, try adding more > > LLAP cache or load data into HDFS prior to analysis. > > > > If you are using the Metron Alerts UI, you will need solr which works > well > > only on fast disk. To keep costs down, reduce the context stored in > Solr > > using the following techniques: > > 1. Only index the fields you might search on. > > 2. Reduce the formats you store in Solr to only those you will want to > see > > in the Alerts UI. > > 3. Reduce the length of time you store data in Solr. > > > > Thanks > > Carolyn Duby > > Solutions Engineer, Northeast > > cd...@hortonworks.com > > +1.508.965.0584 > > > > Join my team! > > Enterprise Account Manager – Boston - http://grnh.se/wepchv1 > > Solutions Engineer – Boston - http://grnh.se/8gbxy41 > > Need Answers? Try https://community.hortonworks.com < > > https://community.hortonworks.com/answers/index.html> > > > > > > > > > > > > > > > > > > On 10/19/18, 7:18 AM, "deepak kumar" wrote: > > > > >Hi All > > >I have a quick question around HCP deployments in cloud infra such as > AWS. > > >I am planning to run persistent cluster for all event streaming and > > >processing. > > >And then run transient cluster such as AWS EMR to run batch loads on the > > >data ingested from persistent cluster. > > >Have anyone tried this model ? > > >Since data volume is going to be humongous ,cloud is charging lot of > money > > >for data io and storage. > > >Keeping this in mind , what could be the best cloud deployment of hcp > > >components assuming there is going to be ingest rate of 10TB per day . > > > > > >Thanks in advance. > > > > > > > > >Regards, > > >Deepak > > > -- A.Nazemian
