The feature branch has been merged into master. On Thu, Aug 16, 2018 at 5:53 PM, Michael Miklavcic < michael.miklav...@gmail.com> wrote:
> I'm +1, thanks for adding that fix, Ryan. (Note, for purposes of vote, I > was a contributor in the feature branch). > > Mike > > On Thu, Aug 16, 2018, 4:17 PM Ryan Merriman <merrim...@gmail.com> wrote: > > > We discovered a bug in our testing and felt it should be fixed before we > > merge. There is a PR up for review that already has a +1: > > https://github.com/apache/metron/pull/1168. I don't anticipate this > > changing anyone's vote but wanted to be clear about the state of the > > branch. If anyone is concerned with this and would like more discussion > > before we merge, let me know. > > > > On Thu, Aug 16, 2018 at 8:25 AM, James Sirota <jsir...@apache.org> > wrote: > > > > > +1 on the merge as well > > > > > > 16.08.2018, 05:46, "Casey Stella" <ceste...@gmail.com>: > > > > I'm +1 on the merge. This is great work and congrats to those who > > > > contributed to it! > > > > > > > > On Thu, Aug 16, 2018 at 8:27 AM Otto Fowler <ottobackwa...@gmail.com > > > > > wrote: > > > > > > > >> Looks good, thanks! > > > >> > > > >> On August 15, 2018 at 19:38:12, Ryan Merriman (merrim...@gmail.com > ) > > > wrote: > > > >> > > > >> Otto, I believe the items you requested are in the feature branch > > now. > > > Is > > > >> there anything outstanding that we missed? The Jiras for the Pcap > > > feature > > > >> branch should be up to date: > > > >> https://issues.apache.org/jira/browse/METRON-1554 > > > >> > > > >> On Mon, Aug 13, 2018 at 5:13 PM, Ryan Merriman < > merrim...@gmail.com> > > > >> wrote: > > > >> > > > >> > - Date range limits on queries > > > >> > > > > >> > I will add a warning in the Job cleanup PR. That seems like an > > > >> > appropriate place for it (ie. make sure you don't cause health > > > issues in > > > >> > your cluster). > > > >> > > > > >> > - UI should manage a queue/history of jobs > > > >> > > > > >> > I can add some documentation around killing jobs manually with > the > > > YARN > > > >> > CLI. However if they haven't set up a YARN queue, I'm not sure > how > > > you > > > >> > would view only Pcap jobs. I'm also not sure how you would get > the > > > >> > application id for the job to kill because it's not displayed > > > anywhere in > > > >> > the UI. However, I believe we are wired for a job name but REST > > > doesn't > > > >> > set this. Maybe we could get a proper job name associated with > pcap > > > >> > queries and then this would be possible to document? > > > >> > > > > >> > - Documentation/blueprint for YARN configuration > > > >> > > > > >> > You make a good point. A YARN tuning guide for Metron does sound > > > useful. > > > >> > I will add a follow on Jira. > > > >> > > > > >> > On Mon, Aug 13, 2018 at 4:53 PM, Otto Fowler < > > > ottobackwa...@gmail.com> > > > >> > wrote: > > > >> > > > > >> >> > > > >> >> - Date range limits on queries > > > >> >> > > > >> >> I took the point the wrong way apparently, sorry, I withdraw. I > > > thought > > > >> >> you meant allow specifying a limit on the query, not the system > > > imposing > > > >> a > > > >> >> limit. > > > >> >> This should be documented with a warning or something > > > >> >> > > > >> >> - UI should manage a queue/history of jobs > > > >> >> > > > >> >> I was thinking that if there where multiple users/jobs, there > > should > > > >> >> be some thought or documentation + script on how to manage them. > > > >> >> “To see all the jobs still running on your cluster, across users > > > and ui > > > >> >> instances do XXXXX” > > > >> >> “If there is an issue with the jobs you can’t resolve in the UI > > for > > > that > > > >> >> user, or you are an admin and want to do something then XXXXX" > > > >> >> > > > >> >> - Documentation/blueprint for YARN configuration > > > >> >> > > > >> >> I agree with what you are saying. Although, we offer guidance on > > > storm > > > >> >> tuning, and that is conceptually the same isn’t it? That is why > it > > > comes > > > >> >> to mind. > > > >> >> Maybe this can be a follow on, in the tuning guide? > > > >> >> > > > >> >> On August 13, 2018 at 17:36:41, Ryan Merriman ( > > merrim...@gmail.com) > > > >> >> wrote: > > > >> >> > > > >> >> - Date range limits on queries > > > >> >> > > > >> >> Can you describe what you think is needed here? Each Metron user > > > could > > > >> >> have different volumes of pcap data spread out over different > time > > > >> >> periods. Are you saying we should limit the data range to > > something > > > >> either > > > >> >> > > > >> >> constant or configurable? Are we sure all users would want this? > > Am > > > I > > > >> >> misinterpreting this requirement? > > > >> >> > > > >> >> - UI should manage a queue/history of jobs > > > >> >> > > > >> >> What should we document here? Reading that bullet point again, > > it's > > > sort > > > >> >> of vague and not very description. What I am referring to is a > > > design > > > >> that > > > >> >> > > > >> >> provides users a way to view and manage jobs in the UI. > Currently > > > jobs > > > >> can > > > >> >> > > > >> >> only be run 1 at a time and progress is shown with a status bar, > > so > > > it's > > > >> >> somewhat interactive. > > > >> >> > > > >> >> - Documentation/blueprint for YARN configuration > > > >> >> > > > >> >> > > > >> > > > > > > > ------------------- > > > Thank you, > > > > > > James Sirota > > > PMC- Apache Metron > > > jsirota AT apache DOT org > > > > > > > > >
