David, how much RAM do you have in the test system where you run full dev?
On 2/25/17, 8:46 AM, "David Lyle" <[email protected]> wrote:
+1 (binding)
Checked signatures
Tests passed
Full dev ran sucessfully-ish (bit more below, tl;dr- it's ugly, but much
better after METRON-671)
Data flowed into the index/dashboard/hdfs
On full-dev: Monit got into a state where it started killing topologies
because the Storm cli was non-responsive due to memory pressure. That
caused cascading failures. I let it be for about 20 minutes, and it seemed
to calm down. I then killed pcap-service and Monit to make some headroom. I
haven't seen this in my work with METRON-671 as only sensor probes and
pcap-service are managed by Monit. Once the topologies are monitored by
Ambari, we no longer use the Storm CLI for status so that frees up some
additional memory.
-D...
On Sat, Feb 25, 2017 at 10:42 AM, David Lyle <[email protected]> wrote:
> I'm running it up now. I don't think it should affect the release, but
> it's extremely important that full-dev work. It's how quick-dev is
created.
> Any notion that it's been deprecated is incorrect.
>
> Fwiw, I've been working with it for the last few weeks as part of
> METRON-671 and it's been (knocking loudly on wood) reliable.
> I think we should leave full-dev in as the release verification step,
> particularly in the case that a release changes the Hadoop versions.
> There's a bit of a chicken-egg problem when we bump versions and quick dev
> will lag.
>
> I'll update with run results shortly.
>
> -D...
>
>
> On Sat, Feb 25, 2017 at 9:24 AM, Otto Fowler <[email protected]>
> wrote:
>
>> I see this with full dev on my laptop pretty much every time.
>> I think it is a resource issue. I see not enough memory errors trying to
>> start things.
>>
>>
>>
>> On February 25, 2017 at 09:15:15, Ryan Merriman ([email protected])
>> wrote:
>>
>> When I go to Ambari to ensure the services are all up, HDFS is down. I
>> tried it 4 or 5 times and got the same result each time. I've seen others
>> validate with quick dev so I assumed full dev was not used anymore. I'll
>> spin it up this morning and get a more detailed error.
>>
>> Is anyone else able to validate it in full dev?
>>
>> > On Feb 25, 2017, at 7:21 AM, Casey Stella <[email protected]> wrote:
>> >
>> > What exactly are the errors that you saw, Ryan?
>> >> On Sat, Feb 25, 2017 at 07:31 David Lyle <[email protected]> wrote:
>> >>
>> >> Is there any reason full dev shouldn't be working?
>> >>
>> >>> On Fri, Feb 24, 2017 at 9:19 PM, Casey Stella <[email protected]>
>> wrote:
>> >>>
>> >>> Sounds like a good idea to me; thanks Ryan!
>> >>>> On Fri, Feb 24, 2017 at 21:11 Ryan Merriman <[email protected]>
>> wrote:
>> >>>>
>> >>>> +1 binding
>> >>>>
>> >>>> Verified the signature
>> >>>> Passed maven tests
>> >>>> Started quick-dev, verified data in ES, kibana, and checked the
>> >>> topologies
>> >>>> for errors (bro topology has parsing errors but I think a couple bad
>> >>>> messages in bro data set is normal)
>> >>>> Tested REPL
>> >>>> RPMs built fine
>> >>>>
>> >>>> The recommended build validation wiki page (
>> https://cwiki.apache.org/
>> >>>> confluence/display/METRON/Verifying+Builds
>> >>>> <https://cwiki.apache.org/confluence/display/METRON/Verifying+Builds
>> >)
>> >>>> has some mistakes. This did
>> >>>> not run successfully in full-dev-platform and the HDFS paths look
>> like
>> >>> they
>> >>>> are old. I am happy to update the wiki page if everyone agrees these
>> >> are
>> >>>> legitimate mistakes.
>> >>>>
>> >>>> On Fri, Feb 24, 2017 at 9:22 AM, Justin Leet <[email protected]>
>> >>>> wrote:
>> >>>>
>> >>>>> +1 (non-binding)
>> >>>>>
>> >>>>> Verified signature
>> >>>>> Ran build and tests in maven
>> >>>>> Ran up in quick-dev and saw data flow through topologies into the
UI
>> >>>>> Ensured the REPL spun up and performed some basic tasks
>> >>>>> Built rpms
>> >>>>>
>> >>>>> Justin
>> >>>>>
>> >>>>>> On Thu, Feb 23, 2017 at 11:18 AM, Casey Stella <[email protected]
>> >
>> >>>>> wrote:
>> >>>>>
>> >>>>>> This is a call to vote on releasing Apache Metron 0.3.1-RC5
>> >>> incubating
>> >>>>>>
>> >>>>>> Full list of changes in this release:
>> >>>>>> https://dist.apache.org/repos/dist/dev/incubator/metron/0.3.
>> >>>>>> 1-RC5-incubating/CHANGES
>> >>>>>>
>> >>>>>> The tag/commit to be voted upon is apache-metron-0.3.1-rc5-
>> >>> incubating:
>> >>>>>> https://git-wip-us.apache.org/repos/asf?p=incubator-metron.
>> >>>>>> git;a=shortlog;h=refs/tags/apache-metron-0.3.1-rc5-incubating
>> >>>>>>
>> >>>>>> The source archive being voted upon can be found here:
>> >>>>>> https://dist.apache.org/repos/dist/dev/incubator/metron/0.3.
>> >>>>>> 1-RC5-incubating/apache-metron-0.3.1-rc5-incubating.tar.gz
>> >>>>>>
>> >>>>>> Other release files, signatures and digests can be found here:
>> >>>>>> https://dist.apache.org/repos/dist/dev/incubator/metron/0.3.
>> >>>>>> 1-RC5-incubating/
>> >>>>>>
>> >>>>>> The release artifacts are signed with the following key:
>> >>>>>> https://git-wip-us.apache.org/repos/asf?p=incubator-metron.
>> >>>>>> git;a=blob;f=KEYS;h=8381e96d64c249a0c1b489bc0c234d
>> >>>>> 9c260ba55e;hb=refs/tags/
>> >>>>>> apache-metron-0.3.1-rc5-incubating
>> >>>>>>
>> >>>>>> The book associated with this RC is located at
>> >>>>>> https://dist.apache.org/repos/dist/dev/incubator/metron/0.3.
>> >>>>>> 1-RC5-incubating/book-site/index.html
>> >>>>>>
>> >>>>>> Please vote on releasing this package as Apache Metron 0.3.1-RC5
>> >>>>> incubating
>> >>>>>>
>> >>>>>> When voting, please list the actions taken to verify the release.
>> >>>>>>
>> >>>>>> Recommended build validation and verification instructions are
>> >> posted
>> >>>>> here:
>> >> https://cwiki.apache.org/confluence/display/METRON/Verifying+Builds
>> >>>>>>
>> >>>>>>
>> >>>>>> This vote will be open for at least 72 hours.
>> >>>>>>
>> >>>>>> [ ] +1 Release this package as Apache Metron 0.3.1-RC5 incubating
>> >>>>>>
>> >>>>>> [ ] 0 No opinion
>> >>>>>>
>> >>>>>> [ ] -1 Do not release this package because...
>> >>
>>
>
>
