Following up on this thread - Envelope 0.6.0 is available, now using all
upstream Apache dependencies (rather than CDH services)
At a minimum, for what we would use it for, it requires:
Apache Spark 2.1.0 or above
Apache Kafka 0.10 or above
Thanks
Curtis
On Wed, May 9, 2018 at 4:35 PM, Curtis Ho
Hi all,
As a follow up to this thread, I've confirmed with the Envelope team that
the next release (0.6.0, ETA later this summer) will move to using upstream
dependencies rather than Cloudera's (for Spark, Kafka, HBase, etc.).
Envelope will also begin taking public code contributions soon - likely
Curtis,
Excited to take a look as well :). Thanks for the hard work on this.
Thank you,
Tadd Wood
> On May 2, 2018, at 4:45 AM, Austin Leahy wrote:
>
> Curtis this is very cool thanks for putting so much time into this will
> check out the PR and comment.
>
> On Tue, May 1, 2018 at 3:37 PM
Curtis this is very cool thanks for putting so much time into this will
check out the PR and comment.
On Tue, May 1, 2018 at 3:37 PM Curtis Howard wrote:
> Hi Nathanael,
>
> So far only https://github.com/Open-Network-Insight/spot-nfdump.git
>
> The PR code is a proof-of-concept at this point -
Hi Nathanael,
So far only https://github.com/Open-Network-Insight/spot-nfdump.git
The PR code is a proof-of-concept at this point - look forward to your
thoughts on next steps though!
Thanks again
Curtis
On Tue, May 1, 2018 at 6:28 PM, Nate Smith wrote:
> Curtis,
>
> Have you tested this with
Curtis,
Have you tested this with a standard version of nfdump? Or only spot-nfdump?
- Nathanael
> On May 1, 2018, at 1:12 PM, Curtis Howard wrote:
>
> Hi all,
>
> We had discussed prototyping Envelope for ingest in the past - I've
> submitted a PR for this which includes:
> - Kafka -> Spar
Thank you for all the hard work Curtis,
I will start reviewing.
- Nathanael
> On May 1, 2018, at 1:12 PM, Curtis Howard wrote:
>
> Hi all,
>
> We had discussed prototyping Envelope for ingest in the past - I've
> submitted a PR for this which includes:
> - Kafka -> Spark streaming -> ODM Hive
Hi all,
We had discussed prototyping Envelope for ingest in the past - I've
submitted a PR for this which includes:
- Kafka -> Spark streaming -> ODM Hive table applications for dns, flow
and proxy raw source data
- a simple alternative for source data collection/dissection using
tshark/nfdump
