Github user anandsubbu commented on the issue:
https://github.com/apache/metron/pull/907
Hi @cestella , I did a 12-node deploy on CentOS 7 with this patch.
Post-kerberization, I noticed the following errors in Metron REST. Is this a
related issue or a different one?
```
1
Github user ottobackwards commented on the issue:
https://github.com/apache/metron/pull/902
I have another script I want to add for tracking master in feature branches
after this as well
---
Github user cestella commented on the issue:
https://github.com/apache/metron/pull/906
+1
---
Github user asfgit closed the pull request at:
https://github.com/apache/metron/pull/906
---
Github user merrimanr commented on a diff in the pull request:
https://github.com/apache/metron/pull/909#discussion_r163859462
--- Diff:
metron-platform/metron-indexing/src/test/java/org/apache/metron/indexing/dao/SearchIntegrationTest.java
---
@@ -724,6 +503,52 @@ public void sor
Github user merrimanr commented on a diff in the pull request:
https://github.com/apache/metron/pull/909#discussion_r163859481
--- Diff:
metron-platform/metron-indexing/src/test/java/org/apache/metron/indexing/dao/SearchIntegrationTest.java
---
@@ -724,6 +503,52 @@ public void sor
Github user cestella commented on the issue:
https://github.com/apache/metron/pull/907
@anandsubbu Yeah, That's that lingering craziness where sometimes Kafka is
expecting `PLAINTEXTSASL` and sometimes `SASL_PLAINTEXT`. I just went ahead
and normalized it to whatever the kafka librar
Github user cestella commented on the issue:
https://github.com/apache/metron/pull/909
Should this be against master or should this be committed against the Solr
branch? It *seems* like this is general purpose goodness and maybe fits in
master, but I wanted to double check.
---
Github user mmiklavc commented on the issue:
https://github.com/apache/metron/pull/909
@cestella This is going into master per the merge notes, right?
---
Github user cestella commented on the issue:
https://github.com/apache/metron/pull/909
@mmiklavc I just wanted to make sure it was submitted against the right
branch. I am totally ok with that given the content of the PR, but the
question is whether there's anything solr specific her
Github user merrimanr commented on the issue:
https://github.com/apache/metron/pull/909
I think this is useful outside of any Solr work and I intended for it to go
into master.
---
Github user cestella commented on the issue:
https://github.com/apache/metron/pull/909
Sounds good, +1 Good work here, that test was confusing.
---
Github user mmiklavc commented on the issue:
https://github.com/apache/metron/pull/909
+1 pending travis
---
Github user mmiklavc commented on the issue:
https://github.com/apache/metron/pull/909
@cestella You're right about that - the general goodness. We (I) had made
the mistake of doing some test refactoring in the ES upgrade branch that would
have been better done in a separate PR, commi
Github user cestella closed the pull request at:
https://github.com/apache/metron/pull/907
---
GitHub user cestella reopened a pull request:
https://github.com/apache/metron/pull/907
METRON-1427: Add support for storm 1.1 and hdp 2.6
## Contributor Comments
Right now our ambari mpack won't run cleanly on HDP 2.6 and Storm 1.1
because of some classpath issues and not suppo
Github user ottobackwards commented on a diff in the pull request:
https://github.com/apache/metron/pull/901#discussion_r163868868
--- Diff:
metron-deployment/packaging/ambari/metron-mpack/src/main/resources/common-services/METRON/CURRENT/package/scripts/enrichment_commands.py
---
Github user ottobackwards commented on the issue:
https://github.com/apache/metron/pull/903
Trying this now.
Only comment on the content here is there is a _lot_ going on in this pr.
A lot of while I'm here I might as well work.
It might have been better to have kept this m
While it would be preferred if all data streamed into the parsers is
already in ‘stream’ form, as opposed to ‘batched’ form, it may not always
be possible, or possible at every step of system development.
I was wondering if it would be worth adding optional support to the JSONMap
Parser to support
Github user ottobackwards commented on the issue:
https://github.com/apache/metron/pull/903
Failure during vagrant up for metron-on-ubuntu
```
2018-01-25 10:41:49,302 p=37541 u=ottofowler | [0;31mfatal: [node1]:
FAILED! => {"changed": true, "cmd": "dpkg-scanpackages
Github user asfgit closed the pull request at:
https://github.com/apache/metron/pull/909
---
Hi Otto,
Oddly, I had reason a couple weeks ago to try to figure out a streaming parser
for very large json objects -- altho it was in Python rather than Java.
Search showed two basic approaches, both unsurprisingly modeled on xml
processing:
- SAX-like parsing
- XPath-like parsing
Both are capa
JSONPath is indeed what nifi uses. I used their implementation as a guide.
I believe starting with a path would be a good minimum viable, a good start.
We could support multiple paths of course.
Beside the fact that I knew NiFi used this approach, I believe that
JSONPath provides a flexible mecha
In other words, I don’t believe the issue is parsing, but rather searching
and extracting.
I have used SAX with xml as well, can you point me to the json equivalent
you found?
On January 25, 2018 at 13:01:58, Otto Fowler (ottobackwa...@gmail.com)
wrote:
JSONPath is indeed what nifi uses. I use
Github user nickwallen commented on the issue:
https://github.com/apache/metron/pull/903
> @ottobackwards : Failure during vagrant up for metron-on-ubuntu
Thanks, Otto. Yep, I messed that up. I pushed the fix, but I am going to
run through full CentOS and Ubuntu deployments j
I would like to propose a Metron user community meeting. I propose that we
set the meeting next week, and will throw out Wednesday, January 31st at
09:30AM PST, 12:30 on the East Coast and 5:30 in London Towne. This meeting
will be held over a web-ex, the details of which will be included in the
ac
GitHub user cestella opened a pull request:
https://github.com/apache/metron/pull/910
METRON-1430: Isolate jackson from being used as arguments or returns from
JSONUtils
## Contributor Comments
Currently jackson is used as part of our internal API to JSONUtils. The
problem her
Github user ottobackwards commented on the issue:
https://github.com/apache/metron/pull/903
@nickwallen let me know when you feel ok about it, I'll run it through
again.
---
Github user ottobackwards commented on the issue:
https://github.com/apache/metron/pull/910
This looks awesome at first look. I'm a big fan of doing this regardless
of the shading issue. The only thought that comes to mind is that there is a
tipping point where a *Utils class isn't
Heh, as I said, I was looking in Python. For SAX-like JSON parsers I found
numerous libraries, most built on top of an underlying Python library named
ijson, which is itself based on a C library called yajl.
The yajl page (http://lloyd.github.io/yajl/ ) lists a double handful of
language bindi
Sure it helps, but I am not sure I answered __your__ questions?
As I mentioned, we already use
Map rawMap = JSONUtils.INSTANCE.load(originalString, new
TypeReference>() {
});
So, using JSONPath which is using the same object mapper operation under
the covers is not a change.
We were already rea
Oh, my understanding was that you were proposing to read the inbound json as a
stream, potentially before the generation of that json was finished, thereby
saving memory (well, allowing it to be recycled sooner and in smaller pieces),
and decreasing latency.
And I thought JsonPath was capable o
Github user mmiklavc commented on the issue:
https://github.com/apache/metron/pull/910
@ottobackwards I was envisioning a metron-deps project of some sort.
Jackson and Guava are perennial offenders in the Hadoop stack, and a facade of
some sort would be useful to insulate us from the
Github user nickwallen commented on the issue:
https://github.com/apache/metron/pull/901
+1 Looks good @ottobackwards. Thanks for fixing this!
I have not tested this myself, but it looks solid. Let me know if you'd
prefer me to spin this up to get a second test run in. Othe
Github user ottobackwards commented on the issue:
https://github.com/apache/metron/pull/910
@mmiklavc I think that is a great idea for the project. And the
appropriately factored facade classes as well.
---
Github user nickwallen commented on the issue:
https://github.com/apache/metron/pull/902
I merged with master, so I expect Travis to be happy now. Just need +1s
and I'll get this in to allow for any follow-ons.
---
Github user cestella commented on the issue:
https://github.com/apache/metron/pull/910
Yeah, this is the first step to moving it out, I'd say. This was useful
independent of #907 because it made some things a lot cleaner (namely fewer
`new TypeReference>() {}` and more
`JSONUtils.MA
Github user ottobackwards commented on the issue:
https://github.com/apache/metron/pull/902
+1
---
Github user mmiklavc commented on a diff in the pull request:
https://github.com/apache/metron/pull/910#discussion_r163962696
--- Diff:
metron-platform/metron-indexing/src/main/java/org/apache/metron/indexing/dao/IndexDao.java
---
@@ -140,9 +147,9 @@ default Document getPatchedDoc
Github user cestella commented on a diff in the pull request:
https://github.com/apache/metron/pull/910#discussion_r163967410
--- Diff:
metron-platform/metron-indexing/src/main/java/org/apache/metron/indexing/dao/IndexDao.java
---
@@ -140,9 +147,9 @@ default Document getPatchedDoc
Github user nickwallen commented on the issue:
https://github.com/apache/metron/pull/903
I spun this up again on both Ubuntu and CentOS. Both worked successfully.
I am happy with it now @ottobackwards . Give her another go when you can.
Thanks.
---
Github user ottobackwards commented on the issue:
https://github.com/apache/metron/pull/903
Sure.
Question. Do we expect there to be issues with 2.6? Is this PR and
Casey's 2.6 pr going to conflict or have issues? How will we know to retest
this after that one lands if this l
Github user nickwallen commented on the issue:
https://github.com/apache/metron/pull/903
> @ottobackwards : Do we expect there to be issues with 2.6? Is this PR and
Casey's 2.6 pr going to conflict or have issues?
Yes, we will need to retest one or the other. I am open to hel
Github user ottobackwards commented on the issue:
https://github.com/apache/metron/pull/901
I'm going to wait for @lvets a chance to try his scenario
---
Github user ottobackwards commented on the issue:
https://github.com/apache/metron/pull/903
+1 -> ran up both images, everything checked out
Ship it
---
Github user nickwallen commented on the issue:
https://github.com/apache/metron/pull/903
Thanks @ottobackwards . I'll see if we can get any more reviewers before I
merge this.
---
I wanted to give everyone fair warning on PR #903 [1] before it gets
merged. The path to "Full Dev" would change based on this PR.
Instead of running...
cd metron-deployment/vagrant/full-dev-platform
vagrant up
You would now run...
cd metron-deployment/vagrant/metron-on-centos
vagrant up
Of
Github user asfgit closed the pull request at:
https://github.com/apache/metron/pull/902
---
Github user cestella commented on the issue:
https://github.com/apache/metron/pull/903
Honestly my only beef with this is the directory naming. I think I'd
prefer `metron-deployment/vagrant/{centos_$VERSION,ubuntu_$VERSION}` because
it's possible that we may want 2 separate versions
Github user lvets commented on the issue:
https://github.com/apache/metron/pull/903
- Wouldn't `dev-on-centos` and `dev-on-ubuntu` or even `metron-dev-centos`
and `metron-dev-ubuntu` make more sense? I don't think we want people to expect
that this VM is sufficient to run in productio
Github user JonZeolla commented on the issue:
https://github.com/apache/metron/pull/903
@lvets trusty is 14.04. As far as I'm aware the only newer LTS is 16.04,
with a new one expected in April. https://wiki.ubuntu.com/Releases
---
Github user nickwallen commented on the issue:
https://github.com/apache/metron/pull/903
> @lvets: Just for my understanding, but why Ubuntu Trusty? In April that
will be 2 full Ubuntu LTS versions behind the then current one...
Because that's the requirement that I need to su
Github user nickwallen commented on the issue:
https://github.com/apache/metron/pull/903
I am liking a combination of the suggestions from @lvets and @cestella.
Something like this maybe?
* `dev-on-centos6`
* `dev-on-ubuntu14`
I like the name because of points made by
Thanks Otto, I'm in to attend at that time/place.
Jon
On Thu, Jan 25, 2018, 14:45 Otto Fowler wrote:
> I would like to propose a Metron user community meeting. I propose that we
> set the meeting next week, and will throw out Wednesday, January 31st at
> 09:30AM PST, 12:30 on the East Coast and
Thanks! I'll be there. Excited to hear Ahmed's successes and challenges.
-Kyle
On Thu, Jan 25, 2018 at 7:44 PM zeo...@gmail.com wrote:
> Thanks Otto, I'm in to attend at that time/place.
>
> Jon
>
> On Thu, Jan 25, 2018, 14:45 Otto Fowler wrote:
>
>> I would like to propose a Metron user commu
Maybe we need an adding support for a new platform doc
On January 25, 2018 at 19:37:47, nickwallen (g...@git.apache.org) wrote:
Github user nickwallen commented on the issue:
https://github.com/apache/metron/pull/903
> @lvets: Just for my understanding, but why Ubuntu Trusty? In April that
wil
At the moment, when a grok file or something changes in HDFS, how do we
know? Do we have to restart the parser topology to pick it up?
Just trying to clarify for myself.
ottO
Right now you have to restart the parser topology.
On Thu, Jan 25, 2018 at 10:15 PM, Otto Fowler
wrote:
> At the moment, when a grok file or something changes in HDFS, how do we
> know? Do we have to restart the parser topology to pick it up?
> Just trying to clarify for myself.
>
> ottO
>
58 matches
Mail list logo