Yes, we specially did some more tests with EPS 100 and every time got a 10
second window with reindexing
On 2019/12/17 18:42:17, Michael Miklavcic wrote:
> Is it always a 10 second window, or thereabouts?
>
> On Sun, Dec 15, 2019 at 11:11 PM Vladimir Mikhailov <
>
Is it always a 10 second window, or thereabouts?
On Sun, Dec 15, 2019 at 11:11 PM Vladimir Mikhailov <
v.mikhai...@content-media.ru> wrote:
> So, we conducted a number of additional tests that confirmed the problem.
>
> The test is very simple:
>
> we generated a very stable stream of events (2
So, we conducted a number of additional tests that confirmed the problem.
The test is very simple:
we generated a very stable stream of events (2 EPS) and did a parser restart
and each time we received re-indexing of events with a 10 second window.
Parser is simple JSONMap without enrichment
Thanks for the clarification!
So we need to conduct a few more tests to understand the cause of this problem.
I will write about the results.
On 2019/12/11 14:01:45, Nick Allen wrote:
> > And now the question: does stopping the parser with the "storm kill
> " command mean that the topology
> And now the question: does stopping the parser with the "storm kill
" command mean that the topology will complete the processing of all
current events that were read by kafkaSpout and commit the corresponding
offset to kafka?
Yes, it will wait as long as the topology's message timeout (by
It seems that I misinformed our situation by writing about the option
"set_wait_secs (0)".
We performed all our tests by disabling parsers in the Metron Management UI, so
I suppose they were all stopped using "storm kill " (and not "storm kill
-w 0"). And in these tests we got reindexing some
It only does that if the arg stopNow is true. It's always false per the
previous snippets I shared.
On Tue, Dec 10, 2019, 10:54 PM Vladimir Mikhailov <
v.mikhai...@content-media.ru> wrote:
> Hi Michael
>
> I think the problem is not on the REST side, but in the "StormCLIWrapper",
> which it
Hi Michael
I think the problem is not on the REST side, but in the "StormCLIWrapper",
which it uses:
https://github.com/apache/metron/blob/88f4d2cefe4bbb389732da3b4f5cbcf02b7b949a/metron-interface/metron-rest/src/main/java/org/apache/metron/rest/service/impl/StormCLIWrapper.java#L145
Each of
Where are you seeing this? As far as I can tell, the UI and REST endpoints
default to a graceful shutdown.
https://github.com/apache/metron/blob/master/metron-interface/metron-config/src/app/service/storm.service.ts#L154
Hi
We found the unpleasant consequences of each restart of the parsers: each time
part of the events are reindexed again. Unfortunately, this was confirmed by
several special tests.
Perhaps the reason for this is the method used to immediately stop the storm
topology using
10 matches
Mail list logo
