Hey Domenic,
Yep, it can be challenging to track down the docs here. Let me try to run
down the options for you:
- topology.workers : Under the config section of the topology
- topology.acker.executers : Under the config section of the topology
- parallelism : Under the individual bolts and spouts
Take for instance the following example:
> name: "enrichment"
> config:
> topology.workers: 1
> topology.acker.executors: 0
>
> bolts:
> # Enrichment Bolts
> - id: "enrichmentSplitBolt"
> className: "org.apache.metron.enrichment.bolt.EnrichmentSplitterBolt"
> constructorArgs:
> - "${kafka.zk}"
> configMethods:
> - name: "withEnrichments"
> args:
> - ref: "enrichments"
> parallelism: 1
>
>
On Tue, Jun 7, 2016 at 11:29 AM, Puzio, Domenic <
[email protected]> wrote:
> I have a related question. Is there any documentation on setting the Storm
> configurations like parallelism.hint and num.tasks for individual bolts in
> the Enrichment topology? I know this is done in Flux, but I am not sure
> what these configs will look like and where they need to go.
>
>
> Thank you!
>
> Domenic
>
> On 6/7/16, 11:09 AM, "Rider,Jonathan" <[email protected]>
> wrote:
>
> >Hello,
> >
> >We’re currently testing all of our parsers on our QA environment and as
> we added more parsers, the enrichment topology started slowing down
> dramatically. Namely, the indexingBolt had a process latency of 12ms and
> now it’s 200ms, (Execute latency went from 2 to 25).
> >
> >Upon further inspection, the indexing bolt has lots and lots of
> NullPointerExceptions. The logs are less than helpful, as they look like
> this:
> >
> >
> >2016-06-07 11:03:29.360 b.s.d.executor [ERROR]
> >java.lang.NullPointerException
> >2016-06-07 11:03:29.360 b.s.d.executor [ERROR]
> >java.lang.NullPointerException
> >2016-06-07 11:03:29.360 b.s.d.executor [ERROR]
> >java.lang.NullPointerException
> >2016-06-07 11:03:29.360 b.s.d.executor [ERROR]
> >java.lang.NullPointerException
> >2016-06-07 11:03:29.361 b.s.d.executor [ERROR]
> >java.lang.NullPointerException
> >2016-06-07 11:03:29.361 b.s.d.executor [ERROR]
> >java.lang.NullPointerException
> >2016-06-07 11:03:29.361 b.s.d.executor [ERROR]
> >java.lang.NullPointerException
> >2016-06-07 11:03:29.361 b.s.d.executor [ERROR]
> >java.lang.NullPointerException
> >2016-06-07 11:03:29.361 b.s.d.executor [ERROR]
> >java.lang.NullPointerException
> >2016-06-07 11:03:29.361 b.s.d.executor [ERROR]
> >java.lang.NullPointerException
> >2016-06-07 11:03:29.361 b.s.d.executor [ERROR]
> >java.lang.NullPointerException
> >2016-06-07 11:03:29.362 b.s.d.executor [ERROR]
> >java.lang.NullPointerException
> >2016-06-07 11:03:29.362 b.s.d.executor [ERROR]
> >java.lang.NullPointerException
> >2016-06-07 11:03:29.362 b.s.d.executor [ERROR]
> >java.lang.NullPointerException
> >2016-06-07 11:03:29.362 b.s.d.executor [ERROR]
> >java.lang.NullPointerException
> >2016-06-07 11:03:29.362 b.s.d.executor [ERROR]
> >java.lang.NullPointerException
> >2016-06-07 11:03:29.362 b.s.d.executor [ERROR]
> >java.lang.NullPointerException
> >2016-06-07 11:03:29.363 b.s.d.executor [ERROR]
> >java.lang.NullPointerException
> >2016-06-07 11:03:29.363 b.s.d.executor [ERROR]
> >java.lang.NullPointerException
> >2016-06-07 11:03:29.363 b.s.d.executor [ERROR]
> >java.lang.NullPointerException
> >2016-06-07 11:03:29.363 b.s.d.executor [ERROR]
> >java.lang.NullPointerException
> >2016-06-07 11:03:29.363 b.s.d.executor [ERROR]
> >java.lang.NullPointerException
> >
> >Has anyone seen this kind of error before and know what is causing it?
> >
> >Thanks,
> >
> >Jonathan Rider
> >________________________________________________________
> >
> >The information contained in this e-mail is confidential and/or
> proprietary to Capital One and/or its affiliates and may only be used
> solely in performance of work or services for Capital One. The information
> transmitted herewith is intended only for use by the individual or entity
> to which it is addressed. If the reader of this message is not the intended
> recipient, you are hereby notified that any review, retransmission,
> dissemination, distribution, copying or other use of, or taking of any
> action in reliance upon this information is strictly prohibited. If you
> have received this communication in error, please contact the sender and
> delete the material from your computer.
> ________________________________________________________
>
> The information contained in this e-mail is confidential and/or
> proprietary to Capital One and/or its affiliates and may only be used
> solely in performance of work or services for Capital One. The information
> transmitted herewith is intended only for use by the individual or entity
> to which it is addressed. If the reader of this message is not the intended
> recipient, you are hereby notified that any review, retransmission,
> dissemination, distribution, copying or other use of, or taking of any
> action in reliance upon this information is strictly prohibited. If you
> have received this communication in error, please contact the sender and
> delete the material from your computer.
>
