Jeff Dasch updated RYA-359:
    Affects Version/s: 3.2.12

> Update PeriodicNotificationApplication to scale beyond one YARN container.
> --------------------------------------------------------------------------
>                 Key: RYA-359
>                 URL: https://issues.apache.org/jira/browse/RYA-359
>             Project: Rya
>          Issue Type: Improvement
>    Affects Versions: 3.2.12
>            Reporter: Jeff Dasch
>            Priority: Minor
> In theory, this can be achieved in two different ways:
> 1) Update code to support concurrent PeriodicNotificationApplications in 
> separate YARN containers that split the workload for a single rya instance.
> 2) Update code to support the various Runnables executing in separate YARN 
> containers.  This would likely mean replacing the blocking queue being used 
> with serialization and kafka, or possibly adopting the kafka streams library.
> The second option has the benefit that it allows for finer grained control of 
> concurrency, though it does introduce an added serialization overhead.
> Other goal for this effort should be to ensure graceful degradation of 
> performance in the event of a container failure.

This message was sent by Atlassian JIRA

Reply via email to