[ 
https://issues.apache.org/jira/browse/RYA-359?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jeff Dasch updated RYA-359:
---------------------------
    Affects Version/s: 3.2.12

> Update PeriodicNotificationApplication to scale beyond one YARN container.
> --------------------------------------------------------------------------
>
>                 Key: RYA-359
>                 URL: https://issues.apache.org/jira/browse/RYA-359
>             Project: Rya
>          Issue Type: Improvement
>    Affects Versions: 3.2.12
>            Reporter: Jeff Dasch
>            Priority: Minor
>
> In theory, this can be achieved in two different ways:
> 1) Update code to support concurrent PeriodicNotificationApplications in 
> separate YARN containers that split the workload for a single rya instance.
> 2) Update code to support the various Runnables executing in separate YARN 
> containers.  This would likely mean replacing the blocking queue being used 
> with serialization and kafka, or possibly adopting the kafka streams library.
> The second option has the benefit that it allows for finer grained control of 
> concurrency, though it does introduce an added serialization overhead.
> Other goal for this effort should be to ensure graceful degradation of 
> performance in the event of a container failure.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Reply via email to