> On Nov. 13, 2014, 4:27 a.m., Kevin Sweeney wrote:
> > src/main/java/org/apache/aurora/scheduler/events/PubsubEventModule.java, 
> > line 98
> > <https://reviews.apache.org/r/27947/diff/1/?file=760831#file760831line98>
> >
> >     I don't think we want enqueuing a pubsubevent for processing to block, 
> > ever (otherwise we open ourselves to the possibility of deadlocks here). 
> > Consider using a LinkedBlockingQueue instead here. As this queue is still 
> > bounded at MAX_INT we might further consider adding a 
> > https://docs.oracle.com/javase/7/docs/api/java/util/concurrent/RejectedExecutionHandler.html
> >  that logs an appropriate message and shuts down the scheduler to make 
> > debugging easier.

This is actually how the Executors.newCachedThreadPool() configures the async 
bus executor now. The only difference in this diff is capping the maxPoolSize 
to something more sane than Integer.MAX_VALUE.

Looking at the ThreadPoolExecutor and the SynchronousQueue implementation it's 
not going to block when threads are busy/exhausted. The 
SynchronousQueue.offer() is going to timeout immediately (return false) and the 
ThreadPoolExecutor.execute() will proceed straight to reject(), where the 
default AbortPolicy would result in a RejectedExecutionException thrown in a 
thread calling EventBus.post(). So, the debugging story should be addressed 
well here. What is not addressed though is the fact that we are going to fail 
on start any time producers outrun consumers - pretty much guaranteed in a 
large cluster. 

I am going to switch to the newFixedThreadPool() with a few threads as Bill 
suggested. It already uses LinkedBlockingQueue and it's unlikely that we ever 
exceed Integer.MAX_VALUE on queueing side with multiple consuming threads.


- Maxim


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/27947/#review61203
-----------------------------------------------------------


On Nov. 13, 2014, 1:12 a.m., Maxim Khutornenko wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/27947/
> -----------------------------------------------------------
> 
> (Updated Nov. 13, 2014, 1:12 a.m.)
> 
> 
> Review request for Aurora and Bill Farner.
> 
> 
> Bugs: AURORA-926
>     https://issues.apache.org/jira/browse/AURORA-926
> 
> 
> Repository: aurora
> 
> 
> Description
> -------
> 
> Setting the max thread limit on AsyncEventBus.
> 
> 
> Diffs
> -----
> 
>   src/main/java/org/apache/aurora/scheduler/events/PubsubEventModule.java 
> 24cd750c676a280d1cc953c96df126f19fc478f2 
> 
> Diff: https://reviews.apache.org/r/27947/diff/
> 
> 
> Testing
> -------
> 
> ./gradlew -Pq build
> 
> 
> Thanks,
> 
> Maxim Khutornenko
> 
>

Reply via email to