I created some pull requests for review. The main purpose is to improve the local stream builder runtime. The idea was to remove busy waits, unnecessary waiting, and allow more transparency into what is happening when a stream is running. This only scratching the service on all of the needs improvements but it is a start.
In the pull request I have included code that allows monitoring of the queues that linked together the processors/writers/providers through JMX. This should allow you to view where data is backing up and help you configure your streams in the future. I plan on continuing to add more JMX enabled monitoring in the future. The next step would be to refactor the DatumCountable objects to use JMX instead of printing results to the console. This will reduce thread usage, and make the information available remotely and easier to read. Please let me know if there is any feed back on adding JMX functionality. I will create JIRA tickets and would love help in creating these improvements. -Ryan Ebanks
