https://issues.apache.org/jira/browse/SAMZA-654 - Elasticsearch Producer
Please comment there if you have any opinions on requirements for the clients / configuration / serialization with using this producer. It's currently fit for our use case but I know there will be more. https://issues.apache.org/jira/browse/SAMZA-655 - Env variable rewriter Again any comments on the format of the variables names welcome! - Dan On 15 April 2015 at 17:38, Chinmay Soman <chinmay.cere...@gmail.com> wrote: > +1 ! I was going to do this for my use case as well. Would love to have > this ! > > On Wed, Apr 15, 2015 at 9:24 AM, Roger Hoover <roger.hoo...@gmail.com> > wrote: > > > Dan, > > > > This is great. Would love to have a common ElasticSearch system > producer. > > > > Cheers, > > > > Roger > > > > On Tue, Apr 14, 2015 at 1:34 PM, Dan <danharve...@gmail.com> wrote: > > > > > Thanks Jakob, I agree they'll be more maintained and tested if they're > in > > > the main repo so that's great. > > > > > > I'll sort out Jira's and get some patches of what we've got working now > > out > > > for review. > > > > > > - Dan > > > > > > > > > On 14 April 2015 at 19:56, Jakob Homan <jgho...@gmail.com> wrote: > > > > > > > Hey Dan- > > > > I'd love for the Elastic Search stuff to be added to the main code, > as > > > > a separate module. Keeping these in the main source code keeps them > > > > more likely to be maintained and correct. > > > > > > > > The EvironemtnConfigRewriter can likely go in the same place as the > > > > ConfigRewriter interface, since it doesn't depend on Kafka as the > > > > current RegexTopicRewriter does. > > > > > > > > If you could open JIRAs for these, that would be great. Happy to > > > > shepard the code in. > > > > -Jakob > > > > > > > > On 14 April 2015 at 11:46, Dan <danharve...@gmail.com> wrote: > > > > > Hey, > > > > > > > > > > At state.com we've started to write some generic extensions to > Samza > > > > that > > > > > we think would be more generally useful. We've got > > > > > a ElasticsearchSystemProducer/Factory to output to an Elasticsearch > > > index > > > > > and EnvironmentConfigRewriter to modify config from environment > > > variable. > > > > > > > > > > What's the best way for us to add things like this? Do you want > more > > > > > modules in the main project or should we just create some separate > > > > projects > > > > > on github? > > > > > > > > > > It would be good to get core extensions like these shared to be > > tested > > > > and > > > > > used by more people > > > > > > > > > > Thanks, > > > > > Dan > > > > > > > > > > > > > -- > Thanks and regards > > Chinmay Soman >
