So I'm looking to expand the ElasticSearchIO writer to support dynamic indexes. Wanted to kick off talking about design. I'll create a Jira for it once I get going.
Right now I'm thinking to expand the ConnectionConfiguration class to accept either an index (Current), dot-path string to look into the JSON with to pull the index value, or a user provided function that takes the document and returns an index. The Write class will change to use a MultiMap where keys are the index. The whole Multimap will be treated as the source for the batch sizes. I opt for that since I see the index usually being the same until time window endings. I'm aiming for the simplest change in design here. To not impact the work Chet is doing on BEAM-3201. -- Nick Verbeck - NerdyNick ---------------------------------------------------- NerdyNick.com TrailsOffroad.com NoKnownBoundaries.com
