Hi, In Malhar we have an operator AbstractBatchTransactionableStoreOutputOperator which creates batches based on tuples received in a window. At the end of the window these batches are sent to database for processing. There is no way to configure MAX_SIZE on these batches. Based on input rate the batch sizes can grow very high, and we might want to restrict batch size.
Any operator can extend and do batch management on their own, but I see it as generic requirement and IMO we should change base class i.e. AbstractBatchTransactionableStoreOutputOperator class to accept MAX_SIZE for batch from outside. Any opinion on this? -Priyanka
