nicknezis edited a comment on pull request #3725: URL: https://github.com/apache/incubator-heron/pull/3725#issuecomment-965989191
One thought for the `ondemand` approach would be that perhaps we should be leveraging the `VolumeClaimTemplate` feature of a StatefulSet. Here is an [example](https://stackoverflow.com/questions/53927587/kubernetes-multiple-pvc-with-statefulset-for-each-pod-vs-single-pvc-for-all-pods). This would allow for a PVC per pod without having to manually create them. Kubernetes will automatically create with the pod's ordinal number. This might also help answer how we handle the `addContainer` call which can be called when someone rescales a running Heron topology. In this case any new pods would get a new PVC. I think it's ok to leave the PVC if the analytic is scaled down because we will cleanup all PVCs when the topology is killed. **_Edit:_** This would also allow us to keep most of your logic the same. I do agree there is benefit in having the single shared volume so don't want to lose that. I perhaps was describing things with too strong a focus on the `ondemand` use case. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
