[
https://issues.apache.org/jira/browse/SAMZA-675?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Naveen Somasundaram updated SAMZA-675:
--------------------------------------
Description:
One of the nice things we get for coordinator stream is, we have the previous
configs we launched the job with. We can use this to our advantage for config
validation. For e.g.,
1. The number of partitions in the Kafka stream has changed, this would break
our change log partition mapping.
2. Store was opened with TTL, but reopened again without TTL in RocksDB, this
can cause a corruption.
And many more checks like this. We need a place (probably JobCoordinator) to do
these validations on, validate(oldConfig from CoordinatorStream -
newConfigsUsers have passed)
was:
One of the nice things we get for coordinator stream is, we have the previous
configs we launched the job with. We can use this to our advantage for config
validation. For e.g.,
1. The number of partitions in the Kafka stream has changed, this would break
our change log partition mapping.
2. Store was opened with TTL, but reopened again without TTL in RocksDB, this
can cause a corruption.
And many more checks like this. We need a place (probably JobCoordinator) to do
these validations.
> Check backwards incompatible config changes
> --------------------------------------------
>
> Key: SAMZA-675
> URL: https://issues.apache.org/jira/browse/SAMZA-675
> Project: Samza
> Issue Type: Bug
> Components: container
> Affects Versions: 0.10.0
> Reporter: Naveen Somasundaram
>
> One of the nice things we get for coordinator stream is, we have the previous
> configs we launched the job with. We can use this to our advantage for config
> validation. For e.g.,
> 1. The number of partitions in the Kafka stream has changed, this would break
> our change log partition mapping.
> 2. Store was opened with TTL, but reopened again without TTL in RocksDB, this
> can cause a corruption.
> And many more checks like this. We need a place (probably JobCoordinator) to
> do these validations on, validate(oldConfig from CoordinatorStream -
> newConfigsUsers have passed)
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
