I think its a common problem in distributed system:
I only need a redis pool in one VM(or less complicated, one redis pool for
a bolt class), and I want to be able to change the host/port in the config
file without recompiling code.
so when I submit the topology, i read from file, and put into storm config,
so the config (host/port) are all available in prepare() of bolt. Normally,
one would think of doing the following:
class MyBolt{
static JedisPool jedispool;
public void prepare(Map stormConf, TopologyContext context,
OutputCollector collector) {
jedisPool = new JedisPool(poolConfig,
stormConfig.get(host),
stormConfig.get(port));
}
But if MyBolt's parallelism is 200, then I will still create 200
JedisPools, right? and I would not know which one is actually used during
execute()..
So how do you guys normally handle this scenario? It would be the same
problem for creating db connection pool etc.
Thanks for your input.
Chen