I'm implementing my own ITridentSpout and would like to make it scalable. When I set parallelism hint to 3 with pipelining for my spout, I see the following
[Thread-16-$spoutcoord-spout0] MyEmitter - initializeTransaction() txid 1 prevMetadata null currMetadata null generated metadata 9002537a [Thread-26-spout0] MyEmitter - emitBatch() txId 1 attempt id 0 coordinatorMeta 9002537a this.hashCode() 45c8d2d9 emitted e5f7a9d3 [Thread-30-spout0] MyEmitter - emitBatch txId 1 attempt id 0 coordinatorMeta 9002537a this.hashCode() 38ac85a emitted 1f08bdab [Thread-28-spout0] MyEmitter - emitBatch txId 1 attempt id 0 coordinatorMeta 9002537a this.hashCode() 58ed567b emitted ee35006c It creates 3 emitter instances and 3 threads as expected. But coordinator propagates the same tx id 1 to _all_ emitters, this doesn't make much sense since we should emit tuples for given tx id only once. I was expecting to get the following [Thread-16-$spoutcoord-spout0] MyEmitter - initializeTransaction() txid 1 .. [Thread-26-spout0] MyEmitter - emitBatch() txId 1 ... [Thread-16-$spoutcoord-spout0] MyEmitter - initializeTransaction() txid 2 .. [Thread-30-spout0] MyEmitter - emitBatch txId 2 ... [Thread-16-$spoutcoord-spout0] MyEmitter - initializeTransaction() txid 3 .. [Thread-28-spout0] MyEmitter - emitBatch txId 3 ... Is there any way to achieve this?
