Hi, All I am testing my trident code, using OpaqueTridentKafkaSpout to receive data from kafka, then write all we received into postgresDB. All the tests done are not streaming data, I am testing the existing topics in kafka. Here what I see, if I send small amount of message, say 1000, I have no data lost, I can see 1000 in DB. However, if I increase the messages in that topic, for example, 200000 messages, I see 3.7% data drop.
Which part of topology might lead to data loss. latency time, batch size, etc? I am using trident state to populate into DB. thanks AL
