Hi, I was looking at code where HDFSBolt writes the file with tuple coming to it. I also had a look at JIRA STORM-969<https://issues.apache.org/jira/browse/STORM-969>, I have following question to it
1)Lets say I have setup fileRotation policy at 64 MB, and I have written file upto 59MB. Now, my worker failed, the file which I was writing to will be never get rotated to final location. 2) As per 969 Jira, they have added the forceSync way but they are keeping all tuples in memory and delaying the acks sent back to spout. In our case, to write 64 MB data we need to store 5,400,000 process messages, which leads to lot of data in memory. This may lead to unnecessary reply of tuple from spout(I am aware its at least once and I can increase TOPOLOGY_MESSAGE_TIMEOUT_SECS to full fill my requirement but is there other way?) Thanks, Sachin
