Theo Van Dinter wrote: > The patch does make things *much* slower though, around 3x: [...] > Without the patch, lots of issues starting after 80%.
I don't claim that the patch is the most efficient way of dealing with it... I just wanted to use SHA1 to ensure that there was no chance of an ID collision. I think we have now verified that ID collision is the likely proximate cause of the problem. Still, can three SHA1 calculations compare to the time it takes for a DNS query? I don't see how the computation would slow things down by a factor of three. Perhaps what you are seeing is the difference in wall clock time between processing a reply to an old packet when it arrives right away vs rejecting those packets and waiting for the actual reply. If that's what's happening you are not going to get the faster time when everything works, as it is the nameserver's response time that is slowing down the run. The faster time is perhaps just a symptom of the bug? There is still the question of where the collisions are coming from. Here's another idea -- Instead of using substr(sha1_base64($host . $id), -7) use something that combines the pid and id into a six byte string in the three places in the code where sha1 is used. That will be faster than using SHA1, which will let you know if the slowdown is due to computation or waiting for good packets to arrive, and it will let you know if the problem is with different processes using the same source port for sending the UDP queries. If it is the latter, we may be able to avoid the collisions if we are better about picking the source ports. Are you up for some more few-hour tests? :-) -- sidney
