On Fri, 7 Aug 2009, Oliver Stöneberg wrote:
> There are a few cases in that XMail is distributing data randomly to
> various destinations (e.g. relay domains or internal spool folders -
> the code with "rand() % count"). It's not distributing the data
> evenly in theses cases, so you end up with e.g. one server getting or
> traffic or one spool folder being filled more than others. Especially
> the first case has been problematic for some of our customers.
>
> On a side note: all the different cases use basically the same logic,
> so fixing it in one case would actually fix all of them.
>
> A collegue did quite some testing on it and here are his findings:
>
> When using load balancing with more than 2 relays, XMail is unevenly
> distributing the load upon the configured relays.
>
> Here is an example of the distribution to 3 mailservers. The data was
> captured in a test running for 9 hours with approximately 25500
> mails:
>
> 1. relay: 55.31%
> 2. relay: 22.31%
> 3. relay: 22.36%
>
> The reason for this behaviour is the randomization / shuffling code
> that is used for picking a mail relay to deliver an e-mail to.
>
> Let's stick with the setup above using 3 mail relays. In this case
> the randomization code will choose only one relay in the list of
> available relays (floor(3 / 2)) and swap it with another relay, which
> is also chosen at random and completely indepent of the first choice.
> Thus, swapping the first with the first relay, the second with the
> second, etc. is a possible outcome.
>
> The easiest way to visualize the randomization in this case is the
> following table:
>
> 1 2 3
> 1 x
>
> 2 x x
>
> 3 x x
>
> The rows of this table represent the first chosen index of a relay,
> the colums denote the index of the second choice (and thus the relay
> that is swapped to the position of the first chosen relay). The 'x'
> in the table is set, whenever the first element in the list will not
> change (for example, swapping the second and the third relay).
> As you can easily see in this table, in 5 out of 9 cases (55%) the
> first relay is not swapped to another position and, consequently, 55%
> of the e-mails get delivered to the first relay (which is almost
> exactly the number we have
> measured in our test environment).
Thank you for reporting it. The problem, as I tried on my Linux box, is
the srand() done just before the randomization loop.
You can try to swap the position of the srand() in the code below, to see
how the distribution changes.
The problem is that across various Unix and Windows, the preservation of
the random seed changes WRT threads.
Removing the srand() and replacing it with a single one at program start,
works fine for systems using a shared (among threads) seed.
This is how it has been fixed ATM.
- Davide
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <time.h>
static int StrStringsCount(char **s) {
int i;
for (i = 0; s[i] != NULL; i++);
return i;
}
static int MscRandomizeStringsOrder(char **ppszStrings)
{
int i, iCount = StrStringsCount(ppszStrings);
// srand((unsigned int) time(NULL));
for (i = 0; i < iCount - 1; i++) {
int iChoice = rand() % (iCount - i);
char *pszTmp = ppszStrings[i];
ppszStrings[i] = ppszStrings[i + iChoice];
ppszStrings[i + iChoice] = pszTmp;
}
return 0;
}
int main(int ac, char **av) {
int i, j, count = 7, samples = 10000;
int *vals;
char **strs;
srand((unsigned int) time(NULL));
vals = (int *) malloc(count * sizeof(int));
strs = (char **) malloc((count + 1) * sizeof(char *));
for (i = 0; i < count; i++)
vals[i] = 0;
for (j = 0; j < samples; j++) {
for (i = 0; i < count; i++) {
char *s;
asprintf(&s, "%d", i);
strs[i] = s;
}
MscRandomizeStringsOrder(strs);
vals[atoi(strs[0])]++;
for (i = 0; i < count; i++)
free(strs[i]);
}
for (i = 0; i < count; i++)
printf("%d = %.2lf%%\n", i, 100.0 * (double) vals[i] / (double)
samples);
return 0;
}
_______________________________________________
xmail mailing list
[email protected]
http://xmailserver.org/mailman/listinfo/xmail
