Am 04/07/2012 10:03 PM, schrieb Rob Weir:
On Sat, Apr 7, 2012 at 3:35 PM, Marcus (OOo)<[email protected]> wrote:
Am 04/07/2012 07:38 PM, schrieb Marcus (OOo):
Am 04/07/2012 05:34 PM, schrieb Rob Weir:
On Sat, Apr 7, 2012 at 10:13 AM, Marcus (OOo)<[email protected]>
wrote:
For our new release we will have a changed set of conditions:
- new central mirror hosts for redirecting download requests
- therefore also a new set of mirror server that host our builds
- new set of supported platforms and languages
- slighty changed file name schema
To get this all under one hat, we have to improve the download logic
that is
currently done by JavaScript.
To fulfill the first condition the download requests have to be
split-up to
more than 1 central mirror redirector.
Here is a suggestion for choosing when (by random number) to redirect to
which mirror:
http://www.openoffice.org/download/test/index_new_dl.html
Test results:
- to make debugging easier I've included some screen output and comments
- I've tested it locally with different combinations of active/inactive
hosts and it works well
- the fallback method works well, too
- I've tested different percent values as ratio, but not yet for the
2nd case
What do you think?
The logic on the test pages works for me.
But two additional complexities:
1) MirrorBrain let's us link directly to a download file. We then
send the user to http://www.openoffice.org/download/contribute.html
while the file is downloading. But SourceForge and Apache mirror
systems take the user to a different page (not controlled by us). So
in those cases the user does not see the contribute.html page.
This should work also for SourceForge. The root path is always the same.
We just need to add a subdir and file name.
I don't know how to do it for Apache.
Since the contribute.html page has very useful information for the
user, like links to the install instructions, information on support,
contributing to the project, etc., I think it is important that the
user sees this information in all paths.
ACK
What could we do? Pop-up (or pop-under) a new HTML page? But that is
annoying to some users.
Right
The Apache and SF approaches both direct a user off of the
OpenOffice.org website, which is risky.
2) What if we have more than three mirror systems? It sounds like you
prefer to keep MirrorBrain. That is fine with me. So maybe we have
Apache, SF and MirrorBrain?
Yes, would be great. Another available system as possible backup is not
bad.
For any change of the number of available systems we have to adapt the
JS logic.
Imagine an array of mirror networks, each with weights, in sorted
order from smallest to largest. (or sort the array in code)
Apache:1
MirrorBrain:2
SourceForge:3
TotalWeight = MirrorBrain + Apache + SourceForge == 6
x = rand()
if (x< Apache/TotalWeight)
doApache()
else if (x< MirrorBrain/TotalWeight)
doMirrorBrain()
else
doSourceForge()
Something like that.
Redirecting due to weighted values looks good. But also this is working
with a fixed set of mirror hosts. A forth had to be build in and
therefore the logic has to be adapted.
Thanks for your feedback. I'll adapt the script later in the evening.
I've adapted it now. But I'm not sure if it's working correct with some
disabled mirrors.
I'm not sure my logic was right in the above code. For example, if
two mirrors have the same weight it would always pick the first one in
the array.
But take a look at the approach in this article:
http://www.javascriptkit.com/javatutors/weighrandom2.shtml
I think that is much clearer and easier to use.
Essentially, it says if you want weighted random selection with weights like:
Apache:1
MirrorBrain:2
SourceForge:3
Then create an array with each item repeated according to its weight, so:
Apache
MirrorBrain
MirrorBrain
SourceForge
SourceForge
SourceForge
Then just pick a random number 0...length-1 and return the value at that index.
Thanks again for this hint. I've rebuild the script and published it as
3rd case. Now even the backups are working well. Looks like the best
solution for 3 redirectors up to now.
As for unavailable mirrors, or even unavailable servers within a
mirror network, is there really anything we can do? Other than tell
the user to refresh and try again?
We could ping all redirector hosts frequently (e.g., every 5 minutes).
If the result is a delay higher than a pre-defined timeout, then we
could set this redirector to 0 and therefore inactive.
However, I've no idea how to do this not with every download request
(would this already be a kind of (D)DoS ?) but as a job that runs beside
this.
Marcus
