Hello,
So it looks like my load-balancing solution is interesting for someone
except me so I will try to port it in next one, two weeks.
There are two more things left:
1) Currently when one of the servers in the group fails to respond it is
ignored and search results are returned without data present in segments
from this server (see org.apache.nutch.ipc.Client.call(Writable[]
params, InetSocketAddress[] addresses)). This behavior is a problem for
load-balancing solution as second request by the same user can be
handled by different set of servers we can have duplicated search
results (when user wants to see next page of results) or missing
information when user wants to see explanation or cached data.
In my opinion in load -balancing solution such situation should be
considered an error and other set of the servers should be used for
handling user query.
To support new and old behavior some changes to RPC and Client classes
are required - I used to overwrite inherited method in
DistributedClient.Client but because of recent changes it is not
possible to do it without affecting RPC and Client in ipc package.
2)As behavior of the system will change probably old version of search
code should also be supported - I have to investigate how to handle both
versions of the code at the same time and not to create a mess.
I will post my first results as soon as I make it work for comments.
Regards,
Piotr
Andrzej Bialecki wrote:
Piotr Kosiorowski wrote:
Hello Stefan,
I have already written a component that implements this round robin
searching functionality some time ago - but right now it is not working
[..]
So if there is enough interest for it I can downgrade it to JDK 1.4 (I
am using java.util.concurrent) and send it as a patch.
That would be very nice! concurrent-1.3.4.jar is already a part of the
Nutch distribution - I don't remember how different it is from JDK5, but
this should ease the porting effort...
