The by-HTL stats include the average distance between the key and the ideal location for each HTL.
Yesterday on my node, the closest point for CHKs is 0.0003 at HTL 13, and for SSKs it is 0.0041 at HTL 14. Note the extra zero! So we are nearly 12 times better at routing for SSKs than for CHKs! Another time (today), the ideal for SSKs is 0.0006 at HTL 15 and 13, but for CHKs it is 0.0063 at HTL 15. So the good news is we converge pretty quickly - the bad news is for CHKs we seem to not be able to get close to the target. IMHO this is because of my turning off some of the fairness between types logic. It was not possible to make it work with the new load management changes. There is a new version that will be deployed with the final new load management. But it also shows what is possible, and IMHO confirms my theory that misrouting as a result of mishandling of load is the main reason why performance is relatively poor. I doubt that reinstating the old fairness between types logic would do anything more than converge them both on 0.003 or thereabouts. What we need is the new load management logic. NEXT STEPS: * 1310 includes the first part of the bulk flag. This is necessary for new load management because new load management involves queueing i.e. is relatively high latency; by marking low latency requests explicitly, we can optimise them for latency, while using more of the available capacity for requests marked bulk that only care about throughput (most requests imho). Realtime requests have shorter timeouts, have priority for transfers (but with a scheme to prevent starvation), are accepted by load limiting in much fewer numbers, and are assumed to be very bursty. * Next week I will introduce code to actually set the bulk flag. For now it is always set to bulk. Next week we will set it to realtime for fproxy requests, and allow FCP apps to set it if they want. * We will also eliminate turtles, and increase the number of bulk requests allowed significantly. * Requests which are rejected due to short-term load management are marked as "soft" rejects. A very small change to make the node not remember such requests, so that other nodes can try again and maybe get accepted, should improve routing slightly. It is also a prerequisite for new-load-management which uses a limited number of retries in some cases (because while we know how many requests we are responsible for, we don't know how many requests other nodes are doing on our peer). * Two-stage timeout is important for new load management: If a downstream node times out, we need our peer to tell us that it has recognised the timeout. This allows us to know exactly when it is no longer running the request, and thus have an accurate count of how many of our requests are running on our peer, which is vital for new-load-management. * We will start sending the messages which indicate exactly how many requests are running and how many can be accepted from the peer. No more than one of these (for each of bulk and realtime) will be included in any given packet and it will be up to date at time of sending. * Then we can actually use them! The core of new load management is queueing so that we can get routed to a node reasonably close to our ideal. If a node is way below the median capacity, or if it is severely backed off, we won't wait for it, but otherwise we try to wait for our first choice. Lots of work has already been done towards this, for instance recently the changes making requests essentially threadless are a prerequisite for the bulk flag's greatly increasing the number running.
signature.asc
Description: This is a digitally signed message part.
_______________________________________________ Support mailing list Support@freenetproject.org http://news.gmane.org/gmane.network.freenet.support Unsubscribe at http://emu.freenetproject.org/cgi-bin/mailman/listinfo/support Or mailto:support-requ...@freenetproject.org?subject=unsubscribe
