Author: toad
Date: 2008-02-08 20:53:19 +0000 (Fri, 08 Feb 2008)
New Revision: 17723
Modified:
trunk/freenet/src/freenet/node/Version.java
Log:
1109: Mandatory Tuesday
Messaging layer:
- Fix a major bug causing requests and anything else using
MessageCore.waitFor() to hang. This caused the "my node's bandwidth goes to
zero after N hours" bug which was widely reported.
- Some paranoia.
Client layer:
- Use Object's and not int's as tokens in the SendableRequest API. (Lots of
refactoring, needed by ULPRs).
ULPRs/FailureTable:
- Lots of bug fixes!
-- FailureTableEntry fixes e.g. NPEs, ArrayIndexOutOfBounds, using wrong
variable in a loop. Minor refactoring within complex methods.
-- FailureTable fixes e.g. leaks, pushing wrong object.
-- Various refactoring, deleting dead code etc, simplify some functions.
-- Add nodes as requested-from when we RejectedOverload them due to load.
-- Add all nodes we routed to to the list of routed-to nodes.
-- Don't send an offer to the same node as both requestor and requested from.
-- FNPOfferKey now contains an authenticator. We use this to sign our own
offers (with an HMAC) so we don't have to remember whether we made them.
- Wire in ULPRs (mostly). RecentlyFailed isn't used yet (it probably will be at
some point), and there is a lot more to do e.g. failure stats, FCP integration.
But when we find a key, we offer it to nodes who recently asked for it, and we
respond to such offers if we want the key.
-- Node now has a FailureTable.
-- Call onFound() when we store a block, so the failure table can offer the
newly discovered data to subscribers.
-- RequestSender: Call node.failureTable.onFailed() or onFinalFailure() to tell
the failure table about nodes we've sent the key to (when they have been the
cause of the failure), and which have asked for it. Include a timeout for DNFs
and RFs, for generating RecentlyFailed's.
-- If a request takes so long that the predecessor will have timed out, we
won't send the data to it since it would be dropped on the floor. But we can
offer it the data in case it's still interested.
-- Queue incoming offers in the OfferedKeysList, we keep one list for each
priority class. The request scheduler will request keys on the offered list in
preference to other keys on that priority directly 50% of the time (or if there
are no other keys queued). Offers for keys that we don't want but other nodes
do are put in at IMMEDIATE_SPLITFILE_PRIORITY_CLASS. In RequestSender, we first
try any nodes that have offered the key (via FNPGetOfferedKey, short timeout),
then we do a regular request if this wasn't from OfferedKeysList.
-- Clean up the failure table structure every 30 minutes to minimise memory
usage and maximise privacy.
- Per-node failure tables. Defeat cancer nodes, turn repeated requests for the
same key into something vaguely useful instead of going down the same paths
repeatedly.
-- After any failure, we avoid the node causing the failure for a period.
-- For a DNF, fatal timeout, or other fatal error, this is 10 minutes. For
RecentlyFailed it's the period given.
-- For a non-fatal timeout, it's the time it took to get the error, to provide
some protection against nodes returning requests very slowly to cause a DNF on
the following node.
-- Routing order:
--- Non-backed-off nodes which we haven't had failures from, nearest location
to the target first.
--- Non-backed-off nodes which we have had failures from, oldest failure first.
--- Backed-off nodes which we haven't had failures from, nearest location to
the target first.
--- Backed-off nodes which we have had failures from, oldest failure first.
- Comments.
- Some interesting supporting code:
-- Node.asyncGet(Key key, boolean cache) - used by OfferedKeysList for really
simple fetch of a Key (not a ClientKey). Relies on back-door coalescing, of
course, since it won't return the data.
Requests:
- Don't decrement HTL in RequestHandler, we already decrement it in
RequestSender. Get rid of spurious decrements in AnnounceSender too.
Stats serialisation:
- Persister: don't rename until after closed the file, and don't rename if we
didn't succeed.
- ConfigurablePersister: filenames are relative to the nodeDir
L10n:
- German update.
Simulators:
- Implement a one-VM-many-nodes test of ULPRs. Use it to debug ULPRs.
- Implement a one-VM-many-nodes test of the network ID code.
-- It does 100 tests, each may take up to 5 minutes, but if it fails it kills
the whole simulation. They are expected to succeed, and fast.
-- The simulation consists of creating 10 nodes, creating a key, asking each
node for the key, then inserting it on one node and timing how long it takes
for it to propagate to every node. The average here is 3 seconds over 256
attempts. Of course, there are only 10 nodes.
- Eliminate some bugs in the simulator layer.
-- Allow connections to local addresses in the config for test nodes.
-- Delete everything in the test dir when we start up.
-- Wait until all nodes are connected (in ULPR test) before commencing testing.
This should be factored out and used by all the tests.
-- Don't insert ARKs when fetching. New expert mode config option.
-- Increase thread limit proportionally to number of nodes running in the VM.
It's based on a global thread count.
-- disableProbabilisticHTL does NOT mean never decrement HTL, it means always
decrement it.
-- Node ports 5000+nodeNum not 5001+nodeNum.
-- No non-daemon plug thread is needed in simulations, because it wastes a
thread per node and the main() normally runs the simulation anyway.
-- Remove all peers at the beginning of every simulation.
- Make the network immediately routable for request/insert test.
- Lots of logging/output changes.
Config etc:
- New options enableARKs, enablePerNodeFailureTables,
enableULPRDataPropagation. Likely to be mainly used by simulations.
- Persister bugfix: close the file before trying to move it. This may have
caused corruption/data loss on windows???
Datastore:
- Commit the pubkey before the SSK. Otherwise it's possible to get a race
condition and the SSK be deleted because the pubkey hasn't been stored yet.
- Complain earlier and more accurately when no pubkey is found in SSKStore.
- Refactor: Move TYPE_ constants to FreenetStore, and change pointers to BDBFS
to FS where possible.
- Node.storeType: can be "bdb-index" (current BerkeleyDBFreenetStore), or "ram"
(new temporary datastore class that keeps everything including data in RAM).
The latter is useful mainly for many-nodes-in-one-VM simulations. There is no
minimum size for the storeSize if storeType is set to ram.
Requests:
- RequestHandler: Treat an unrecognised completion status as an internal error
- send a local RejectedOverload.
JPEG filter:
- Return the filtered data when we need to modify the image to strip EXIF data!
- Fix two minor bugs that were corrupting filtered data.
Crypto:
- HMAC: Use java's MessageDigest (not our slow pure java SHA256 impl), and
recycle them. Offer utility methods for simple cases, and use them across the
codebase.
Network ID manager:
- Initial attempt by robert to color the network into reasonably well connected
subnetworks. Turned off at the moment.
Dev stuff:
- Logging.
- Imports.
- Better thread names.
- Better toString()s.
- A little paranoia (throw on unrecognised key type etc).
Other commits:
jSite 0.4.9.5 : fix redirect inserting
thingamablog : initial import into repository by nextgens for a fork by dieppe.
dieppe's initial changes: blog type (internet or freenet), allow freenet keys
if it's of type freenet
0.5 branch: up build to 5109.
bombe
jolantern (0.5)
NEOatNHNG
robert
toad
Modified: trunk/freenet/src/freenet/node/Version.java
===================================================================
--- trunk/freenet/src/freenet/node/Version.java 2008-02-08 20:49:23 UTC (rev
17722)
+++ trunk/freenet/src/freenet/node/Version.java 2008-02-08 20:53:19 UTC (rev
17723)
@@ -24,17 +24,17 @@
public static final String protocolVersion = "1.0";
/** The build number of the current revision */
- private static final int buildNumber = 1108;
+ private static final int buildNumber = 1109;
/** Oldest build of Fred we will talk to */
- private static final int oldLastGoodBuild = 1106;
- private static final int newLastGoodBuild = 1108;
+ private static final int oldLastGoodBuild = 1108;
+ private static final int newLastGoodBuild = 1109;
static final long transitionTime;
static {
final Calendar _cal =
Calendar.getInstance(TimeZone.getTimeZone("GMT"));
// year, month - 1 (or constant), day, hour, minute, second
- _cal.set( 2008, Calendar.FEBRUARY, 5, 0, 0, 0 );
+ _cal.set( 2008, Calendar.FEBRUARY, 12, 0, 0, 0 );
transitionTime = _cal.getTimeInMillis();
}
