Author: toad
Date: 2008-02-19 18:18:30 +0000 (Tue, 19 Feb 2008)
New Revision: 18077
Modified:
trunk/freenet/src/freenet/node/Version.java
Log:
1112: MANDATORY THURSDAY. BIG CHANGES. FEATURE FREEZE WILL BE IN EFFECT RIGHT
AFTER THIS COMMIT.
HTL changes:
- Delete nearestSoFar. Keep the old messages with it for now but always set it
to 0.0. Don't reset HTL when nearestSoFar gets closer to the target.
- Reduce pDropAtMaxHTL from 50% to 20% and increase HTL from 10 to 15.
- Next stable build will reduce pDropAtMaxHTL to 10% and set HTL back to 10.
- This is to try to avoid too much network trouble with the migration. Some
disruption is however inevitable.
- Justification: SECURITY! nearestSoFar gives away a *lot* of information about
the requestor.
- Simplify code handling HTL: Always decrement on start of a Sender loop, and
nowhere else. Always use source for the first instance and next for the rest
for determining whether to decrement at max/min. Some places were doing wierd
things like in SSKInsertSender where on a disconnected node we'd actually htl--
and then loop and call node.decrementHTL as well!
NativeThread: Native thread priority (niceness) on Linux. Note that
Thread.setPriority() is a no-op on linux because SCHED_OTHER threads don't
support it.
- Requires a JNI library. contrib/NativeThread contains the source code. All
the library does is expose get/setpriority(PRIO_PROCESS, 0..).
- We have compiled the JNI library for x86 and amd64. freenet-ext.jar #19
features them. The node will update to it.
- NativeThread.java controls this. On Linux, we set the niceness for the thread
(this must be done *on* the thread so is in run()), scaling java's priorities
from MAX = our initial niceness to MIN = 19.
- We detect, complain, and disable the mechanism if we are reniced (we pass a
flag to tell us whether our priority should be the maximum).
- PooledExecutor, PacketSender.queueTimedJob now support passing in a priority
via the PrioRunnable interface (the old code assumed we could change priorities
at any time, we can't on linux, we can only decrease the priority and that only
from actually on the thread).
- Don't run ,0 jobs in PacketSender.queueTimedJob() directly through the
executor if we are using native threads, because we might need to increase the
thread's priority beyond that of the caller.
- PooledExecutor: run from the ticker if we need to boost priority. Pass in a
flag if we are running from the ticker to indicate not to requeue to the ticker.
- Delete some unnecessary Thread.yield()'s which were put in to deal with the
same problem.
Changes to thread priorities etc:
- NativeThread provides 5 different static priority classes, and complains if
it can't get 5 distinct nice levels.
- DiffieHellman precalculation runs at LOW_PRIORITY.
- FEC encoding/decoding runs at LOW_PRIORITY. (Not MIN_, so it will still run
in the presence of idle jobs at nice 19).
- JFK exponentials replacement runs at NORM_PRIORITY.
- The keepalive thread runs at MAX_PRIORITY (but doesn't do anything).
- Background block encoder (for inserts) runs at MIN_PRIORITY.
- PacketSender, UdpSocketHandler, and their watchdogs, run at MAX_PRIORITY.
- RequestSender, RequestHandler, BlockTransmitter, etc (including inserts and
announcements) run at HIGH_PRIORITY (i.e. above client stuff).
- Don't change thread priorities on the fly, for reasons given above.
- Make UdpSocketHandler and watchdog run through the executor.
- Start FECRunner threads through the executor.
- Preserve thread names on jobs run through the ticker.
- Read downloads.dat.gz before starting up the node rather than after, since it
is moderately heavy (especially on RAM), and runs at max priority. Note this is
only the first stage, the second, deserialisation, runs off thread.
Stats page:
- Show count of threads running (and waiting) at each priority level.
- HTML fixes.
L10n:
- German update by tommy.
JPEG filter:
- Support a much wider range of JPEG markers. Unfortunately JPEG is very
complex and the spec is not widely (gratis) available, we need to support more
markers than I had heard on the web.
Request cooldown queue:
- After a key has been requested 3 times, stick it on a queue so it won't be
rerequested for 30 minutes (actually the timeout is set to 5 minutes right now
to shake the bugs out...). ULPRs will probably report the data if found in the
meantime. Means authors of polling clients can just queue keys with
MaxRetries=-1 and they won't cause major load.
- Requires a binary search with fromIndex,toIndex, which is implemented in
Fields.
- Implemented as a big sorted array of keys and one of times: we add to one
end, remove from the other, and when keys are unregistered we remove from the
middle (set key to null to make a hole, it will be tidied up on expansion or on
removing keys at that time).
- Don't remove key from pendingKeys list in CRS.removeRandom(). It will however
be removed when it completes, finds the key etc. But unregister a
SplitFileFetcherSubSegment before registering a new one for a given key.
- SendableGet: Don't call onSuccess(), backdoor coalescing will do it (because
of not unregistering) and we don't want it called twice.
- Various fixes to backdoor coalescing to make it work for splitfiles, and to
make it work when a key has been requested and therefore taken off the queue.
- If a SendableGet is already queued, and a key is added to it (straight
retries, blocks taken off cooldown queue), tell CRS about the new key.
- SendableGet's are required to know when the key will become valid again, so
that it can be easily removed.
- Keys are pulled off the queue and restored to the queue at the beginning of
the RequestStarter loop.
- Pass in scheduler in onSuccess/onFailure/internalError in SendableGet. Keep
it as a member on SingleFileFetcher.
ULPRs/failure table:
- Don't add a node to the failure table if we RejectedOverload the request.
Otherwise an attacker could very quickly and cheaply overwhelm our failure
table.
- Prevent an IllegalStateException and missed callback when transfer coalescing
happens on an asyncGet().
- Fix an NPE when we've tried all nodes for a given key.
Resources:
- Maybe fix a tempfile leak in NativeBigInteger.
- Implement read(byte[]): slight optimisation in CountedInputStream (used by
filters).
- Wake up the ticker if we need to in queueTimedJob().
- Use node.fastWeakRandom in UdpSocketHandler for random-drop.
USKs and ARKs:
- ARK edition numbers must not be negative (parseARK()).
- USK edition numbers too. Add some paranoia (we were accepting the negative
edition number from the broken ARK, trying to fetch it and logging an error
when we failed).
Probe requests:
- Don't allow PROBE:-1.
- Start at best=2.0 not -1 by default. So the comparisons used in Sender will
always work and not leave best at -1.
- Send the probe trace after updating nearest/best.
- Don't start the sender if we lose the upstream connection.
- Fix a NullPointerException causing truncated probes.
Simulations:
- Tweak parameters.
- node.enablePacketCoalescing config option for faster simulations.
Network coloring:
- Bugfixes: Don't add the first peer twice.
- Sort the networks before assigning IDs.
- Locking.
- Probabilistic getConsensus().
Dev stuff:
- Comments.
- Indenting.
- Logging.
- Improve error message when BucketTools.copyFrom/copyTo fails to move enough
data.
- Delete dead code: timeoutTime in FailureTableEntry.
- Thread names.
- PooledExecutor thread names/logging: Thread counter number/counter
name/counter number when given a job should all agree.
- Minor refactoring e.g. RequestStarter.wakeUp().
- Check for _isDone == true and _active also true in UdpSocketHandler watchdog.
- Minor synchronization fixes / delete unnecessary synchronization.
Installer:
- Reduce default nice from 15 to 12 to give the node more space to scale its
priorities within. Most heavy stuff will be at 17.
Web site:
- Fix a broken link.
Thingamablog fork:
- Store edition numbers.
- More UI fixes and bugfixes.
- Connect to the node and generate a keypair when creating a new flog if the
button is clicked.
dieppe
nextgens
robert
toad
tommy
Modified: trunk/freenet/src/freenet/node/Version.java
===================================================================
--- trunk/freenet/src/freenet/node/Version.java 2008-02-19 17:42:19 UTC (rev
18076)
+++ trunk/freenet/src/freenet/node/Version.java 2008-02-19 18:18:30 UTC (rev
18077)
@@ -24,17 +24,17 @@
public static final String protocolVersion = "1.0";
/** The build number of the current revision */
- private static final int buildNumber = 1111;
+ private static final int buildNumber = 1112;
/** Oldest build of Fred we will talk to */
- private static final int oldLastGoodBuild = 1108;
- private static final int newLastGoodBuild = 1111;
+ private static final int oldLastGoodBuild = 1111;
+ private static final int newLastGoodBuild = 1112;
static final long transitionTime;
static {
final Calendar _cal =
Calendar.getInstance(TimeZone.getTimeZone("GMT"));
// year, month - 1 (or constant), day, hour, minute, second
- _cal.set( 2008, Calendar.FEBRUARY, 18, 0, 0, 0 );
+ _cal.set( 2008, Calendar.FEBRUARY, 21, 0, 0, 0 );
transitionTime = _cal.getTimeInMillis();
}
