On Tuesday 07 July 2009 21:10:19 sashee wrote:
> On Tue, Jul 7, 2009 at 10:08 PM, Matthew
> Toseland<toad at amphibian.dyndns.org> wrote:
> > On Tuesday 07 July 2009 20:41:54 sashee wrote:
> >> Maybe I don't see something, but I think it is much more simpler. Here it
> >> is:
> >>
> >> When the processing of RequestElement starts, the ClientRequest object
> >> is already activated, because FCPClient:305 activates all requests,
> >> and puts to the v List, and those object are the very same objects
> >> RequestElement works with. So they are initialized, but maybe the
> >> Fetch(Insert)Context isn't. So activate it. And also the EventProducer
> >> if not activated. As the constructor has the ObjectContainer object,
> >> it can activate. As I see this is used at ClientGet:785. invoked from
> >> RequestElement:122 , and it seems to be working.
> >
> > Yes. But these objects may become deactivated shortly after the constructor
> > ends.
>
> Why can they deactivated? They are actively fetching a file, so they
> are needed. Where are they deactivated?
They are deactivated to save memory. We have lots of files to fetch and we
can't afford to keep everything in RAM.
> >>
> >> The only problem seems to be that these listeners shouldn't be
> >> persisted, they aren't supposed to survive a restart. So I think I
> >> should create a mechanism for transient eventlisteners.
> >
> > Which is pretty much what I was saying. You can't just add a listener to
> > each request because it may be deactivated. But if you add it at e.g. the
> > FCPClient level, you can get a callback while it is active.
> >>
> >> What you think about it?
> >>
> >> sashee
> >>
> >> On Tue, Jul 7, 2009 at 7:56 PM, Matthew
> >> Toseland<toad at amphibian.dyndns.org> wrote:
> >> > On Tuesday 07 July 2009 18:50:58 Matthew Toseland wrote:
> >> >> On Tuesday 07 July 2009 18:26:40 Matthew Toseland wrote:
> >> >> > ? ? ?public void addEventListener(ClientEventListener cel) {
> >> >> > + ? ? ? if(listeners==null){
> >> >> > + ? ? ? ? ? ? ? //Don't know how it can happen, but it did, and
> >> >> > checking for null isn't going to hurt anything
> >> >> > + ? ? ? ? ? ? ? listeners=new Vector<ClientEventListener>();
> >> >> > + ? ? ? }
> >> >> > ? ? ? ? if(cel != null)
> >> >> > ? ? ? ? ? ? listeners.addElement(cel);
> >> >> > ? ? ? ? else
> >> >> > ? ? ? ? ? ? throw new IllegalArgumentException("Adding a null
> >> >> > listener!");
> >> >> > ? ? ?}
> >> >> >
> >> >> > It can't happen for transient requests. For persistent requests, a
> >> >> > persistence-related bug in the code might well cause it to happen
> >> >> > however. :(
> >> >> >
> >> >> Okay, now I see what is going on ...
> >> >>
> >> >> + ? ? ? public RequestElement(ClientRequest clientRequest, int[]
> >> >> columns, String path, ObjectContainer container, boolean
> >> >> advancedModeEnabled, String[] priorityClasses, boolean isUpload,
> >> >> ToadletContext ctx) {
> >> >> ...
> >> >> + ? ? ? ? ? ? ? if (clientRequest instanceof ClientGet) {
> >> >> + ? ? ? ? ? ? ? ? ? ? ? if (((ClientGet)
> >> >> clientRequest).getFetchContext().eventProducer != null) ((ClientGet)
> >> >> clientRequest).getFetchContext().eventProducer.addEventListener(progressListener);
> >> >> + ? ? ? ? ? ? ? } else if (clientRequest instanceof ClientPutBase) {
> >> >> + ? ? ? ? ? ? ? ? ? ? ? if (((ClientPutBase)
> >> >> clientRequest).getInsertContext().eventProducer != null)
> >> >> ((ClientPutBase)
> >> >> clientRequest).getInsertContext().eventProducer.addEventListener(progressListener);
> >> >> + ? ? ? ? ? ? ? } else {
> >> >> + ? ? ? ? ? ? ? ? ? ? ? System.err.println("Dont know this type! type:"
> >> >> + clientRequest.getClass());
> >> >> + ? ? ? ? ? ? ? }
> >> >> + ? ? ? ? ? ? ?
> >> >> clientRequest.getClient().addRequestCompletionCallback(progressListener);
> >> >> + ? ? ? }
> >> >>
> >> >>
> >> >> Anything on the download/upload page is a persistent request. A
> >> >> persistent request is either persistence=reboot or persistence=forever,
> >> >> but the latter is far more common. These requests are *stored in a
> >> >> database*, and are not always in memory. Using them as variables will
> >> >> cause them to be pinned in memory but will not ensure they stay
> >> >> "activated" i.e. that all their fields are loaded and non-null. The
> >> >> only safe way to access them is to schedule a job on the database
> >> >> thread (the DBJobRunner, aka NodeClientCore). Hence, the
> >> >> SimpleEventProducer may well be null, and even if it isn't, it may not
> >> >> be activated, meaning the listeners array may be null.
> >> >>
> >> >> Hence subscribing to the SimpleEventProducer and listening for events
> >> >> is not going to be sufficient. You need to implement some sort of
> >> >> global hook, so that when an event is generated against a request in
> >> >> the global queue (only the global queue is shown on the
> >> >> downloads/uploads pages), you get a notification, at that point, while
> >> >> the request is still active.
> >> >>
> >> >> What you need to do is as follows:
> >> >> - FCPServer.globalRebootClient is the half of the global queue that is
> >> >> persistence=reboot, that is, the part that isn't persisted to the
> >> >> database. You can ignore it for all practical purposes, but you should
> >> >> really deal with it when you get around to it.
> >> >> - FCPServer.globalForeverClient is the half of the global queue that is
> >> >> persistence=forever, and in practice has almost all global requests
> >> >> (requests or inserts shown on the downloads/uploads pages). It is never
> >> >> deactivated.
> >> >> - This is an FCPClient.
> >> >> - You need to create a callback method and a registry of listeners on
> >> >> FCPClient that gets fed every FCP event, at the time it is created,
> >> >> along with the ClientRequest which created it. This happens in
> >> >> ClientGet.receive, ClientPutBase.receive, and probably that's about it;
> >> >> check the call hierarchy for the constructors for e.g.
> >> >> SimpleProgressMessage if you are concerned. This always happens on the
> >> >> database thread, or at least, if it's not on the database thread, it
> >> >> schedules a job on the database thread to send the message (e.g. in
> >> >> ClientPutBase.trySendProgressMessage). So call the callback at that
> >> >> point.
> >> >> - The callback should go to some sort of manager object, which
> >> >> determines which RequestElement(s) want the message. Or you can just
> >> >> register all of them but it will be slower that way.
> >> >> - Don't store the pointers, store the UID of the request (this is
> >> >> because we don't want to pin the request object in memory). This is an
> >> >> long value for a specific request in the database which doesn't change
> >> >> unless we defrag (which is an offline operation). You can get it by:
> >> >>
> >> >> container.ext().getID(<object>).
> >> >>
> >> >> Please let me know if you need any more information on this! The db4o
> >> >> zip file contains javadocs, you can get it from db4o.com, we use
> >> >> version 7.4.
> >> >>
> >> > Also, you should not store the ObjectContainer, and should not access
> >> > the database except on the database thread (i.e. when an ObjectContainer
> >> > has been passed in ultimately from a DBJob).
> >> >
> >> > _______________________________________________
> >> > Devl mailing list
> >> > Devl at freenetproject.org
> >> > http://emu.freenetproject.org/cgi-bin/mailman/listinfo/devl
> >> >
> >> _______________________________________________
> >> Devl mailing list
> >> Devl at freenetproject.org
> >> http://emu.freenetproject.org/cgi-bin/mailman/listinfo/devl
> >>
> >>
> >
> >
> >
>
>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 835 bytes
Desc: This is a digitally signed message part.
URL:
<https://emu.freenetproject.org/pipermail/devl/attachments/20090707/873e02ae/attachment.pgp>
