I include some documentation on what I am working:
Notes on changing Library API /WoT Requirements: ============= whatever you do to refactor Library will still need to support Spider Current main problem:? ===================== Library can only handle, one tree at once (and it generates the USK for that itself).? We need: ======= -Generalise both existing messages in Library API -Modify the frequency of properties update in WoT (see things done by WoT) -an FCP command to fetch the index so we can show it on the Spider UI, NOT SURE see with toad. -Mid to Long term: To search USK of other indexes (from other identities) Things done by Curator? ====================== Curator gets all identities of the local node from WoT plugin.? It offers the client to select the identity that will share indexes. Each identity only has one index (do we need more?). Each identity may have a list of topics to curate (terms).? Curator derives the index' key from the identity. The index will use as prefix the Request URI and change the file's name part (e.g. "index"). It passes to Library new entries to populate indexes. Each entry corresponds to an on-freenet webpage or a on-freenet document. Eventually it may read info automatically from the webpage to share (page description, usk, etc). In the case of documents, the user need to complete the form manually. Each entry is in a buffer, then we send pushBuffer message to Library.? It provides Library with the USK for the index. Things done by Library ====================== Maintains on disk different indexes. Basically mergeToDisk() parses the data (which is a SimpleFieldSet) and adds it to the on-disk tree? Currently it only handles one for Spider. Library generates the USK (randomly) and uploads it to Freenet. At the moment, Library API can handle two messages designed for Spider: 1)getSpiderURI getSpiderURI calls getPublicUSKURI(): handle -> handleGetSpiderURI() -> getPublicUSKURI() getPublicUSKURI() is a method on SpiderIndexUploader.java It is the method in SpiderIndexUploader. Is is useful to know where is the tree uploaded on Freenet (currently a random USK). Of course the tree itself is uploaded to a CHK - uploading a pointer to a USK is something on top 2)pushBuffer The pushBuffer FCP command takes a Bucket of data, which is a series of pages ... eventually mergeToDisk() gets called (the outer layers are performance hacks/buffering) Things done by WoT (mail p0s) ================== We need to update the USK edition number periodically. We need update because USK is not so efficient to get the last version soon. It may take some time to get it.? Updating an identity's property causes an identity re-insert (i.e. upload the identity to freenet). We need WoT to update the IndexRoot property frequently but not every time we update indexes. We don't want to cause so many re-inserts. So the optimisation is that our property gets updated only once, in the daily identity re-insertion triggered by WoT. p0s can change this behaviour. Leuchtkaefer
