On Sat, Jul 25, 2009 at 3:53 PM, Matthew Toseland<[email protected]> wrote: > The basic answer is that plugins should be written to a defined, fixed, > sandboxed, external API. But there is no such API. We will need to implement > one eventually. But this takes considerable time, which could be spent on > things which give a more direct benefit to end users in terms of usability, > performance or security.
My general complaint is not that the plugin api isn't well defined and perfect, though that would be nice. It's that what does exist isn't documented. My current take on coding for Freenet is that the API isn't perfect, but that I could work with it if I knew what it did. However, figuring out what it does requires reading significant quantities of Freenet source code as often as not. > > On Thursday 23 July 2009 00:12:15 Evan Daniel wrote: >> I've been slowly working on writing a plugin for Freenet over the past >> couple weeks. (Details aren't particularly relevant to what follows; >> for the curious, it's a microblogging / chat application aiming at >> keeping latencies as low as possible. RFC available at >> u...@cf9ctasza8w2jafeqmiln49tfrpdz2q5m68m1m5r9w0,NQiPGX7tNcaXVRXljGJnFlKhnf0eozNQsb~NwmBAJ4k,AQACAAE/Fritter-site/1/ >> ) >> >> Having not written much actual Freenet code before, I'm learning a lot >> about how Freenet works in the process -- which is harder than it has >> any reason to be. Why? NOTHING IS DOCUMENTED. For example, after >> retrieving some text from Freenet, my plugin would like to display it >> on a web page, including filtering it so that it doesn't break >> anything, even in the case of malicious input. The method >> HTMLEncoder.encode() sounds like it ought to do that. Let's take a >> look at the Javadoc: >> >> encode >> >> public static java.lang.String encode(java.lang.String s) >> >> That's it. Nothing about what the method is supposed to do. So, >> after hunting through several source files (when reading the Javadoc >> should have sufficed), I have a fairly good sense of what the method >> does. I'm pretty sure it doesn't do precisely what I'm looking for >> (though the HTML specs are complicated enough, and I don't know them >> well enough, that I'm not certain either way). Neither the Javadoc >> nor inline comments reference any standard that it is trying to >> conform to. If there was a contract specified, I could fairly easily >> determine whether that contract matched what I needed -- and therefore >> whether I should be writing my own function or submitting a patch (or >> whether I'm misreading the relevant specs, for that matter). > > HTMLEncoder is supposed to encode text so that it can be safely inserted into > HTML, as its name suggests. It is used widely and hopefully it is sufficient. See https://bugs.freenetproject.org/view.php?id=3335 >> >> If this were an isolated incident, it wouldn't matter much. It isn't. >> It is the norm for Freenet. For a platform whose primary impediment >> to wider adoption (IMO, of course) is a lack of things to do with it, >> rather than a lack of underlying functionality, this is a problem. I >> haven't tracked it, but I wouldn't be surprised if I've spent nearly >> as much time trying to figure out how the plugin API works (or even >> which classes it consists of) as I have actually writing code. > > That is an interesting theory. I guess since we have a load of money from > Google it would be possible to spend some time on the plugin API??? I think it would be *very* wise to spend time on the Javadoc for what currently exists, and making assorted small improvements. (For example: InsertContext should be cloneable, like FetchContext.) As far as writing a better API, or sandboxing, I'm not as certain. The current one is probably workable, and there are lots of important things that need doing. >> >> In case I haven't made my point yet, here are a few questions I've >> had. Can anyone point me to documentation that answers them (Javadoc >> or wiki)? I've spent some time looking, and I haven't found it. Most >> (but not all) I've answered for myself by reading lots of Freenet code >> -- a vastly slower process. Some of them I believe represent bugs. >> >> How do I make a request keep retrying forever? > > Set maxRetries to -1 Yeah, I figured that out eventually -- by asking in IRC, I think. But it isn't documented. It would take all of one sentence in the Javadoc to make it easy for me or any other plugin author to find, and that's a lot easier than answering the question every time someone has it. > >> How do I determine whether a ClientGetter represents an active request? > > Dunno. https://bugs.freenetproject.org/view.php?id=3336 > >> Are there any circumstances under which ClientGetCallback.onSuccess() >> will be called more than once, and do I need to handle them? > > Shouldn't be, if there are it's a bug. It happens on every fetch, afaict. https://bugs.freenetproject.org/view.php?id=3331 > >> Why doesn't ClientGetCallback.onFetchable() get called (more than a >> trivial time) before onSuccess()? > > You mean ClientPutCallback, right? This should probably be gotten rid of, > it's only relevant if EarlyEncode is enabled. Yes, I do. Thanks. > >> How do I guarantee that an insert won't generate a redirect? > > Stay within the size limit for the key type, don't set a MIME type or target > filename (you *can* set an SSK filename). It would be nice to have flags like FetchContext.allowSplitfiles and FetchContext.followRedirects so that I don't have to verify after the insert. Verifying before the insert is harder, especially with compression turned on. > >> What, if anything, should I be doing with this "ObjectContainer >> container" that gets passed around everywhere? > > If you see that you are looking at a fairly low level API not really intended > for plugins. Unfortunately the official API (HLSC) is very limited. For > transient requests, the container is always null. I'm using the HLSC. Starting a fetch (or insert) with the HLSC needs a callback, which needs to handle the container. For that, I'm just ignoring it. Changing the priority class on the files requires passing it in to ClientPutter.setPriorityClass. Speaking of which, should the priority class be a field in FetchContext / InsertContext? > >> Under what circumstances will I see a FetchException or >> InsertException thrown when using the HighLevelSimpleClient? > > When a fetch or insert fails? :) When will that happen? Specifically, is it ok for me to assume that if that happens something has gone very wrong, and I can either fail-fast at a rather high level, or simply continue on a best effort basis ignoring the error? Or do I need to be able to recover and continue in an intelligent manner? > >> Why does the Hello World plugin break if I turn it into a FredPluginHTTP? > > No idea offhand. I eventually figured this one out (and submitted https://bugs.freenetproject.org/view.php?id=3310 along the way). The HelloWorld plugin has a loop that runs forever inside of runPlugin that generates a log event every second. Apparently, this is acceptable behavior only in non-HTTP plugins. HTTP plugins are expected to return from runPlugin before they finish loading. Suffice to say this behavior is both undocumented and weird. >> >> None of these seem like overly complex or unexpected questions for >> someone trying to write a plugin for Freenet. They should all be >> answerable by reading documentation. >> >> On a closely related note, here are a few related issues -- imho they >> need fixes in the code, but proper documentation of the actual >> behavior would have left me far less confused when debugging: >> FreenetURI.setDocName(String) doesn't. > > Yes it does, but FreenetURI is immutable or almost immutable - it returns the > new URI. It returns a new URI. That is not the same as setting the docName variable on this URI. I admit my exposure to formal naming standards is (very) limited, but I'm used to setSomething methods having a contract of setting something internal to the object that can be accessed by the corresponding getSomething method. Having a setSomething method on a mostly-immutable object is therefore rather weird. Furthermore, this behavior is not documented, either specifically for this function or the fact that FreenetURIs are mostly immutable. I spent some time chasing the bug, and for a while my code mostly handled URIs as Strings, only turning them into URIs when required. Eventually I asked on IRC and got the answer, and now my code is simpler and better, but I wasted an annoying amount of effort on something that would have been prevented by simply having "@return A new FreenetURI created from this URI with the docName set to the requested value." > >> Creating a ClientSSK from a FreenetURI and then attempting to insert a >> file to it throws an exception about "wrong extra bytes" even though >> the extra bytes are unchanged from the FreenetURI the node generated >> for me. > > You need an InsertableClientSSK. Again you are using a low level interface > and complaining that it isn't well documented. I agree it should be, but it > is internal code, not external API. Sadly right now all we have pretty much > is internal code... Because of a certain lack of documentation, I started with the approach of picking a plugin I'd looked at a bit before and copying it; it happened to use the lower level stuff, so I followed suit. Now that I'm using the HLSC, I don't have this particular problem. Regardless, "Wrong extra bytes" is an entirely inappropriate exception text for this error. Something about how the ClientSSK doesn't support inserts would be far more appropriate. I can't imagine it would be any more helpful debugging internal Freenet code than plugins. > >> At this point, I think I have a much better understanding of why >> Freenet has so little software that makes use of it, despite the fact >> that Freenet itself seems to work fairly well. > > Sorting out the plugin API might take months. Do we have the time? Having a coherent and pristine plugin API would be wonderful, but it's not what I'm complaining about. I'd be very happy if what existed was documented! I honestly don't know whether a good plugin API should take precedence over other things. For now, my inclination would be to mark it as a requirement for 1.0. Evan Daniel _______________________________________________ Devl mailing list [email protected] http://emu.freenetproject.org/cgi-bin/mailman/listinfo/devl
