On Wednesday 26 August 2009 19:27:54 Evan Daniel wrote: > On Wed, Aug 26, 2009 at 1:30 PM, Matthew > Toseland<[email protected]> wrote: > > On Wednesday 26 August 2009 18:21:11 Evan Daniel wrote: > >> On Wed, Aug 26, 2009 at 11:48 AM, Matthew > >> Toseland<[email protected]> wrote: > >> > On Saturday 25 July 2009 22:05:40 Evan Daniel wrote: > >> >> On Sat, Jul 25, 2009 at 3:53 PM, Matthew > >> >> Toseland<[email protected]> wrote: > >> >> > On Thursday 23 July 2009 00:12:15 Evan Daniel wrote: > >> >> >> How do I guarantee that an insert won't generate a redirect? > >> >> > > >> >> > Stay within the size limit for the key type, don't set a MIME type or > >> >> > target filename (you *can* set an SSK filename). > >> >> > >> >> It would be nice to have flags like FetchContext.allowSplitfiles and > >> >> FetchContext.followRedirects so that I don't have to verify after the > >> >> insert. Verifying before the insert is harder, especially with > >> >> compression turned on. > >> > > >> > IMHO that is overkill, we should fix the bug if we are creating > >> > redirects in such cases though. > >> > >> The size limits are on compressed sizes; I don't see an easy way to > >> check whether a given chunk of uncompressed data will fit. > >> > >> After the fact verification would provide a way to find such bugs > >> earlier. Fail fast is better than letting weird junk get sent out and > >> then having to chase that down. > > > > Hmmm, ok, file a bug if you need this. It is probably more efficient to > > compress it yourself and use the lower level APIs mind you. > > Well, I measure efficiency in lines of code and bugs I have to hunt > down later, not CPU time :) > > For now, I plan to just require the uncompressed size be small enough > and not worry about it.
But in future you'd like an option so it fails if it's too big so you can make it smaller. File a bug. > > >> > >> >> >> What, if anything, should I be doing with this "ObjectContainer > >> >> >> container" that gets passed around everywhere? > >> >> > > >> >> > If you see that you are looking at a fairly low level API not really > >> >> > intended for plugins. Unfortunately the official API (HLSC) is very > >> >> > limited. For transient requests, the container is always null. > >> > > >> > Right. I have tried to document this to some degree on the new javadocs. > >> > >> Thanks. > >> > >> >> > >> >> I'm using the HLSC. Starting a fetch (or insert) with the HLSC needs > >> >> a callback, which needs to handle the container. For that, I'm just > >> >> ignoring it. Changing the priority class on the files requires > >> >> passing it in to ClientPutter.setPriorityClass. Speaking of which, > >> >> should the priority class be a field in FetchContext / InsertContext? > >> > > >> > The priority class is a property of the ClientGetter or ClientPutter; we > >> > could pass it in through FetchContext perhaps, but I'm not sure it would > >> > be beneficial. > >> > >> OK. > >> > >> >> > >> >> >> Under what circumstances will I see a FetchException or > >> >> >> InsertException thrown when using the HighLevelSimpleClient? > >> >> > > >> >> > When a fetch or insert fails? :) > >> >> > >> >> When will that happen? Specifically, is it ok for me to assume that > >> >> if that happens something has gone very wrong, and I can either > >> >> fail-fast at a rather high level, or simply continue on a best effort > >> >> basis ignoring the error? Or do I need to be able to recover and > >> >> continue in an intelligent manner? > >> > > >> > Not sure what you're asking here. Failures range from data not found to > >> > the data was corrupt to internal errors. > >> > >> On inserts, I presume my code produces nice clean data. If it isn't, > >> that's a bug, and as long as I detect it rapidly and log the message, > >> I don't really care whether it can recover intelligently; either way I > >> need to fix the bug. If it fails due to the node crashing or similar, > >> again I don't really care and am happy with just presenting the user > >> with an error message and not recovering from it. If, however, they > >> fail in the normal course of events (network load, etc), then I need > >> to handle that. > >> > >> Similarly, I would think that the only errors I should see on a > >> request with maxretries=-1 should be due to malformed data on the > >> network or node problems. The latter I want to fail fast on, and the > >> former I'm happy to simply blame the inserter and give up on rather > >> than handling gracefully. > > > > Generally it would be node problems on an insert yes. On a fetch it could > > be node problems, the data might just not be there, the data might have > > been corrupt when it was inserted... > >> > >> So basically, if I take the simplistic approach described above, are > >> there any errors that I actually need to *handle*, as opposed to > >> simply logging and then halting or ignoring? > > > > Generally for inserts you don't need to do much. For requests obviously > > failures are the common case. > > Why would a request with maxretries=-1 fail, aside from node internal > problems (that I don't plan to handle gracefully at this point) or > malformed data from the network (which I plan to log for debug but > mostly ignore)? > > Again, most of this is a request for documentation; I'd like to be > able to tell from reading javadoc what error conditions I need to > handle intelligently, and which can simply be logged and ignored. If > the error condition is because of a bug in my code, I need to know > about it and fix the bug, but at this stage in development it's ok for > such bugs to cause the plugin to fail completely, as long as I get a > helpful stack trace. That's simpler than trying to handle weird error > cases that shouldn't ever happen in a graceful manner. If MaxRetries=-1, it will only fail on an internal error or if bad data was inserted. I have documented this in FetchContext. > > >> >> >> Why does the Hello World plugin break if I turn it into a > >> >> >> FredPluginHTTP? > >> >> > > >> >> > No idea offhand. > >> >> > >> >> I eventually figured this one out (and submitted > >> >> https://bugs.freenetproject.org/view.php?id=3310 along the way). The > >> >> HelloWorld plugin has a loop that runs forever inside of runPlugin > >> >> that generates a log event every second. Apparently, this is > >> >> acceptable behavior only in non-HTTP plugins. HTTP plugins are > >> >> expected to return from runPlugin before they finish loading. Suffice > >> >> to say this behavior is both undocumented and weird. > >> > > >> > Ahh, so FredPluginHTTP is assumed to be FredPluginThreadless. > > > > FredPluginHTTP now extends FredPluginThreadless, to clarify this. > >> >> > >> >> >> None of these seem like overly complex or unexpected questions for > >> >> >> someone trying to write a plugin for Freenet. They should all be > >> >> >> answerable by reading documentation. > >> >> >> > >> >> >> On a closely related note, here are a few related issues -- imho they > >> >> >> need fixes in the code, but proper documentation of the actual > >> >> >> behavior would have left me far less confused when debugging: > >> >> >> FreenetURI.setDocName(String) doesn't. > >> >> > > >> >> > Yes it does, but FreenetURI is immutable or almost immutable - it > >> >> > returns the new URI. > >> >> > >> >> It returns a new URI. That is not the same as setting the docName > >> >> variable on this URI. I admit my exposure to formal naming standards > >> >> is (very) limited, but I'm used to setSomething methods having a > >> >> contract of setting something internal to the object that can be > >> >> accessed by the corresponding getSomething method. Having a > >> >> setSomething method on a mostly-immutable object is therefore rather > >> >> weird. > >> > > >> > Suggestions for better naming? > >> > >> Not off hand; I'll think about it. > >> > >> >> Furthermore, this behavior is not documented, either specifically for > >> >> this function or the fact that FreenetURIs are mostly immutable. I > >> >> spent some time chasing the bug, and for a while my code mostly > >> >> handled URIs as Strings, only turning them into URIs when required. > >> >> Eventually I asked on IRC and got the answer, and now my code is > >> >> simpler and better, but I wasted an annoying amount of effort on > >> >> something that would have been prevented by simply having "@return A > >> >> new FreenetURI created from this URI with the docName set to the > >> >> requested value." > >> > > >> > It is documented now. > >> > >> Thanks. > >> > >> >> >> Creating a ClientSSK from a FreenetURI and then attempting to insert > >> >> >> a > >> >> >> file to it throws an exception about "wrong extra bytes" even though > >> >> >> the extra bytes are unchanged from the FreenetURI the node generated > >> >> >> for me. > >> >> > > >> >> > You need an InsertableClientSSK. Again you are using a low level > >> >> > interface and complaining that it isn't well documented. I agree it > >> >> > should be, but it is internal code, not external API. Sadly right now > >> >> > all we have pretty much is internal code... > >> >> > >> >> Because of a certain lack of documentation, I started with the > >> >> approach of picking a plugin I'd looked at a bit before and copying > >> >> it; it happened to use the lower level stuff, so I followed suit. Now > >> >> that I'm using the HLSC, I don't have this particular problem. > >> > > >> > I have added some more javadocs. > >> >> > >> >> Regardless, "Wrong extra bytes" is an entirely inappropriate exception > >> >> text for this error. Something about how the ClientSSK doesn't > >> >> support inserts would be far more appropriate. I can't imagine it > >> >> would be any more helpful debugging internal Freenet code than > >> >> plugins. > >> > > >> > It is not immediately obvious how that exception would come about. > >> > Without a code sample I don't see how I can fix it. > >> > >> I don't have a code example handy, but roughly: > >> Create a String representation of a SSK insert URI. Create a > >> ClientSSK from that. Attempt to insert to it. > > > > Attempt to insert it how? > > Um, I don't remember. I changed all my code to use the HLSC. How > would you insert to an InsertableClientSSK? I got the problem > originally by modifying some piece of code that I think you suggested > as an example for how to turn a String into some sort of insertable > data type, but now I can't find what I was looking at. Sorry :/ > > (Should all this be on devl?) Yes. > > Evan Daniel
signature.asc
Description: This is a digitally signed message part.
_______________________________________________ Devl mailing list [email protected] http://emu.freenetproject.org/cgi-bin/mailman/listinfo/devl
