On Wednesday 19 March 2008 22:48, juergen urner wrote: > Matthew Toseland schrieb: > > On Wednesday 19 March 2008 14:27, juergen urner wrote: > > > >> Just dropping by to tell, that I finally resign on writing a highlevel > >> Python wrapper > >> for the protocol. There is still too many show stoppers in there to > >> write it nice and > >> clean. > >> > >> > >> I'd like to give some feedback on what made my live hard. Maybe s.o. can > >> make > >> use of it. In no particular order.... > >> > >> ------------------------------------------------------------------------------------------------------------- > >> [Global queue and persistence] > >> Already reported. Maybe one last note regarding this. If apps feel the > >> need to share data > >> it should be up to them to organize this. > >> > > > > Noted. As you know we disagree. > > > >> [Persistent requests vs. non persistents] > >> x. one may want to modify or remove non persistent requests aswell > >> this should eliminate all the case switches if > >> Persistence==connection else... > >> x. there should be an explicite way to stop, resume all kinds of requests > >> > > > > I agree, I just haven't got around to dealing with it yet. There didn't seem > > to be much demand. > > > >> [DataFound vs. AllData] > >> Another case switch that looks rather unnecessary. AllData is not send > >> for persistent > >> requests. > >> > > > > Not true. AllData is sent with the data, for any request where > > returntype=direct. We don't send the data for persistent requests until the > > client asks for it. > > Sorry, sloppy tying. For persistents you have to explicitely query it by > sending > GetRequestStatus. To me it looks like a special case that breaks > consistency.
You mean we should always force the client to do GetRequestStatus? What's your proposed solution? > > >> Looks loke a very special case to me, maybe some Frost. Imo, > >> if there > >> is a need to split data fetching and whatever-client-wants-to-do-with-data > >> apart, doing it explicitely could be much clearer. Something like... > >> > >> client --> node > >> GetData > >> End > >> > >> node-->client > >> DataFound > >> End > >> > >> client-->node > >> GetDataAs > >> Filename= > >> Direct= > >> End > > > > This is pretty much what we do, with the exception that you have to specify > > where to write the data to on starting the request, and have to do a DDA > > verification. I'm not entirely averse to the possibility of doing a fetch with returntype=direct, and then telling the node where to write the data to after it's been fetched. However, it's more efficient (disk space) to tell the node in advance where the data should go. > > > >> [SimpleFiledSet and Fcp] > >> This is a nasty one: dotted names. It may be easy or not for some > >> languages to deal with > >> this data type. Actually it is easy in python, but doing additional > >> processing based on > >> message signature (like type conversions) is pretty painful and costy. > >> Flat messages > >> shopuld be way easier to handle. Already made a suggestion regarding this... > > > > I really don't understand what the difference is. We provide a Count. If you > > are parsing into a tree structure, it's trivial. Even if you're not, you've > > got the Count, so you can allocate the array before you parse the sub-items. > > Given a multiplexable protocol, using separate messages would mean having an > > identifier in each one which would be just as messy. The only reason to do it > > the way you propose would be to avoid very large messages. > > Thank God no pre-allocation here :-) :) > > ...an identifier would be required, came to my mind too. Not nice. > But you don't have to multiplex here if the number of subitems can > be assumed to be small enough for clients to deal with at once. In which case they are submessages rather than messages. In which case why not simply include them in the parent message? > > >> ContainerMessage > >> NItems=N > >> End > >> Item > >> End > >> > >> [Message signatures and consts] > >> (already mentioned before) a good to have would be a file containg all > >> consts and message > >> signatures used in Fcp. > > > > You mean error codes? Or what? > > <PseudocodePython> > > class Message: > name = 'Foo' > Fields = { > 'Bar': DataTypeBool, > } > > ErrorCode = 1 > Const = 1 > > </PseudocodePython> > > This is could be fed to a dedicated parser, saving much work > for any protocol wrapper in any language. Ahhh. Well, patches are welcome. So are bug reports (on the bug tracker). > > > >> It should go into very detail regarding types > >> aswell. TypeIPAddress. TypeIPAddressList (...) > >> This would save much typing work and could help in autogenarating huge > >> parts of protocol wrappers. > > > > I don't understand. > >> [Organize requests hirarchically] > >> This may sound exotic, but it could help in organizing requests. > >> Especially when it comes to restoring > >> requests, where it may be desireable to process requests on demand. > >> Something like > >> GetRootRequest ..ListChildRequests > > > > It *is* exotic. > > Probably you are right. I thought you didn't like hierarchy? You seem to have a lot of trouble with it above. > > >> [Reserve DataLenght as indicator of data attatched to a message] > >> (Already mentioned) makes message parsing much easier if one can rely on > >> this field being > >> reserved as message metadata. > > > > What else can it mean? DataLength always means the data is the given length. > > And if the message ends with Data, there is data attached of that length. > > IIRC we may use DataLength in some places where there is no data attached, > > but in that case we don't end with Data. > > if ...else Right, but do you really want DataFound to not tell you what the length of the data is? > > >> [ConfigData, NodeData, Peer should be traversable] > >> See [Organize requests hirarchically] > > > > Whatever. Very few apps use that stuff. > > Mine does. What do you mean by traversible? They should be multiplexible, in that they should have an Identifier. > > >> [Drop Started field in Persistent* messages] > >> Maybe use simple progress instead, and / or some dedicated field to > >> indicate request status. > >> This could be helpful anyways if requests may be stopped, resumed (...) > >> at some point > > > > Won't fix. If you don't like the field, ignore it. > def restoreClientGet(msgPersistentGet): > if 'Started' in msgPersistentGet.fields: > del msgPersistentGet.fields['Started'] > msgClientGet = ClientGet(fields=msg.fields) LOL why? > > >> [Dedicated Result field in messages] > >> In case something goes wrong, it would be nice to have the message > >> casing the error > >> at hand right away. ++ ProtocolError, FetchError, InsertError are not > >> very handy. They > >> should be numbered consecutively. I found myself writing a wrapper to > >> emulate this > >> (uniformity). > > > > You mean the different error code types should be in different numerical > > ranges? > > > Yes. Afaics for clients it makes no difference if an error is a protocol > error > or a fetch error. > > ProtocolErrorMin = 0 > ProtocolErrorMax = 1000 > FetchErrorMin = ProtocolErrorMax + 1 > (...) A protocol error indicates a bug in the library, or something that the library can deal with (e.g. you need to do DDA verification). A fetch error indicates a Freenet level problem. A client should never see a protocol error ... I suppose that's not strictly true any more... DDA stuff you can generally avoid, since you should be handling that transparently in the library, but if the client passes you a malformed URL, you need to pass that back to the client; if it tries to access an unknown plugin, peer with itself etc, you may need to pass it back ... hmmm. At this point, some clients may be relying on error codes, so I don't really think changing it is a good idea... > > >> [ClientToken field for all messages] > >> It may be desireable to keep track of all messages and responses. Like > >> for example GetConfig > >> and Config to tell when all tasks a user has posted are completed. > >> ClientToken would come in > >> handy. > > > > I agree that some messages need an Identifier. But please file specific bugs, > > or better provide specific patches. > No Java for me. Off cause you could embedd a python interpreter ;-) > Rule of thumb is: it is useful in all messages. Right. Well I will get around to reviewing them eventually. -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: not available URL: <https://emu.freenetproject.org/pipermail/tech/attachments/20080320/5eaede2e/attachment.pgp>