Hi Kevin, I followed your suggest and played with libcurl in multi mode.. it's cool ! I tried some examples found in the net.. but I need more flexibility (e.g. an arbitrary number of easy handles) So I wrote a simple wrapper to deal with the async curl calls with ease, 'cmhw' (Curl Multi Handle Wrapper) The idea is to have a component which take care of all the aspects of the multi curling, you just need to create an instance and add the urls you want to fetch, and it just do the job. it's an early beta version, there's a lot of work which should be done (e.g. all the possible error checking, code optimization / safety checks, a better api interface, a better name... ) but it works. could it be useful for the gadget processing library?
an example: /* our main struct. it holds the curl multi handle and the hash of all the needed resources */ cmhw_context_t *ctx = NULL; /* represents a single remote resource. holds an 'easy' curl handle and a chunk of memory for the fetched content data */ cmhw_item_t *item = NULL; /* init... */ ctx = cmhw_init(); /* add some request (with an optional name, just to get them later with ease) */ cmhw_add_url(ctx, "google", "http://www.google.com";); cmhw_add_url(ctx, "myspace", "http://www.myspace.com";); /* download them all! */ cmhw_perform(ctx); /* make something interesting with our results */ item = cmhw_item_get(ctx, "google"); fprintf(stderr, "content for '%s'(%s):\n%s\n#####\n\n", item->name, item->url, item->chunk->memory); item = cmhw_item_get(ctx, "myspace"); fprintf(stderr, "content for '%s'(%s):\n%s\n#####\n\n", item->name, item->url, item->chunk->memory); /* cleanup.. like a nice application... */ cmhw_cleanup(ctx); please let me know what you think about this and how I could improve it. thanks leo On Mon, Jun 23, 2008 at 1:42 AM, Kevin Brown <[EMAIL PROTECTED]> wrote: > On Sun, Jun 22, 2008 at 3:16 PM, Leonardo Foderaro <[EMAIL PROTECTED]> > wrote: > >> Hi, >> I'm trying to write a simple prototype of the library we talked about >> some days ago. >> It is a really early version at the moment.. just some experiments (of >> course the lucky codename is test.c ) >> >> What I'm using: >> - libcurl for the remote content fetching >> - libxml2 for the gadget xml parsing >> - some data structures from the glibc2.0 (e.g. GHashTable) for storing >> dynamic variables >> - a lexical scanner (generated using flex) for the hangman substitution >> >> Actually I'm able to fetch a gadget from a remote URL (or a local >> file), load it into the XmlParser and perform the hangman vars >> substitution of the Content node through the flex scanner (only >> __MODULE_ID__ for now). >> >> Of course there's a lot of work to do. >> >> Let me know what could be the next logical step.. I think I should >> construct a C representation of the gadget with all its properties, >> right? (it just writes to stdout at the moment) > > > Yeah, you'll probably want a struct similar to the JSON output from > /metadata in Shindig. As a library user / module writer, that's what I'd > want to consume. > > Definitely try libcurl in multi mode -- it's quite possible to implement a > high concurrency http client in this model using async events -- you can use > libevent for this. > > The ideal library would probably look something like this: > > os_request *req = os_request_create(); > req->locale = ... > os_request_add_gadget(req, "http://example.org/gadget.xml";); > os_get_data(req, &callback); > > void callback(os_request *req, os_response *resp) {...} > > Rendering could then be implemented as a consumer of this library. > > > >> >> thanks to all, >> leonardo >> >
