Re: [Haskell-cafe] Google Android
2008/9/24 Galchin, Vasili <[EMAIL PROTECTED]>: > Do there currently (or in the works) exist FFI bindings for Google's > Android API? The Android API is obviously a Java API, additionally Android will only execute DEX bytecode programs, so you would first need a way to compile Haskell to Java, Java bytecode or DEX directly. A Google search for "haskell java" turns up at least one good candidate[1], but if you manage to get that working well, binding the APIs is a rather trivial task ;) [1] http://www.cse.unsw.edu.au/~pls/thesis-topics/ghcjava.html AGL -- Adam Langley [EMAIL PROTECTED] http://www.imperialviolet.org ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] Re: Iteratee-based IO
On Sat, Sep 20, 2008 at 8:17 PM, Don Stewart <[EMAIL PROTECTED]> wrote: > Wonderful. Thanks Oleg, I'll be studying these on the trip up to > Victoria. See you there! Something similar is available in binary-strict[1], although there I have (versions of) IE_Done, IE_Cont and (additionally) IE_Fail, for indicating a parse failure. The actual implementation is unpleasant, although functional. [1] http://darcs.imperialviolet.org/darcsweb.cgi?r=binary-strict;a=headblob;f=/src/Data/Binary/Strict/IncrementalGet.hs -- Adam Langley [EMAIL PROTECTED] http://www.imperialviolet.org ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] control-timeout with gtk
On Thu, Sep 18, 2008 at 11:38 AM, Marco Túlio Gontijo e Silva <[EMAIL PROTECTED]> wrote: > Maybe it's better that they stay there, even with nobody maintaining > them. Except that I feel shitty everytime someone someone emails me with questions or problems that I don't have the time to deal with. It's a constant cost for my name to be associated with them and, if nobody cares enough to take the maintainership, they should probably die - Hackage has a lot of packages these days. Do you want control-timeout? :) AGL -- Adam Langley [EMAIL PROTECTED] http://www.imperialviolet.org ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] control-timeout with gtk
On Thu, Sep 18, 2008 at 10:05 AM, Judah Jacobson <[EMAIL PROTECTED]> wrote: > Just a guess, but this might be a problem with control-timeout's use > of the unsafePerformIO global variables hack. It's missing the > standard NOINLINE annotations which prevent multiple copies of the > global variable from being created. See the haddock docs for > System.IO.Unsafe.unsafePerformIO for more information. That might well be it, should be an easy fix to try. I'm afraid that I don't have enough time to do justice to my packages on Hackage these days so I've just sent an email requesting that they be removed. Anyone who wants to maintain any of them should grab it now. The repos are all at http://darcs.imperialviolet.org On the chopping block: binary-strict codec-libevent control-timeout ctemplate LRU network-connection network-dns network-minihttp network-rpca system-inotify Cheers -- Adam Langley [EMAIL PROTECTED] http://www.imperialviolet.org ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] Libevent FFI problems
On Sat, Jul 26, 2008 at 4:34 AM, Levi Greenspan <[EMAIL PROTECTED]> wrote: > client: internal error: awaitEvent: descriptor out of range >(GHC version 6.8.3 for i386_unknown_linux) >Please report this as a GHC bug: > http://www.haskell.org/ghc/reportabug > Aborted > > This is caused by having more file descriptors than FD_SETSIZE (which is > 1024) on my system. So clearly select() doesn't scale to the numbers I > need :-( Now, *thats* a good point. I'd still suggest that taking the time to add epoll support to the threaded runtime would be better for you(, and for everyone else!) than avoiding the use of GHC threads. But, with that error, you're correct that the current select based system is insufficient. AGL -- Adam Langley [EMAIL PROTECTED] http://www.imperialviolet.org ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] Libevent FFI problems
On Fri, Jul 25, 2008 at 1:32 PM, Levi Greenspan <[EMAIL PROTECTED]> wrote: > Thank you (and Christopher) for the link. I have one question though - > I read this ticket in the GHC trac: > http://hackage.haskell.org/trac/ghc/ticket/635 which plans to use > epoll instead of select. The reason I thought of libevent is exactly > the support for epoll and other better-than-select mechanisms. Is > there progress in GHC with regard to this? This is very important for > me since I awill not write a web-server, but rather want to play with > Comet (i.e. having thousands of open connections as it is common for > long-polling in addition to many relatively short request/response > based connections). Bryan O'Sullivan make an off-the-cuff remark about doing this work at a talk a few months back, however I haven't heard anything about him starting. I think it's safe to assume that no one has taken it up yet. Personally, some of my other projects are higher priorities at the moment. I'd suggest that you write your server on the select() based system as-is for now. Then, when you need epoll you'll be sufficiently motivated to hack up the RTS to include it ;) AGL -- Adam Langley [EMAIL PROTECTED] http://www.imperialviolet.org ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] Libevent FFI problems
2008/7/25 Krzysztof Skrzętnicki <[EMAIL PROTECTED]>: > "Developing a high-performance web server in Concurrent Haskell" > http://www.haskell.org/~simonmar/papers/web-server-jfp.pdf (see page 15) > > Perhaps you might be interested in this paper also because of its topic. That's a good reference. Also note that the paper is 6 years old and GHC has come a long way since then. I'd suspect that the graph on page 15 would look much more favourable to Haskell these days. AGL -- Adam Langley [EMAIL PROTECTED] http://www.imperialviolet.org ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] Libevent FFI problems
2008/7/23 Levi Greenspan <[EMAIL PROTECTED]>: > I would be grateful for any advices, hints or comments. And I really > look forward to the FFI section in the "Real World Haskell" book. Generally it looks pretty good. I think I'm missing some C code (function "wrapper"). However, libevent is an odd choice to wrap. The RTS already multiplexes IO for Haskell programs. In order not to block the RTS, the libevent using code would have to be in its own kernel thread. Really, the RTS needs to be ported to libevent rather than an FFI wrapping. For your specific problem, I see you're allocating xptr, but I don't see that you're ever poking a value into it. Indeed, I can't see that createEvent ever uses 'x'. Hope that helps. AGL -- Adam Langley [EMAIL PROTECTED] http://www.imperialviolet.org ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] Haskell, Microsoft, and interview questions
On Thu, Jun 26, 2008 at 8:11 AM, Andrew Wagner <[EMAIL PROTECTED]> wrote: > I did some reading up on this, and it seems interesting. It would be > need to implement something like this in Haskell, but I can't seem to > find any detailed specs on the data-structure. Got any > recommendations? Specialised for 2d only, but: http://www.imperialviolet.org/binary/NearestNeighbour2D.hs AGL -- Adam Langley [EMAIL PROTECTED] http://www.imperialviolet.org ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] safe vs unsafe
On Tue, Jun 24, 2008 at 12:08 PM, Galchin, Vasili <[EMAIL PROTECTED]> wrote: > yes I am invoking a callback function written in Haskell. Then you should just need to remove the "unsafe" from the foreign import decl which leads to the callback getting called. AGL -- Adam Langley [EMAIL PROTECTED] http://www.imperialviolet.org ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] safe vs unsafe
2008/6/23 Galchin, Vasili <[EMAIL PROTECTED]>: > aio: schedule: re-entered unsafely. >Perhaps a 'foreign import unsafe' should be 'safe'? > > I want to understand from an implementation viewpoint .. deeper. This is a > GC issue yes? Are you reentering Haskell on a path which involves an unsafe FFI call? e.g. you enter C land via an unsafe FFI call and, on the same call path, reenter Haskell with a callback? http://www.cse.unsw.edu.au/~chak/haskell/ffi/ffi/ffise3.html#x6-130003.3 says: "Optionally, an import declaration can specify, after the calling convention, the safety level that should be used when invoking an external entity. A safe call is less efficient, but guarantees to leave the Haskell system in a state that allows callbacks from the external code. In contrast, an unsafe call, while carrying less overhead, must not trigger a callback into the Haskell system. If it does, the system behaviour is undefined. The default for an invocation is to be safe. Note that a callback into the Haskell system implies that a garbage collection might be triggered after an external entity was called, but before this call returns. Consequently, objects other than stable pointers (cf. Section 5.6) may be moved or garbage collected by the storage manager." -- Adam Langley [EMAIL PROTECTED] http://www.imperialviolet.org ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] Re: "C" buffer suggestions??
On Mon, Jun 23, 2008 at 2:27 PM, Don Stewart <[EMAIL PROTECTED]> wrote: > So heap allocated and collected, but not moved. My bad. In that case, you might want to work with ByteStrings all the way since it might make building the visible interface (which probably should use ByteStrings) easier. AGL -- Adam Langley [EMAIL PROTECTED] http://www.imperialviolet.org ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] Re: "C" buffer suggestions??
On Mon, Jun 23, 2008 at 1:15 PM, Galchin, Vasili <[EMAIL PROTECTED]> wrote: > I tried to write a test value of Word8 without success. Can you give an > example? I'm not quite sure what you would want in such an example. See [1] for the malloc function. Then you take the result and wrap it with [2]: fp <- mallocBytes nbytes >>= newForeignPtr finalizerFree When you want to use it, use [3] to unwrap fp and get a raw Ptr Word8. You can pass these to foreign functions any place they expect a uint8_t *, char * etc. [1] http://www.haskell.org/hoogle/hoodoc.cgi?module=Foreign.Marshal.Alloc&name=mallocBytes&mode=func [2] http://www.haskell.org/hoogle/hoodoc.cgi?module=Foreign.ForeignPtr&name=newForeignPtr&mode=func [3] http://www.haskell.org/hoogle/hoodoc.cgi?module=Foreign.ForeignPtr&name=withForeignPtr&mode=func -- Adam Langley [EMAIL PROTECTED] http://www.imperialviolet.org ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] IMAP and NNTP libraries
On Sat, Jun 21, 2008 at 10:52 PM, Maurício <[EMAIL PROTECTED]> wrote: > Are there mature libraries for IMAP and NNTP > available to Haskell? Pretty much, if a Haskell library exist it will be listed at [1]. Since I don't see anything for IMAP, nor NNTP I can be pretty sure that the answer is "no" I'm afraid. Perhaps you would care to write such libraries if you have a need? [1] http://hackage.haskell.org/packages/archive/pkg-list.html -- Adam Langley [EMAIL PROTECTED] http://www.imperialviolet.org ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] Re: "C" buffer suggestions??
2008/6/23 Galchin, Vasili <[EMAIL PROTECTED]>: > Basically I want to model POSIX async io "aiocb"( > http://uw714doc.sco.com/en/man/html.5/aiocb.5.html) in Haskell .. in > particular the aiocb field "aio_buf"! That's a mutable buffer, so this has to be pretty low level in your wrapping of the AIO interface. I would keep it as a Ptr Word8 at that point. ByteStrings are probably the correct type for the user visible interface, but I would guess not here. Plus, ByteString data lives in the Haskell heap so could get moved about by the collector. Since an AIO interface is probably going to return while the IO is still in progress, you don't want to stop the collector, but nor do you want the data moving because the kernel's pointer isn't going to move with it. AGL -- Adam Langley [EMAIL PROTECTED] http://www.imperialviolet.org ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] Re: Bit Streams
On Wed, Jun 18, 2008 at 11:07 AM, Bryan O'Sullivan <[EMAIL PROTECTED]> wrote: > If you provide Applicative and Alternative instances for your getter > monads (perhaps you already do?) I have Alternative instances, and they're used a fair bit in my HTTP parser (network-minihttp). You're right that I should add Applicative too; maybe this weekend. AGL -- Adam Langley [EMAIL PROTECTED] http://www.imperialviolet.org ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] Re: Bit Streams
On Wed, Jun 18, 2008 at 11:00 AM, Evan Laforge <[EMAIL PROTECTED]> wrote: > Inclusion in the official Data.Binary would be nice, especially if it > could be a drop-in replacement, say by changing the Data.Binary > instance definitions to Strict.Binary or something. I have a class defined to abstract the various Get monads, i.e. you can write generic code which will work over a strict get, or an incremental (continuation based) get. I don't have the same for put yet. The trick is to make it so that it doesn't end up with any runtime cost. dons would not be happy if I took his highly tuned binary library and made it slow :) AGL -- Adam Langley [EMAIL PROTECTED] http://www.imperialviolet.org ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] Re: Bit Streams
On Wed, Jun 18, 2008 at 10:52 AM, Don Stewart <[EMAIL PROTECTED]> wrote: > Would you recommend binary-strict over bitsyntax now? > Or are none yet entirely satisfactory Probably, yes. Bitsyntax was, after all, the first Haskell code I ever wrote :) It works, but I think the monad style of binary-strict is better. AGL -- Adam Langley [EMAIL PROTECTED] http://www.imperialviolet.org ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] blas bindings, why are they so much slower the C?
On Wed, Jun 18, 2008 at 9:16 AM, Anatoly Yakovenko <[EMAIL PROTECTED]> wrote: > C doesn't work like that :) Yes it can. You would have to check the disassembly to be sure, but C compilers can, and do, perform dead code elimination. AGL -- Adam Langley [EMAIL PROTECTED] http://www.imperialviolet.org ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] Re: Bit Streams
On Wed, Jun 18, 2008 at 12:46 AM, Dominic Steinitz <[EMAIL PROTECTED]> wrote: > OK, so today I tried to write my first program using the Binary library. > And I've hit a snag: It appears the library will only handle data that > is byte-aligned. So if I try to write three Bool values, it uses three > bytes, not three bits. There's a BitGet[1] and BitPut in the Hackage version of binary-strict. However, these versions work on strict ByteStrings. Putting, for one, is pretty inefficient. In the darcs version[2] of binary-strict, there's a fully lazy BitPut[3]. There was a request for a lazy BitGet, but I never wrote it. Most of the time, bit fields are small and you can use the lazy Get from binary to extract a ByteString and parse that with strict BitGet. However, the BitPut should be just fine (including doing things like writing ByteStrings by shifting each byte etc). If it's missing anything obvious, ping me and I'll see what I can do. Cheers, [1] http://darcs.imperialviolet.org/darcsweb.cgi?r=binary-strict;a=headblob;f=/src/Data/Binary/Strict/BitGet.hs [2] http://darcs.imperialviolet.org/binary-strict/ [3] http://darcs.imperialviolet.org/darcsweb.cgi?r=binary-strict;a=headblob;f=/src/Data/Binary/BitPut.hs -- Adam Langley [EMAIL PROTECTED] http://www.imperialviolet.org ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] who maintains the POSIX/Unix Unistd.hs?
2008/5/7 Galchin, Vasili <[EMAIL PROTECTED]>: > Hello, > > I am trying to find the definition for the data type > "TimeSpec"/"CTimeSpec". http://www.haskell.org/ghc/docs/latest/html/libraries/unix/src/System-Posix-Unistd.html Doesn't look like it's exported from anywhere, just used internally for c_nanosleep AGL -- Adam Langley [EMAIL PROTECTED] http://www.imperialviolet.org ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
[Haskell-cafe] Re: RFC: A standardized interface between web servers and applications or frameworks (ala WSGI)
On Mon, Apr 14, 2008 at 4:54 AM, Daniel Yokomizo <[EMAIL PROTECTED]> wrote: > Both request and response accept any entity headers and 7.1 (of RFC > 2616) says that a valid entity header is an extension header, which > can be any kind of header. Is wasn't suggesting that other headers be dropped, just that they remain as strings. > IMHO it would be better to create a two layered approach. The bottom > layer handles the request as a bunch of strings, just checks for > structural correctness (i.e. break the headers by line and such) > without checking if the headers are correct. The top layer provides a > bunch of parser combinators to validate, parse and sanitize the > request so a library can create its own contract: Ok, I think I'm convinced by this argument. I'd hope that a standard set of header parsers be defined, and that an application which only cares about 2616 headers can do call a single function to parse them all, but I no longer advocate that the base interface use parsed forms of headers. Also, parsing URLs seems to be pretty uncontroversial (maybe parsing key, value pairs from the path, maybe not) AGL -- Adam Langley [EMAIL PROTECTED] http://www.imperialviolet.org ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
[Haskell-cafe] Re: RFC: A standardized interface between web servers and applications or frameworks (ala WSGI)
On Sun, Apr 13, 2008 at 6:32 PM, Chris Smith <[EMAIL PROTECTED]> wrote: > Does old code that handled these headers stop working, just because it > was looking in the "other" section, but now needs to check a field > dedicated to that header? Yes, but it would be very sad if we couldn't do common header parsing because of this. I'd suggest that all the headers given in RFC 2616 be parsed and nothing else. That leaves the question of how we would handle the addition of any extra ones in the future. Firstly, packages could depend on a given version of this interface and we declare that the set of handled headers doesn't change within a major version. Better would be some static assertion that the interface doesn't handle some set of headers. Maybe there's a type trick to do this, but I can't think of one, so we might have to settle for a non static: checkUnparsedHeaders :: [String] -> IO () Which can be put in 'main' (or equivalent) and can call error if there's a mismatch. AGL -- Adam Langley [EMAIL PROTECTED] http://www.imperialviolet.org ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
[Haskell-cafe] Re: RFC: A standardized interface between web servers and applications or frameworks (ala WSGI)
On Sun, Apr 13, 2008 at 4:59 AM, Johan Tibell <[EMAIL PROTECTED]> wrote: > * Using a different set of data types would work better. Give that this is Haskell, I'd suggest more types ;) HTTP headers aren't just strings and, at the risk of tooting my own horn, I'll point to the Headers structure in [1]. Likewise, URLs have lots of structure that should just be handled in one place [2] [1] http://darcs.imperialviolet.org/darcsweb.cgi?r=network-minihttp;a=headblob;f=/Network/MiniHTTP/Marshal.hs [2] http://darcs.imperialviolet.org/darcsweb.cgi?r=network-minihttp;a=headblob;f=/Network/MiniHTTP/URL.hs AGL -- Adam Langley [EMAIL PROTECTED] http://www.imperialviolet.org ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] ghc
2008/4/10 Cetin Sert <[EMAIL PROTECTED]>: > Is GHC required to be installed on the target OS I compile Haskell binaries > for in order for these binaries to run? I need a quick answer on that! Short answer: no. GHC can produce binaries which don't have any runtime deps on GHC libs. One thing that you might need on the target host is GMP, however that's pretty common. AGL -- Adam Langley [EMAIL PROTECTED] http://www.imperialviolet.org ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] [GSoC] Student applications deadline extended one week
On Mon, Mar 31, 2008 at 12:00 PM, Manlio Perillo <[EMAIL PROTECTED]> wrote: > Since Nginx is asynchronous, how can be solved the producer-consumer > problem (that is, the Haskell program produces more data that Nginx can > send to the client without blocking)? I assume that the Haskell process is connected to nginx over a pipe or socket. In which case, nginx can use flow control to block the sending side of the pipe and the Haskell code will backup on that. If many connections need to be multiplexed over the same flow-controlled entity (i.e. a pipe), without head-of-line blocking then you can just suspend the current thread using an MVar or the STM objects. Alternatively, with Network.MiniHTTP the problem is turned inside out. Request handlers give return a Source object, which can be asked to generate more data on request. This would be similar to generators in Python. AGL -- Adam Langley [EMAIL PROTECTED] http://www.imperialviolet.org ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] [GSoC] X Haskell bindings
2008/3/31 Roman Cheplyaka <[EMAIL PROTECTED]>: > Especially I need help with "Milestones" section. I'm not quite sure how > it should look like for such project. This would seem to be a commendable SoC project. I'm not sure that the milestones are all that important - no plan ever survives contact with reality. However, I'd suggest that these would be important aspects: * A working parser for the XCB (this involves choosing an XML parser and getting useful data structures from it) * A working Haskell code generator which outputs bindings for a minimal subset of the functions (the last two need not be in Haskell, although I suspect that it would be a good idea for them to be so) * The first, minimal, Haskell X client using XCB * Having sensible bindings for 25%/50%/75% and 100% of the XCB interface. AGL -- Adam Langley [EMAIL PROTECTED] http://www.imperialviolet.org ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] Re: HTTP client libraries
On Fri, Mar 28, 2008 at 6:40 AM, John Goerzen <[EMAIL PROTECTED]> wrote: > Your Source type is an interesting approach, too. I'm still not sure if it's actually a good idea. But if you do have any requests, I'm very open to them. I think that a fetchBasic which walks redirects would be a good start but, other than that, I'm rather distracted with elliptic curve groups at the moment. AGL -- Adam Langley [EMAIL PROTECTED] http://www.imperialviolet.org ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] HTTP client libraries
On Thu, Mar 27, 2008 at 12:08 PM, John Goerzen <[EMAIL PROTECTED]> wrote: > * network-minihttp > > Doesn't appear to actually be very useful as a client. > > Also, as far as I have been able to deduce, none of these have > built-in support for https (SSL) URLs. The client doesn't do a lot, but I don't know what you would want from a client. Email me a list of use cases and ;) It does support HTTPS, however. See examples/webcat.hs AGL -- Adam Langley [EMAIL PROTECTED] http://www.imperialviolet.org ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] AMQP framing layer: design help requested.
On Fri, Mar 21, 2008 at 6:41 PM, Adam Langley <[EMAIL PROTECTED]> wrote: > Firstly, I would use the Get and Put monads directly, rather than > making them instances of Binary. Also, while I'm at it - I believe that AMQP messages and small and delineated. In which case, the Data.Binary.Strict.Get monad from the binary-strict package might be a better match (shameless self-plug) AGL -- Adam Langley [EMAIL PROTECTED] http://www.imperialviolet.org ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] AMQP framing layer: design help requested.
On Fri, Mar 21, 2008 at 11:02 AM, Paul Johnson <[EMAIL PROTECTED]> wrote: > This effectively duplicates the AmqpVariant solution. It would work, > but it feels horrible. I get the feeling that there ought to be a > better way, possibly using classes or existential types. Any suggestions? Firstly, I would use the Get and Put monads directly, rather than making them instances of Binary. I think, if I were doing it, I would start off with something like this: class AMQPValue a where putAMQP :: a -> Put instance AMQPValue Word8 where putAMQP v = putWord8 1 >> putWord8 v ... (many instances) instance (AMQPValue a) => AMQPValue [a] where putAMQP vs = putWord8 (length vs) >> mapM_ putAMQP vs You probably want a getAMQP member of the AMQPValue class too. Also > getter <- fmap (amqpGetTable !) getWord8 > getter is just > fmap (amqpGetTable !) getWord8 Hope that helps some AGL -- Adam Langley [EMAIL PROTECTED] http://www.imperialviolet.org ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
[Haskell-cafe] ANN: Network.MiniHTTP 0.2
Since many people are interested in a ByteString HTTP library, here's network-minihttp 0.2 [1]. * Has code for both clients and servers * HTTP and HTTPS support (the latter is a little shaky, but mostly works) * DNS lookups (in the client code) don't block the whole process The Haddock documentation is at: http://darcs.imperialviolet.org/minihttp-docs/index.html Half the interfaces are ill-thought out, and the rest are just as bad. However, if you send me your ideas for how it should look I'll certainly consider them. The major omission at the moment are defensive timeouts for talking to the network. However, these shouldn't be too hard. [1] http://hackage.haskell.org/cgi-bin/hackage-scripts/package/network-minihttp-0.2 -- Adam Langley [EMAIL PROTECTED] http://www.imperialviolet.org ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] Satnam Singh of Microsoft Research
2008/3/18 Galchin Vasili <[EMAIL PROTECTED]>: > Recently Satnam Singh of Microsoft Research gave a talk at Google about > concurrency, Haskell STM, etc. Was there a transcript of this talk? Do you have the exact date of this talk? I can't see that anyone called Satnam has given a talk at Google. (It might have been very informal, in which case there wouldn't be a record of it, and nor would there be a video) Cheers, AGL -- Adam Langley [EMAIL PROTECTED] http://www.imperialviolet.org ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] Re: all threads are blocked by recvFrom
On Tue, Mar 18, 2008 at 3:41 AM, Vitaliy Akimov <[EMAIL PROTECTED]> wrote: > Unfortunally this way seems to be wrong. Error codes for winsockets > and BSD-sockets are different. Hmm, "Networking broken on Windows" would seem to be a pretty big issue. One of the Windows peeps like to speak up here? AGL -- Adam Langley [EMAIL PROTECTED] http://www.imperialviolet.org ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] Re: all threads are blocked by recvFrom
On Mon, Mar 17, 2008 at 2:29 AM, Vitaliy Akimov <[EMAIL PROTECTED]> wrote: > Hi Adam, sorry for late answer. Here is my example [1], but yours > doesn't work on my PC too. And it's strange it works on yours. Are you running Windows? Because you're hpaste example works, nonblocking for me too. > And after changing in the network package FFI declaration for > c_recvfrom from unsafe to safe both examples start working. The important point here is that the recvFrom calls in Network.Socket[1] don't block. They are non-blocking calls so, although other threads may be suspended while in the FFI call, the FFI call returns -EAGAIN if it would block and blocking is passed to the RTS to handle. While sleeping in the RTS (with threadWaitRead etc), other Haskell threads can read. See the code in [1] (esp throwErrnoIfMinus1Retry_repeatOnBlock) So, the only reason that other threads would be blocked is if the socket wasn't marked as non-blocking. Here's a snippet of strace output from compiling your hpaste example: socket(PF_INET, SOCK_DGRAM, IPPROTO_IP) = 5 fcntl64(5, F_GETFL) = 0x2 (flags O_RDWR) fcntl64(5, F_SETFL, O_RDWR|O_NONBLOCK) = 0 You can see that the socket is set to non-blocking immediately. [1] http://haskell.org/ghc/docs/latest/html/libraries/network/src/Network-Socket.html#recvFrom -- Adam Langley [EMAIL PROTECTED] http://www.imperialviolet.org ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] GHC threaded RTS will call the finaliser of a ForeignPtr while references still exist
On Sun, Mar 16, 2008 at 8:45 PM, Don Stewart <[EMAIL PROTECTED]> wrote: > Create a ticket on the bug tracker, and I'm sure Ian or Simon M will have a > look at it. > >http://hackage.haskell.org/trac/ghc/newticket?type=bug Done, thanks: http://hackage.haskell.org/trac/ghc/ticket/2161 AGL -- Adam Langley [EMAIL PROTECTED] http://www.imperialviolet.org ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
[Haskell-cafe] Re: GHC threaded RTS will call the finaliser of a ForeignPtr while references still exist
On Sun, Mar 16, 2008 at 8:40 PM, Adam Langley <[EMAIL PROTECTED]> wrote: > I believe that I've managed to distill a crash which has been driving > me crazy for a few days into a short enough test case (22 lines) that > it might be useful. Cale made a suggestion which shortens it: fptest.hs: http://hpaste.org/6423 -- Adam Langley [EMAIL PROTECTED] http://www.imperialviolet.org ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
[Haskell-cafe] GHC threaded RTS will call the finaliser of a ForeignPtr while references still exist
I believe that I've managed to distill a crash which has been driving me crazy for a few days into a short enough test case (22 lines) that it might be useful. In short: the threaded RTS will call the finialiser of a ForeignPtr while an exception handler still holds a reference to it: % ghc -make fptest.hs cbits.c -threaded [1 of 1] Compiling Main ( fptest.hs, fptest.o ) Linking fptest ... % ./fptest New object getting created: 66f010 Finaliser getting called for 66f010 Use called for 66f010 cbits.c: http://hpaste.org/6420 fptest.hs: http://hpaste.org/6421 I'm hoping that this is useful to someone who knows the RTS. Cheers, -- Adam Langley [EMAIL PROTECTED] http://www.imperialviolet.org ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] Re: all threads are blocked by recvFrom
On Fri, Mar 14, 2008 at 10:13 AM, Adam Langley <[EMAIL PROTECTED]> wrote: > See [1] for an example which works for me. (If you're on Windows, you probably need to wrap main in withSocketsDo) AGL -- Adam Langley [EMAIL PROTECTED] http://www.imperialviolet.org ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] Re: all threads are blocked by recvFrom
On Fri, Mar 14, 2008 at 8:51 AM, Vitaliy Akimov <[EMAIL PROTECTED]> wrote: > > I assume that you're binding the libc function directly here: > > I'm using Network.Socket. Sory if it's not clear from my previous posts. Then everything should Just Work(tm). You might need to paste in code in order to figure out why this wouldn't be so. See [1] for an example which works for me. It starts a thread which prints "working..." once a second and, in another thread, listens for UDP packets on port 4112. I can use `nc -u 127.0.0.1 4112` to get this: "working..." "working..." ("testing\n",8,127.0.0.1:36179) "working..." "working..." ("testing two\n",12,127.0.0.1:36179) "working..." [1] http://hpaste.org/6362 -- Adam Langley [EMAIL PROTECTED] http://www.imperialviolet.org ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] Re: all threads are blocked by recvFrom
On Fri, Mar 14, 2008 at 7:43 AM, Vitaliy Akimov <[EMAIL PROTECTED]> wrote: > Rebuilding of the network package with changed safety helped but I > don't think this is the solution. BTW accept is declared as safe. What > is the reason of declaring recvFrom as unsafe? I think this breaks > highly required feature. Apparently it's impossible to make concurrent > server for non connection based protocols. I assume that you're binding the libc function directly here: In that case, you need to have the RTS manage sleeping your thread for you. You should make the socket non-blocking and handle the EAGAIN and EWOULDBLOCK cases by calling threadWaitRead[1] to block the current Haskell thread only, until the fd is readable. Note that threadWaitRead is GHC only. If you download the source to network-bytestring[2], you can see a very similar pattern in the code for send and recv in there. Alternatively, you can use the functions in Network.Socket, which should work fine. [1] http://haskell.org/ghc/docs/latest/html/libraries/base/Control-Concurrent.html#v%3AthreadWaitRead [2] http://hackage.haskell.org/cgi-bin/hackage-scripts/package/network-bytestring-0.1.1.2 -- Adam Langley [EMAIL PROTECTED] http://www.imperialviolet.org ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] Problem making a program in ghc
2008/3/13 Maverick <[EMAIL PROTECTED]>: > I have writed an application that parses xml files and returns a result > (using HaXml), this is published as a service, I implemented a server > (withSocketsDo), the server listen the port 5760 in a forever fashion, when > a message arrives, then the program creates a new thread (forkIO) for attend > the request. This works fine when I run it from ghci, when I make the > program and run the binary (in Windows), the requests arrive to the program > but I think that the response can´t be sent because the front end can´t > receive the response (I am calling the service from a .Net web application), > I have a log that confirms that the response arrives correctly. I hate to see any requests for help go unanswered here, but this one might be tough. I think you need to give some more information, otherwise the suggestions are going to be very general. Can you put the Haskell source code on a website somewhere and link to it. Since it's a network service, an example request and reply might be good to include. In general, you should check that you are correctly flushing your connection. If you are using Handles to interface to the network, they can buffer the response. hFlush[1] may need to be called when you have finished generating it. [1] http://haskell.org/ghc/docs/latest/html/libraries/base/System-IO.html#v%3AhFlush AGL -- Adam Langley [EMAIL PROTECTED] http://www.imperialviolet.org ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] Starting Haskell with a web application
On Wed, Mar 12, 2008 at 3:49 PM, Adam Langley <[EMAIL PROTECTED]> wrote: > The OpenID example is running in EC2[4] at the moment if anyone wants to > play. Well, thanks to all the people who hit it, there's nothing like users to find the stupid bugs ;) * Caching was wrong on the front page, so it would act funky if you were behind a proxy * Serving from the filesystem (i.e. the CSS) was broken because I missed an action in that code All fixed... AGL -- Adam Langley [EMAIL PROTECTED] http://www.imperialviolet.org ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] Starting Haskell with a web application
On Wed, Mar 5, 2008 at 12:25 PM, Adam Langley <[EMAIL PROTECTED]> wrote: > I'm > working towards, is an OpenID consumer. Once I have that working, I'll > do a second release. It's not that far off, it's just a question of > time. The darcs release of minihttp[1] can now do this. It's not a Hackage release because it needs a darcs version of HsOpenSSL[2] and network-connection[3]. The OpenID example is running in EC2[4] at the moment if anyone wants to play. [1] http://darcs.imperialviolet.org/network-minihttp [2] http://darcs.imperialviolet.org/HsOpenSSL [3] http://darcs.imperialviolet.org/network-connection [4] http://ec2-67-202-22-226.compute-1.amazonaws.com:4112/ AGL -- Adam Langley [EMAIL PROTECTED] http://www.imperialviolet.org ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] Re: Constructing Data.Map from ByteString
On Tue, Mar 11, 2008 at 4:13 AM, Dave Tapley <[EMAIL PROTECTED]> wrote: > I've upgraded to bytestring-0.9.0.5 from Darcs, no improvement. > Also this morning I tried using Data.HashMap with Bytestring's readInt > and HashMap's hashInt.. The result was a Stack space overflow :( Map is probably a better choice than HashMap. Exactly how large is the input? Like I said, I got about 2x bloat (measured by RSS), maybe you can send a fuller example code? AGL -- Adam Langley [EMAIL PROTECTED] http://www.imperialviolet.org ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] Constructing Data.Map from ByteString
On Mon, Mar 10, 2008 at 4:33 PM, Dave Tapley <[EMAIL PROTECTED]> wrote: > Concerning this minimal example: > http://hpaste.org/6268 Change the Data.ByteString.Char8 to Data.ByteString.Lazy.Char8, also the takeWhile, dropWhile pair is better expressed using span. With those two changes, and a 150MB test file, a compiled binary is using about 220MB of resident space. That seems pretty reasonable to me. As a purely subjective atheistic point: importing some modules qualified will help your sanity. Otherwise you have lots of common function names (readFile etc) and you have to try and figure out which module they are going to resolve into all the time! Cheers, AGL -- Adam Langley [EMAIL PROTECTED] http://www.imperialviolet.org ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] STM example code
2008/3/8 Galchin Vasili <[EMAIL PROTECTED]>: > I am playing around with the STM API. I would like to see examples of > STM other than the Santa.hs as I am having problems with STM vs IO. The STM papers were the best documentation for me: http://research.microsoft.com/~simonpj/papers/stm/index.htm -- Adam Langley [EMAIL PROTECTED] http://www.imperialviolet.org ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
[Haskell-cafe] Re: Difficulties implementing an incremental parser using Oleg-style left fold enumerator
On Sat, Mar 8, 2008 at 12:56 AM, Johan Tibell <[EMAIL PROTECTED]> wrote: > The problem -- maybe there are others too -- is that when a parser such as > > many (byte 65) > > is run it will always return a 'Partial' result waiting for more input > even though the enumerator is exhausted. In other words, there's no > way to detect end of input. It appears that you need some way to distinguish the end of input from, "that's all I have for now". You could use an empty Bytestring in S if you were careful that you maintained that, in normal processing, such a state doesn't arise. Otherwise, have a Maybe in your state and set it to Nothing when the input is exhausted. Then have combinators, like many, handle the EOF case sensibly. AGL -- Adam Langley [EMAIL PROTECTED] http://www.imperialviolet.org ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] Re: Network.Socket Woes
On Fri, Mar 7, 2008 at 11:04 AM, Benedikt Schmidt <[EMAIL PROTECTED]> wrote: > The PortNumber type is an instance of Num, so it works if you do not > use the PortNum constructor. > > Prelude Network.Socket> case 10162 :: PortNumber of PortNum p -> p > 45607 And there was the trick that I've been missing all this time ;) I'll probably submit a documentation patch for this since I'm a Bear of Little Brain and this wasn't obvious to me. -- Adam Langley [EMAIL PROTECTED] http://www.imperialviolet.org ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] Network.Socket Woes
On Fri, Mar 7, 2008 at 8:10 AM, Scott Bell <[EMAIL PROTECTED]> wrote: > my test program. The Haskell version, however, does not > return from recvFrom. I've also tried wrapping this in a > withSocketsDo, with no effect. So this is a long standing, ah, issue with the Network modules. Try sending a UDP packet to port 45607 and you'll find that the Haskell code gets it. hex(45607) = 0xb227 0x27b2 = 10162 In short, PortNum doesn't do the endian conversion for you. And I don't know a good way to figure out the endianness of the underlying system from Haskell I'm afraid. I usually end up FFIing out to htons or just assuming that the system is little-endian. We should really fix this unless there's some trick that I've been missing all this time. AGL -- Adam Langley [EMAIL PROTECTED] http://www.imperialviolet.org ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] Starting Haskell with a web application
On Thu, Mar 6, 2008 at 3:08 AM, Bayley, Alistair <[EMAIL PROTECTED]> wrote: > Do you (both) have repos that I could download from? I quite interested > in both projects, esp. the WSGI clone. There was a Hackage release of network-minihttp[1], which I think would serve files from the filesystem quite happily. The darcs repo is just a mess at the moment. (darcs.imperialviolet.org/network-minihttp) [1] http://hackage.haskell.org/cgi-bin/hackage-scripts/package/network-minihttp-0.1 AGL -- Adam Langley [EMAIL PROTECTED] http://www.imperialviolet.org ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] Starting Haskell with a web application
On Wed, Mar 5, 2008 at 11:07 AM, Bryan O'Sullivan <[EMAIL PROTECTED]> wrote: > Indeed. In addition to the code you mention, people like Adam Langley > and Johan Tibbell are taking on corners of the web app problem space in > a more modern context. I should probably speak up then ;) I'm (slowly) writing Network.MiniHTTP. What it does have is a modern, ByteString based HTTP parser and serialiser (in decent shape), SSL support (only in darcs at the moment) and my test case, that I'm working towards, is an OpenID consumer. Once I have that working, I'll do a second release. It's not that far off, it's just a question of time. AGL -- Adam Langley [EMAIL PROTECTED] http://www.imperialviolet.org ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] Connection helpers: for people interested in network code
On Tue, Mar 4, 2008 at 7:31 AM, Bjorn Bringert <[EMAIL PROTECTED]> wrote: > you may want to have a look at the socket abstraction used in the HTTP > package: > > > http://hackage.haskell.org/packages/archive/HTTP/3001.0.4/doc/html/Network-Stream.html > > It would be great to get HTTPS support going! I should have mentioned that I had seen this in the original email. I think the major problem with this interface was that it was written in the time before ByteStrings. Now that we have ByteStrings I think that it makes a lot of sense for networking to use them. However, it shouldn't be too hard to wrap HsOpenSSL in this interface. I might try this this week. Then HTTPS should Just Work (maybe ;) AGL -- Adam Langley [EMAIL PROTECTED] http://www.imperialviolet.org ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] .cabal problem
2008/3/1 Galchin Vasili <[EMAIL PROTECTED]>: > I am trying to install a package on my Linux Ubuntu machine. It chokes > build-depends: base, directory because directory dependency is > unresolvable. Do I have to specify "extra-libs" so that correct library > space is searched? Actually I tried this and it didn't help. ?? So, here's[1] the package "directory", however it ships with GHC, so you very probably already have it installed. Try `ghc-pkg describe directory` to check. Also, run `ghc --version` to see what version of GHC you are running. If it's < 6.8, you may have problems because the directory package didn't exist before then, I believe the same modules were in the base package. In that case, the easy solution is probably to upgrade GHC. [1] http://hackage.haskell.org/cgi-bin/hackage-scripts/package/directory-1.0.0.0 AGL -- Adam Langley [EMAIL PROTECTED] http://www.imperialviolet.org ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
[Haskell-cafe] Connection helpers: for people interested in network code
I generally find that I'm wrapping sockets in the same functions a lot and now I'm looking writings code which works with both Sockets and SSL connections. So I wrote a module, presumptuously called Network.Connection, although I'm not actually going to try and take that name (even in Hackage) unless I get a general agreement that this is a good thing. So, any comments on the interface, similar things that I should look at etc? http://www.imperialviolet.org/binary/network-connection/Network-Connection.html I made the BaseConnection an ADT, rather than a class because I wanted to avoid hitting the monomorphism restriction in code. That might have been a mistake, I'm not sure yet. If it doesn't excite anyone enough to reply, I'll change the name and put it in Hackage, mostly as is. Then I'll tie HsOpenSSL into it so that SSL connections work transparently. Cheers, AGL -- Adam Langley [EMAIL PROTECTED] http://www.imperialviolet.org ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] parsing c
On Sun, Feb 24, 2008 at 4:08 PM, Evan Martin <[EMAIL PROTECTED]> wrote: > So two questions: Is there an alternative C-parsing library? You could use gcc-xml[1] with one of the various XML parsers. [1] http://www.gccxml.org/HTML/Index.html -- Adam Langley [EMAIL PROTECTED] http://www.imperialviolet.org ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] A beginners question
On Sat, Feb 23, 2008 at 8:00 AM, Harri Kiiskinen <[EMAIL PROTECTED]> wrote: > then why does > > let i = fmap (^4) [1,2,3] in shows i " " > > give > > "[1,16,81] " I'll probably mess this up somewhere, but if I do, reset assured that someone else here will correct me ;) fmap (^4) [1,2,3] == [1,16,81] so shows " " of that == "[1,16,81] " (note the trailing space), obvious I hope. However, [1,16,81] >>= \i -> shows i " " operates in the list monad. I can't find the actual instance of Monad [] right now, but it will apply 'shows i " "' to each element in the first list and concat the results. Since strings are just [Char], concating them is string concatenation. And so you have "1 16 81 " (again, note the trailing space) AGL -- Adam Langley [EMAIL PROTECTED] http://www.imperialviolet.org ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
[Haskell-cafe] ANN: ctemplate, SSL server support in HsOpenSSL, network-minihttp, binary parsing, network-dns
Binary parsing: binary-strict now has support for combinator parsing[1] with <|> and friends. This works both for the strict Get and the IncrementalGet. (The latter was more complex than I expected) binary-strict also gained a very fast byte set module[2] (used for scanning ranges of valid bytes) and a surprisingly useful hexDump[3] function (because I kept having to write it in lots of places) [1] http://hackage.haskell.org/packages/archive/binary-strict/0.3.0/doc/html/Data-Binary-Strict-Class.html [2] http://hackage.haskell.org/packages/archive/binary-strict/0.3.0/doc/html/Data-Binary-Strict-ByteSet.html [3] http://hackage.haskell.org/packages/archive/binary-strict/0.3.0/doc/html/Data-Binary-Strict-Util.html DNS: network-dns contains a DNS client library[4] in pure Haskell. The ADNS library (for which a wrapper already exists) is GPL, not BSD. This code is also halfway to being a DNS server should anyone so wish. This is very similar to the DNS client library in libevent (because I wrote them both). HTTP: ctemplate is a wrapping of the template library which Google uses for most of their sites. There are several template systems already in Hackage, but I like ctemplate because escaping is so easy and well supported. Generally the power of a template system doesn't interest me (in fact, it can be a bad thing), but stopping XSS attacks is a big deal. HsOpenSSL now has rudimentary support for writing SSL servers[5] (clients coming soon). In terms of blocking IO, this /should/ work the way you would hope (i.e. like a Haskell function w.r.t. forkIO). network-minihttp is a small HTTP server. Currently it serves files from the disk (with caching and range support), but it's most of the way to being a HTTP client also. Don't do anything too serious with it yet, however. It needs a few limits to stop DoS attackers from, for example, sending an infinite header and using up all the memory :) [4] http://hackage.haskell.org/packages/archive/network-dns/0.1.1/doc/html/Network-DNS-Client.html -- Adam Langley [EMAIL PROTECTED] http://www.imperialviolet.org 650-283-9641 ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] getSymbolicLinkStatus completely broken on some 6.8.2 systems
Filed as: http://hackage.haskell.org/trac/ghc/ticket/2096 -- Adam Langley [EMAIL PROTECTED] http://www.imperialviolet.org 650-283-9641 ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] getSymbolicLinkStatus completely broken on some 6.8.2 systems
On Feb 12, 2008 11:04 PM, Adam Langley <[EMAIL PROTECTED]> wrote: > structure filled in. HsUnix.h has a wrapper around lstat for exactly > this reason, however ltrace shows it calling the wrong one. So (finally!) the real issue: hsc2hs has a C preprocessor prelude (utils/hsc2hs/template-hsc.h) which includes some GHC header files. As a result, hsc processes files in 64-bit mode: % cpp -I../../includes -dM template-hsc.h | grep FILE_OFFSET_BITS #define _FILE_OFFSET_BITS 64 However, when building HsUnix.c (or any c file with cabal), the environment is different and it's built in 32-bit mode. Thus the lstat wrapper called the 32-bit glibc function, but the structure sizes and offsets in Files.hs (from Files.hsc) are all for 64-bit mode. The fix is not quite obvious. Probably hsc2hs should try harder not to change the environment so much. I'm not quite sure why the hsc2hs template pulls in the HsFFI header - it doesn't seem to need it. Although you can remove the #include and rebuild the unix package, I can safely say that you'll be reinstalling ghc if you do. What might work is changing hsc2hs and then rebuilding all of GHC. However, that's an overnight build so I don't know yet. AGL -- Adam Langley [EMAIL PROTECTED] http://www.imperialviolet.org 650-283-9641 ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] getSymbolicLinkStatus completely broken on some 6.8.2 systems
On Feb 12, 2008 10:44 PM, Adam Langley <[EMAIL PROTECTED]> wrote: > Currently I'm looking at hsc2hs for this bug. On a 32 bit box here: hsc2hs is forgiven; if you build with #define _FILE_OFFSET_BITS 32 then the structure is 96 bytes and the 64-bit offset is, indeed, at offset 88. However, if you end up calling the wrong libc lstat (and it's tough to tell, because both lstat and lstat64 in libc call the system call lstat64, so strace can't tell you) you'll only get an 88 byte structure filled in. HsUnix.h has a wrapper around lstat for exactly this reason, however ltrace shows it calling the wrong one. -- Adam Langley [EMAIL PROTECTED] http://www.imperialviolet.org 650-283-9641 ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] getSymbolicLinkStatus completely broken on some 6.8.2 systems
On Feb 12, 2008 8:38 PM, Jeremy Shaw <[EMAIL PROTECTED]> wrote: > I get funny results when I call getSymbolicLinkStatus in ghc 6.8.2 on > Ubuntu 7.10. This happens on several systems and has been confirmed by > other people. Currently I'm looking at hsc2hs for this bug. On a 32 bit box here: int main() { struct stat st; printf("size of structure is %d\n", sizeof(st)); printf("inode is at offset %d\n", __builtin_offsetof(struct stat, st_ino)); } Gives: size of structure is 88 inode is at offset 12 However, building Files.hsc gives: fileID (FileStatus stat) = unsafePerformIO $ withForeignPtr stat $ ((\hsc_ptr -> peekByteOff hsc_ptr 88)) {-# LINE 315 "Files.hsc" #-} e.g. it's reading random memory at the end of the structure. Looking at the code from the documentation: http://haskell.org/ghc/docs/latest/html/libraries/unix/src/System-Posix-Files.html#fileID fileID (FileStatus stat) = unsafePerformIO $ withForeignPtr stat $ ((\hsc_ptr -> peekByteOff hsc_ptr 8)) {-# LINE 315 "System/Posix/Files.hsc" #-} Seems like it might work, but that offset is too small. It appears that it might be a 64-bit build (because the size of the stat structure which when it gets malloced is 144), but a 64-bit build should have larger offsets, not smaller. You can't write your own fileID function, because FileStatus isn't exported from Posix.Files. You can, however, write getSymbolicLinkInode a c function (which does both the lstat and extraction from the structure) and FFI it. AGL -- Adam Langley [EMAIL PROTECTED] http://www.imperialviolet.org 650-283-9641 ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] example packages that use Storable in Hackage ??
2008/2/11 Galchin Vasili <[EMAIL PROTECTED]>: > http://hackage.haskell.org/packages/archive/pkg-list.html .. what are > some packages that use Storable? binary and binary-strict at least. AGL -- Adam Langley [EMAIL PROTECTED] http://www.imperialviolet.org 650-283-9641 ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] ANN: nano-hmac 0.2.0
On Feb 11, 2008 12:54 AM, Hitesh Jasani <[EMAIL PROTECTED]> wrote: > nano-hmac provides bindings to OpenSSL's HMAC interface. With this release > the > set of hashing functions supported is: MD5, SHA, SHA1, SHA224, SHA256, SHA384, > SHA512. Just a heads up; PHO has written nice bindings to much of OpenSSL: http://hackage.haskell.org/cgi-bin/hackage-scripts/package/HsOpenSSL-0.3.1 AGL -- Adam Langley [EMAIL PROTECTED] http://www.imperialviolet.org 650-283-9641 ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] question concerning ANSI "void *"
On Feb 8, 2008 10:57 AM, Galchin Vasili <[EMAIL PROTECTED]> wrote: > basically I am trying to implement ioctl for the Posix library .. so a > possible signtaure would be: > > fdIoctl :: Fd -> Int -> Ptr Word 8 -> IO( Ptr Word8) Ah, ok. You could cover many of the ioctls (the ones which only take a single primitive) by using the Storable class and the types CInt etc. Define the FFI call to ioctl with a type of CInt -> Int -> Ptr () -> IO CInt and then ioctlCInt :: CInt -> CInt -> CInt -> IO CInt ioctlCInt fd call arg = do allocaBytes $ \ptr -> do poke ptr arg result <- ioctl fd call (cast ptr) when (result < 0) $ fail "ioctl error" peek ptr (untested, might work ;) ... and likewise for the other C types. However, for those ioctls which take a complex structure (e.g. many of the networking ones), you'll need to marshal yourself: data SomeIOCtlStruct = CInt CInt CInt ioctlSomeIOCtlStruct :: CInt -> CInt -> SomeIOCtlStruct -> IO () ioctlSomeIOCtlStruct = do ... (see the above linked to pointers to hsc2hs and c2hs about how to write this function) AGL -- Adam Langley [EMAIL PROTECTED] http://www.imperialviolet.org 650-283-9641 ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] question concerning ANSI "void *"
On Feb 8, 2008 9:13 AM, Galchin Vasili <[EMAIL PROTECTED]> wrote: > Let's take a concrete but "made up" case .. suppose we want to call through > to pthread_create and pass the (void *) argument to pthread_create which in > turn gets interpreted by the pthread that is launched. How would one > populate the C struct that is passed to the launched pthread keeping in mind > that this C struct is variable in length? From the FFI how would one model > this C struct? It tough to be helpful with such a generic request. Here are some options that you can consider: 1) Write a wrapper function in C which has a nicer FFI interface to call from Haskell. Using cabal this is pretty painless and, if the interface suits it, it probably the easiest solution. 2) Call pthread_create directly with the FFI. You can give the FFI function a Haskell type with 'Ptr ()' or 'Ptr X', it doesn't really matter. However the type system serves you best, do it that way. This means that you need to populate the struct yourself in Haskell. The issue with this is that the local system defines lots of things like padding and alignment which mean that the layout of the structure in memory is complex and platform specific. Tools like hsc2hs[1] or c2hs will be very helpful here. Dealing with the variable length probably isn't an issue. Usually variable length structures have a fixed header and a variable tail, where the tail is an array of primitives. You can malloc the correct sized region, use one of the previous tools to fill in the fixed header and then use poke to complete the tail. I might be able to be more helpful if you give the actual struct and function prototype that you're trying to wrap. Cheers [1] http://therning.org/magnus/archives/tag/hsc2hs [2] http://www.cse.unsw.edu.au/~chak/haskell/c2hs/ -- Adam Langley [EMAIL PROTECTED] http://www.imperialviolet.org 650-283-9641 ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] question concerning ANSI "void *"
2008/2/7 Galchin Vasili <[EMAIL PROTECTED]>: > Ok .. I am writing a Haskell function that will call down into the ANSI > C library .. blah :: -> Ptr Word8 -> The underlying C function > that "blah" is calling has a "void *" so I am using "Ptr Word 8" to model > the "void *". Depending on the context, "void *" is generally either taken as a Ptr () (for an opaque pointer) or Ptr Word8 (for calls like memcpy). > I propose to have the callers of function "blah" to populate a > data structure something like "Ptr Buz" where "data Buz = { } and > then do a "recast :: Ptr Word 8 -> Ptr Buz" when invoking function "blah". > Does this seem reasonable? Is there a better way? Generally, Ptr x is only used where x is either a shadow type or a Bits type. Having a Ptr Baz where Baz is an ADT seems a little odd. If you need to translate a structure from Haskell to C code, probably you are better off having callers pass in a Baz then, internal to the wrapping, fill out the C structure and call the FFI function with a Ptr CBaz (where CBaz is a shadow type). AGL -- Adam Langley [EMAIL PROTECTED] http://www.imperialviolet.org 650-283-9641 ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] reading from the cdrom drive on Red Hat
On Feb 1, 2008 1:42 PM, Bryan O'Sullivan <[EMAIL PROTECTED]> wrote: > No, it's the Haskell runtime turning a -1 return from read into an > exception. You need to call hIsEOF to check whether you've hit EOF, > then break out of the loop. (assuming you meant 0, not -1) My bad, I thought that the Posix.IO stuff was a closer wrapping than that. It does, indeed, throw an exception on 0. How unfortunate. AGL -- Adam Langley [EMAIL PROTECTED] http://www.imperialviolet.org 650-283-9641 ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] reading from the cdrom drive on Red Hat
2008/2/1 Galchin Vasili <[EMAIL PROTECTED]>: > I wrote a simple program to read the contents of a cdrom: (Note that this is a terribly inefficient way of reading large amounts of binary data. Of course, if this is just meant as an example, that's fine. Otherwise, see the struff about ByteStrings at http://www.haskell.org/haskellwiki/DealingWithBinaryData) The error you are seeing comes from the operating system. You could run the resulting Haskell program under strace to see the exact error, but what's happening is that the kernel is getting an error from the CD drive itself. This is pretty much unique to CDs I guess. If you were reading a hard drive, the kernel knows exactly how large the disk is (see /proc/partitions) and would return EOF (zero bytes). As an aside, hitting EOF in your code would result in an infinite loop, since you don't handle actualByteCount == 0. You can catch and handle the resulting exception with the functions in: http://haskell.org/ghc/docs/latest/html/libraries/base/Control-Exception.html AGL -- Adam Langley [EMAIL PROTECTED] http://www.imperialviolet.org 650-283-9641 ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] Simple network client
On Jan 31, 2008 5:07 AM, Gary Bickford <[EMAIL PROTECTED]> wrote: > Wasn't there a Linux file system (possibly a FUSE user-space one) that > worked on writable CDs? IIRC it worked by marking the previous copy of > the file as erased, and writing a new copy. Probably you're thinking of layering a unionfs over the top. It works well in my experience. AGL -- Adam Langley [EMAIL PROTECTED] http://www.imperialviolet.org 650-283-9641 ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] Simple network client
On Jan 30, 2008 1:07 PM, Adam Langley <[EMAIL PROTECTED]> wrote: > So, if I don't hear otherwise soon, I'll probably push a new version > of binary-strict later on today with the interface above. It's in the darcs now, http://darcs.imperialviolet.org/binary-strict AGL -- Adam Langley [EMAIL PROTECTED] http://www.imperialviolet.org 650-283-9641 ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] Simple network client
On Jan 30, 2008 12:04 PM, Bryan O'Sullivan <[EMAIL PROTECTED]> wrote: > Adam Langley wrote: > I'd have expected it to look more like this: > > data Result a = Failed String > | Finished B.ByteString a > | Partial (B.ByteString -> Result a) > > (The change here is from a list to a singleton.) I don't think I care > much for the extra boxing and reversing this involves. Well, since you're probably the /only/ user you can pretty much say how it works ;) The original interface was designed so that you can yield a list of results as you parse. I guess that, since you get the remainder anyway, you can chain these together if you like anyway. So, if I don't hear otherwise soon, I'll probably push a new version of binary-strict later on today with the interface above. AGL -- Adam Langley [EMAIL PROTECTED] http://www.imperialviolet.org 650-283-9641 ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] ffi and pass by value structs
On Jan 30, 2008 5:16 AM, Federico Squartini <[EMAIL PROTECTED]> wrote: > foreign import ccall "print_couple_ptr" printCouple::Ptr Couple->IO Int > > But not the first, as a struct it's not an atomic type. > So how does one solve this issue? (This applies to x86-64 only!) In the case of your struct couple, I believe that it's small enough that it will be passed in registers: http://www.x86-64.org/documentation/abi.pdf > 1. If the size of an object is larger than two eightbytes, or it contains > unaligned > fields, it has class MEMORY. However, it's larger than this the C compiler will actually call memcpy to make a copy before calling the function with a pointer to the copy on the parent's stack. With the Haskell FFI, the portable solution is probably to write a very simple wrapper .c file which takes a pointer and calls the real function with the contents of that pointer. Then you just call your wrapper function. You can make these functions by the dozen with the preprocessor. AGL -- Adam Langley [EMAIL PROTECTED] http://www.imperialviolet.org 650-283-9641 ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] Simple network client
On Jan 30, 2008 4:32 AM, Jules Bean <[EMAIL PROTECTED]> wrote: > The third, but more sophisticated answer is to use non-blocking IO, read > 'only what is available', decide if it's enough to process, if not > store it in some local buffer until next time. This is much more work > and easy to implement bugs in, but you need it for true streaming > protocols. In that case hGetBufNonBlocking is your friend. If using bytestrings from the network, the network-bytestring package is what you need. If you're parsing small lumps from the network, Data.Binary's failures will probably make life harder than it should be for you since they can only be caught in IO. See http://www.haskell.org/haskellwiki/DealingWithBinaryData about a strict Get which might work for you. Also, if you want the above approach (read a bit, see if it's enough), see IncrementalGet in the binary-strict package which is a Get with a continuation monad that stops when it runs out of bytes and returns a continuation that you can give more data to in the future. AGL -- Adam Langley [EMAIL PROTECTED] http://www.imperialviolet.org 650-283-9641 ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] Performance Issues
> This computes 100!. This version takes 8m29.189s to execute. > Replace foldr1 with foldr and that goes down to 7m4.315s. Replace > product' with the Prelude product and it takes only 6m17.685s. Why is > that so? I'm using ghc 6.8.1 on Mac OS X. I'm guessing that the speedup with the Prelude product is because it's a left fold. foldr goes to the end of the list and starts applying backwards, up the thunks. A left fold just keeps an accumulator as it walks down. The foldr1 to foldr speedup might be because foldr1 has three cases, to foldr's two: http://haskell.org/ghc/docs/latest/html/libraries/base/src/GHC-List.html#foldr1 AGL -- Adam Langley [EMAIL PROTECTED] http://www.imperialviolet.org 650-283-9641 ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] Simple network client
On Jan 29, 2008 6:28 AM, Timo B. Hübel <[EMAIL PROTECTED]> wrote: > Hm, unfortunately not for me (Linux, GHC 6.8.2) ... That's odd, because it works for me on the exact same setup. There was a similar bug in lazy bytestring's hGetContents a while back which involve it waiting for a whole chunk and not returning short reads, but from watching the strace of this code, GHC is reading byte-by-byte (which is actually pretty dumb, but functions). Can you compile both with: % ghc --make file.hs And run them with: % strace -o /tmp/trace ./file (obviously, you're running strace twice, with different binaries and output files) and send me the resulting traces? (They'll be quite big, so I don't know if you want to spam that whole list with them) Cheers AGL -- Adam Langley [EMAIL PROTECTED] http://www.imperialviolet.org 650-283-9641 ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
[Haskell-cafe] ANN: RPCA and fec
Doubling up an announcement email again to reduce the traffic on the list. First: RPCA is an RPC system. This first release is pretty lacking, simple client-server interactions work but there's no lame-duck handling, load balancing, security etc. RPCA uses codec-libevent's tagged data structures for serialisation. Tagged structures allow you to grow you RPCs incrementally, which is very important in large programs. Very simple tests should that 1000's of RPCs a second isn't a problem. There's also the start of a C library for the protocol, written on libevent and glib. To give a taste, here's what you need to write on the server side: addNumbers :: TestRPC.Addnumbersrequest -> ((Maybe TestRPC.Addnumbersreply) -> STM ()) -> IO () addNumbers (TestRPC.Addnumbersrequest { TestRPC.addnumbersrequest_a = a , TestRPC.addnumbersrequest_b = b }) cb = do atomically $ cb $ Just $ TestRPC.addnumbersreplyEmpty { TestRPC.addnumbersreply_c = a + b } and on the client side... rpc chan "add" (TestRPC.Addnumbersrequest 400 300) 0.1 That's a synchronous call with a timeout. You can, of course, call them asynchronously as well (and from multiple threads etc) It's highly experimental and can be found in Hackage and at http://darcs.imperialviolet.org/darcsweb.cgi?r=network-rpca;a=summary FEC is an erasure coding library, based on zfec by Zooko, based on code by Luigi Rizzo. The most widely known example of an erasure code is the RAID-5 algorithm which makes it so that in the event of the loss of any one hard drive, the stored data can be completely recovered. The algorithm in the fec package has a similar effect, but instead of recovering from the loss of only a single element, it can be parameterized to choose in advance the number of elements whose loss it can tolerate. Hackage: http://hackage.haskell.org/packages/archive/fec/0.1.1/doc/html/Codec-FEC.html It's new code, but the library that it's based on is well tested so should be sound. AGL -- Adam Langley [EMAIL PROTECTED] http://www.imperialviolet.org 650-283-9641 ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] Upgrading ByteString causes (seemingly) impossible RTS linker errs
On Jan 27, 2008 12:24 PM, Don Stewart <[EMAIL PROTECTED]> wrote: > It should be possible to specify that your lib depends on exactly > 0.9.0.1 in the .cabal file. In the same general area. When you upgrade something like bytestring and have to update everything that was built with it (otherwise you'll get type errors because the ByteString types from the two different versions are different) cabal-install is of no help. Since it already believes that I have those packages installed (correctly) if refuses to install them again so I have to do it all by hand. I realise that I should be sending darcs patches, not bug reports, but a --force to reinstall an already installed package would be great. I guess, ideally, cabal-install would know to rebuild everything that depends on x when I upgrade x. darcs patches might follow if I can get round to it (or the next time I update bytestring...) Cheers, -- Adam Langley [EMAIL PROTECTED] http://www.imperialviolet.org 650-283-9641 ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] CPP and INLINE pragmas
Thanks Isaac and Malcolm. That neatly solves all my problems! AGL -- Adam Langley [EMAIL PROTECTED] http://www.imperialviolet.org 650-283-9641 ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] Re: ANN: A triple of new packages for talking tothe outside world
On Jan 15, 2008 7:33 PM, Adam Langley <[EMAIL PROTECTED]> wrote: > Ok, no TH ;) I've just uploaded binary-strict 0.2.2 to Hackage which factors most of the common code out via CPP. Hopefully I didn't break anything. AGL -- Adam Langley [EMAIL PROTECTED] http://www.imperialviolet.org 650-283-9641 ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
[Haskell-cafe] CPP and INLINE pragmas
Since CPP mode removes newlines in the out macro expansion. It appears to be impossible to have a macro expand to a function with an INLINE pragma since it appears to need to be in its own line. For example: #define GETHOSTWORD(name, m, type) \ {-# INLINE name #-} \ name :: m type ; \ name = getPtr (sizeOf (undefined :: type)) \ Does work (since you can't follow the INLINE pragma with anything else on the same line. Likewise: #define GETHOSTWORD(name, m, type) \ name :: m type ; \ name = getPtr (sizeOf (undefined :: type)) \ {-# INLINE name #-} Also doesn't work since the INLINE needs to start at the beginning of a line. Has anyone a workaround for this, or a way to get the preprocessor to output a newline? Cheers AGL -- Adam Langley [EMAIL PROTECTED] http://www.imperialviolet.org 650-283-9641 ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] bytea decoding
2008/1/20 Bryan Green <[EMAIL PROTECTED]>: > Does anyone know of a library that will handle bytea encodings from > postgres? The bytea field that I need to access contains a jpg file. I > want to retrieve it from the database and write it out for an image display > program. I'd love to see Don give the correct ByteString solution, but here's a slow version I knocked up (probably buggy): import qualified Data.ByteString as B import Data.ByteString.Internal (c2w) import Data.Maybe (catMaybes) import Data.List (mapAccumL) byteaDecode :: B.ByteString -> B.ByteString byteaDecode = B.pack . catMaybes . snd . mapAccumL f initState . B.unpack where initState = (0, 0) f (0, _) 92 = ((1, 0), Nothing) f (0, _) x = ((0, 0), Just x) f (1, _) 92 = ((0, 0), Just 92) f (3, n) x = ((0, 0), Just (n * 8 + (x - 48))) f (c, n) x = ((c + 1, n * 8 + (x - 48)), Nothing) AGL -- Adam Langley [EMAIL PROTECTED] http://www.imperialviolet.org 650-283-9641 ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] Re: ANN: A triple of new packages for talking tothe outside world
On Jan 17, 2008 11:05 AM, Dominic Steinitz <[EMAIL PROTECTED]> wrote: > I'm not clear what happens if you > are reading from a socket and not all the input has arrived but I'll think > about that over the weekend. At the moment, BitGet deals with strict ByteStrings only. One could use it within a standard Get monad by getting a strict ByteString from the lazy input. I believe that lazy ByteStrings got fixed a while back so that reading from a socket doesn't block reading a whole block. (i.e. if you trickle data, byte by byte, to a socket a lazy ByteString should return a spine of 1 byte strict ByteStrings.) A fully lazy BitGet would also be possible, of course, I've just not written it yet ;) > Adam Langley imperialviolet.org> writes: > Another thought: could e.g. getRightByteString be in the IO monad and then I > don't have to run the Get(?) monad? Or is that a really stupid question? If it were in the IO monad, I guess that you're suggesting that it read from a handle? If that were the case, the remainder of the last byte would have to be discarded because one can only read whole bytes from a Handle and there's no mechanism for pushing back into it. It's certainly possible to do, but I think a quick wrapper around a BitGet would be the way to do it. If it's particually desirable I can add it, although I'll admit that it seems a bit odd and I'm wondering what your use case is. Cheers AGL -- Adam Langley [EMAIL PROTECTED] http://www.imperialviolet.org 650-283-9641 ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] Re: ANN: A triple of new packages for talking tothe outside world
On Jan 16, 2008 2:41 PM, Dominic Steinitz <[EMAIL PROTECTED]> wrote: > tlv_ bin = >do tagValueVal <- getBits bin 5 > tagConstructionVal <- getBits bin 1 > tagTypeVal <- getBits bin 2 > > I'm sure I'm wrong but putting bits into [Bool] doesn't look very efficient. > Of > course, NewBinary didn't address what happened for n >= 8. Some possibilities > are a) not allowing more than 8 b) returning [Word8] or c) (which I thought > was > where we'd go) a ByteString with some sort of padding. BitGet is just an API RFC at the moment, so I'm just describing it here - not trying to justify it. In BitGet there's getAsWord[8|16|32|64] which take a number of bits ($n$) and returns the next $n$ bits in the bottom of a Word$x$. Thus, getAsWord8 is what you call getBits and, if you had a 48 bit number, you could use getAsWord64 and the bottom 48-bits of the resulting Word64 would be what you want. Equally, asking for more than $x$ bits when calling getAsWord$x$ is a mistake, however I don't check for it in the interest of speed. There are also get[Left|Right]ByteString which return the next $n$ bits in a ByteString of Word8's. The padding is either at the end of the last byte (left aligned) or at the beginning of the first byte (right aligned). If you did want a [Bool], you could use: bits <- sequence $ take n $ repeat getBit AGL -- Adam Langley [EMAIL PROTECTED] http://www.imperialviolet.org 650-283-9641 ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] Re: ANN: A triple of new packages for talking tothe outside world
On Jan 15, 2008 5:01 PM, Neil Mitchell <[EMAIL PROTECTED]> wrote: > That would be bad. Then you'll have gone from Data.Binary being > portable code, to being GHC specific code, and I will cry :'-( Ok, no TH ;) AGL -- Adam Langley [EMAIL PROTECTED] http://www.imperialviolet.org 650-283-9641 ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] Re: ANN: A triple of new packages for talking tothe outside world
On Jan 15, 2008 3:26 PM, Don Stewart <[EMAIL PROTECTED]> wrote: > Ok. That's awesome. I guess if you do all the TODOs for Binary like > this, we should merge the code back in :) Well, at the moment I'm pretty unhappy with the amount of code duplication required both within binary-strict and between binary and binary-strict. I think this code needs a whole lot of restructuring (maybe a bit of TH for generating the common bits). I'll get to that when it appears that the API seems reasonable. AGL -- Adam Langley [EMAIL PROTECTED] http://www.imperialviolet.org 650-283-9641 ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] Re: ANN: A triple of new packages for talking tothe outside world
On Jan 10, 2008 10:45 AM, Don Stewart <[EMAIL PROTECTED]> wrote: > That's pretty much what we envisaged as the approach to take. > Monad transformers adding some bit-buffer state over Get/Put. For anyone who's still reading this thread... I've just uploaded[1] binary-strict 0.2.1 which includes Data.Binary.Strict.BitGet - a Get like monad which works by the bit. I'm afraid that Haddock 2 is choaking on {-# UNPACK #-}, so I don't have the HTML documentation to point to. (And I thought that Haddock 2 was going to fix all those parsing issues :( - hopefully I'm just doing something stupid). [1] http://hackage.haskell.org/cgi-bin/hackage-scripts/package/binary-strict-0.2.1 AGL -- Adam Langley [EMAIL PROTECTED] http://www.imperialviolet.org 650-283-9641 ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] Re: ANN: A triple of new packages for talking tothe outside world
On Jan 9, 2008 5:01 PM, David Roundy <[EMAIL PROTECTED]> wrote: > But I can't imagine an implementation in which this change wouldn't slow > down getBytes for the normal case. Perhaps the slowdown would be small, > but it seems unwise to enforce that slowness at the API level, when we've > already got a perfectly good API for fast binary IO. Maybe there's some > type hackery you could do to avoid a speed penalty, but that's a lot to add > for a somewhat dubious benefit. I believe that it would be an additional if statement in the fast path at least. How about a BitGet monad which get be run in the Get monad? > test :: Get () > test = do > runBitGet 2 (do >getBitField 2) So the first argument to runBitGet is the number of bytes to parse for bit fields and then functions in BitGet can extract bit-length ints etc. Anyone like that idea? AGL -- Adam Langley [EMAIL PROTECTED] http://www.imperialviolet.org 650-283-9641 ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] Data.Binary and serialising doubles in IEEE format
On Jan 9, 2008 5:18 AM, allan <[EMAIL PROTECTED]> wrote: > Essentially then, is there anyone that can write my 'putDouble' function > or give some hint as to how I might do it. > For the moment assume that I'm not really concerned with portability > across platforms at least for the time being (I certainly don't think > that the C program I'm attempting to communicate with is particularly > portable anyway). > I happen to have the reverse functions lying around which might be some help in doing this. They aren't terribly beautiful, but they'll take an IEEE double or float from a Bytestring and return the value + the remaining ByteString. (They aren't very well tested, but they have been found to mostly work). ddouble :: BS.ByteString -> (Double, BS.ByteString) ddouble bytes = (encodeFloat fraction $ fromIntegral exp, rest) where fraction = sign * fromIntegral ((n .&. 0xf) .|. (if rawExp > 0 then 0x10 else 0)) sign = if n `shiftR` 63 == 0 then 1 else -1 exp = rawExp - (1023 + 52) rawExp = (n `shiftR` 52) .&. 0x7ff (b, rest) = BS.splitAt 8 bytes n :: Word64 n = foldl1 (.|.) $ map (\(s, v) -> (fromIntegral v) `shiftL` s) $ zip [0,8..56] $ BS.unpack b dfloat :: BS.ByteString -> (Float, BS.ByteString) dfloat bytes = (encodeFloat fraction $ fromIntegral exp, rest) where fraction = sign * fromIntegral ((n .&. 0x7f) .|. (if rawExp > 0 then 0x80 else 0)) sign = if n `shiftR` 31 == 0 then 1 else -1 exp = rawExp - (127 + 23) rawExp = (n `shiftR` 23) .&. 0xff (b, rest) = BS.splitAt 4 bytes n :: Word32 n = foldl1 (.|.) $ map (\(s, v) -> (fromIntegral v) `shiftL` s) $ zip [0, 8, 16, 24] $ BS.unpack b AGL -- Adam Langley [EMAIL PROTECTED] http://www.imperialviolet.org 650-283-9641 ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] ANN: A triple of new packages for talking to the outside world
On Jan 7, 2008 11:17 AM, Bryan O'Sullivan <[EMAIL PROTECTED]> wrote: > > It would seem that there would be three possible outcomes from an > > incremental Get: > > - Failure: some bitstreams are just invalid and no amount of extra > > data will ever fix that > > - Complete [Result]: the last chunk of data has been processed. > > Maybe this should also include the remainder of the data? > > - Partial Cont [Result]: needs more data, but here's a (possibly > > empty) list of results so far. > > Yes, that's more or less exactly what I had in mind. Ok, see http://www.imperialviolet.org/IncrementalGet.hs Hot off the keyboard and lacking documentation and comments ;) There's a small test function at the bottom which demonstrates an infinite parser which extracts 16-bit ints from whatever you feed it. Questions: 1) Should Finished include the remainder of the ByteString (e.g. that which wasn't used by that parser) 2) I've no idea what I've done to the parse speed 3) I removed all the lookahead functions because, if the look ahead runs out of data it's not clean what should be done. We can request more with a partial result, but if the user doesn't have any more data to give there's no way, currently, to signal this and cause the lookahead to backtrack. Also, if we let the lookahead request more data via a Partial, then there's no way for the user to tell that Partial apart from one which has "real" values. But if this is useful to you, make any requests. I'll (hopefully) do them, clean it up and push a new release of binary-strict. AGL -- Adam Langley [EMAIL PROTECTED] http://www.imperialviolet.org 650-283-9641 ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] ANN: A triple of new packages for talking to the outside world
On Jan 6, 2008 9:13 PM, Bryan O'Sullivan <[EMAIL PROTECTED]> wrote: > Ooh, nice. We could really do with an incremental version, too, which > could be spoonfed chunks of bytes, and dole out values as > deserialisation completes. > > Passing back a Left String is in some sense not much of an improvement > over calling error, as if I merely doesn't have enough bytes accumulated > yet, I can't restart parsing from the point the bytes ran out. Any > chance you'd like to hand back a continuation instead? The Left String was just a way to return an error string, otherwise a parse failure can be a little oblique. I think an incremental parser would be sufficiently different that it would be a different monad. Of course, the code duplication (again) makes me a sad panda, but avoiding that involves a lot of work on the binary package and the maintainers were uninterested last time I tried that. It would seem that there would be three possible outcomes from an incremental Get: - Failure: some bitstreams are just invalid and no amount of extra data will ever fix that - Complete [Result]: the last chunk of data has been processed. Maybe this should also include the remainder of the data? - Partial Cont [Result]: needs more data, but here's a (possibly empty) list of results so far. Applying a ByteString to the Cont from the Partial result would result in one of the same three outcomes etc. What do you think of that? It wouldn't be too hard to make up something like that using ContT if it would be useful to you. AGL -- Adam Langley [EMAIL PROTECTED] http://www.imperialviolet.org 650-283-9641 ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
[Haskell-cafe] ANN: A triple of new packages for talking to the outside world
Over the Xmas break I made some headway on writing an RPC package since many of the ideas that I want to play with involve such a thing as a basic building block. However, some might find some of the spin-off packages that I wrote along the way useful. * binary-strict: This is mostly a cut-n-paste job from the excellent binary package which provides Data.Binary.Strict.Get - a monad which is a drop in replacement for Get, but which parses strict ByteStrings and returns an Either, thus one can parse binary data without triggering an exception on failure. * codec-libevent: This parses the "tagging" IDL files introduced in libevent[1] 1.4.0-beta and later and can generate the Haskell code for them. These are platform-independent "structs" (in the C sense) and libevent provides the complementary C infrastructure for them. You should be able to understand almost everything these is to know about them from a quick example: struct msg { string from_name = 1; string to_name = 2; optional struct[kill] attack = 3; array struct[run] run = 4; } [1] http://monkey.org/~provos/libevent/ * control-timeout: This module provides simple timeouts - e.g. actions which occur after a given number of seconds and which can be canceled. It does it in a not-totally-stupid fashion so that you don't need to worry about setting hundreds of them. AGL -- Adam Langley [EMAIL PROTECTED] http://www.imperialviolet.org 650-283-9641 ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
[Haskell-cafe] Getting links to Haddock documentation in Hackage
Some packages[1] have links on their Hackage pages to the haddock generated documentation for each exported module[2]. However, many[3] don't. What's the secret to getting this generated documentation to work with Hackage? Even packages for which `runhaskell Setup.lhs haddock` runs don't (always) get the magic links. Cheers, AGL [1] http://hackage.haskell.org/cgi-bin/hackage-scripts/package/bzlib-0.4.0.1 [2] http://hackage.haskell.org/packages/archive/bzlib/0.4.0.1/doc/html/Codec-Compression-BZip.html [3] http://hackage.haskell.org/cgi-bin/hackage-scripts/package/tar-0.1.1.1 -- Adam Langley [EMAIL PROTECTED] http://www.imperialviolet.org 650-283-9641 ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] very elementary "import" problem .. help
On Nov 23, 2007 1:43 PM, Galchin Vasili <[EMAIL PROTECTED]> wrote: > I am loading the above script in ghci. However, System.Posix, System.Process > and STM cannot be found. Network and Directory are found. I assume when ghc > is installed on Windows that the Windows registry get populated with paths > to GHC 'library" directory paths. Please someone tell me why I am having > trouble importing these modules/libraries. Well, NT used to pretend to have a POSIX layer because of some DOD regulations I believe. I'm not sure if that still stands and wouldn't be surprised if System.Posix just doesn't exist for Windows. System.Process is marked as portable, so I'm not sure why that one wouldn't be found. Finally, I believe that you want Control.Concurrent.STM. AGL -- Adam Langley [EMAIL PROTECTED] http://www.imperialviolet.org 650-283-9641 ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] Distributed haskell using Hadoop
On 10/16/07, Brad Clow <[EMAIL PROTECTED]> wrote: > I would prefer a more Haskell orientated solution and welcome any > suggestions. If not maybe this will be of use to others. Well, Hadoop is aiming towards a Google style of cluster processing and the path towards that is pretty clear: 1) An XDR like serialisation scheme with support for backwards compatibility (which involves unique-for-all-time ids in the IDL and "required", "optional" etc tag). Data.Binary would be a great start for this, but it's sadly lazy in parsing and they never applied my patch for optional strictness so one would probably have to start from scratch. 2) An RPC system which handles the most common use case: arguments and replies are serialised using the above system, TCP transport, simple timeouts, STM for concurrency. Then you can start doing cool stuff like using the GHC API for code motion and building a simple MapReduce like framework etc. -- Adam Langley [EMAIL PROTECTED] http://www.imperialviolet.org 650-283-9641 ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] ANNOUNCE: binary 0.4: high performance, pure binary parsing and serialisation
On 10/6/07, Felipe Almeida Lessa <[EMAIL PROTECTED]> wrote: > May I ask what are the changes? I didn't find some sort of changelog anywhere. There's the darcs changes list. The descriptions there in are .. terse :) But here's a selection: * Add getLazyByteStringNul. -- | Get a lazy ByteString that is terminated with a NUL byte. Fails -- if it reaches the end of input without hitting a NUL. getLazyByteStringNul :: Get L.ByteString * Fix strictness bug in runGetState that led to runtime errors. * Port binary to ghc 6.8 * add parallel driver (appears to be a parallel quicktest - which is cool because the tests took quite a while previously). -- Adam Langley [EMAIL PROTECTED] http://www.imperialviolet.org 650-283-9641 ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] GHC optimisations
On 8/23/07, Neil Mitchell <[EMAIL PROTECTED]> wrote: > It's not even referentially transparent in all cases. a == b may fail > while the double's are in the high precision registers, and then > succeed later on in the program once they are truncated. I think you > have to specify -fexcess-precision with GHC to get this behaviour. Also, if you tell GCC to use SSE (there are a number of flags, like -march and -msse, -msse2 etc which can do this). It can stop using the 80-bit x87 registers for floating point and switch to 64-bit SSE registers. Then you *don't* get this affect, but I'm not sure you can be sure that you don't ever get it. Floating point numbers make me sad :( AGL -- Adam Langley [EMAIL PROTECTED] http://www.imperialviolet.org 650-283-9641 ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] Parsing binary data.
On 8/19/07, Matthew Sackman <[EMAIL PROTECTED]> wrote: > But it's vastly harder to do that for floats / non-integers. Now I know > that the number classes in the Prelude are basically broken anyway and > all really need rewriting, but it does seem completely arbitrary that > Words somehow are only allowed to contain whole numbers! Well, see the attached patch to Data.Binary to add putFloat[32|64][be|le]. I got bored, so adding the Get functions is an exercise for the reader :) (And so because I think it needs unsafeSomethingIO and I'm a little unsure about that). If these functions would be useful for you, you should bug the binary team to add something similar. AGL -- Adam Langley [EMAIL PROTECTED] http://www.imperialviolet.org 650-283-9641 data-binary-float.darcs Description: Binary data ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] Parsing binary data.
On 8/18/07, Matthew Sackman <[EMAIL PROTECTED]> wrote: > Also, one thing to watch out for is the fact the existing Get and Put > instances may not do anything like what you expect. For example, for > some reason I expected that the instances of Get and Put for Float and > Double would send across the wire Floats and Doubles in IEEE floating > point standard. How wrong I was... Ah, those aren't instances of Get and Put, but of Binary[1]. You use the Binary instances via the functions 'get' and 'put' (case is important). Get and Put provide actions like "putWord32be", for which the resulting bits are pretty much universally accepted. Binary has default instances which uses Get and Put to serialise Haskell types like [Int], or (Float, Float). Here the resulting bits aren't documented, but you can read the code and I have some C code for dealing with them somewhere if anyone is interrested. The serialisation of Float is, indeed, nothing like IEEE in either endianness. (* and, although Get isn't currently a class, I have sent patches to dons to make it so, with a default instance which matches current behaviour and speed, and an alternative which returns a Maybe, removing a little bit of lazyness in cases where you want to handle parse failures in pure code. Hopefully something will happen with this at the next sprint ;) ) [1] http://www.cse.unsw.edu.au/~dons/binary/Data-Binary.html#1 -- Adam Langley [EMAIL PROTECTED] http://www.imperialviolet.org 650-283-9641 ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] Parsing binary data.
On 8/18/07, Peter Cai <[EMAIL PROTECTED]> wrote: > As the old version of this component is written in C, it's very natural that > this protocol is base on C structure definitions, which are, unfortunately, > very complicated. And the worse is that every field in every structure must > be converted to Network Endian. You could certainly try Data.Binary[1] for this. It has a nice Get monad with methods such as getWord32be which sounds like it might be what you want. One caveat is that it's fully lazy - you get the result immediately and parse errors can only be caught as exceptions when you actually come to using the result. This is perfect for very large messages, but might be slightly wrong for you. [1] http://hackage.haskell.org/cgi-bin/hackage-scripts/package/binary-0.3 -- Adam Langley [EMAIL PROTECTED] http://www.imperialviolet.org 650-283-9641 ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe