Re: [Haskell-cafe] Google Android

2008-09-25 Thread Adam Langley
2008/9/24 Galchin, Vasili <[EMAIL PROTECTED]>:
> Do there currently (or in the works) exist FFI bindings for Google's
> Android API?

The Android API is obviously a Java API, additionally Android will
only execute DEX bytecode programs, so you would first need a way to
compile Haskell to Java, Java bytecode or DEX directly. A Google
search for "haskell java" turns up at least one good candidate[1], but
if you manage to get that working well, binding the APIs is a rather
trivial task ;)


[1] http://www.cse.unsw.edu.au/~pls/thesis-topics/ghcjava.html

AGL

-- 
Adam Langley [EMAIL PROTECTED] http://www.imperialviolet.org
___
Haskell-Cafe mailing list
Haskell-Cafe@haskell.org
http://www.haskell.org/mailman/listinfo/haskell-cafe


Re: [Haskell-cafe] Re: Iteratee-based IO

2008-09-21 Thread Adam Langley
On Sat, Sep 20, 2008 at 8:17 PM, Don Stewart <[EMAIL PROTECTED]> wrote:
> Wonderful. Thanks Oleg, I'll be studying these on the trip up to
> Victoria. See you there!

Something similar is available in binary-strict[1], although there I
have (versions of) IE_Done, IE_Cont and (additionally) IE_Fail, for
indicating a parse failure.

The actual implementation is unpleasant, although functional.


[1]  
http://darcs.imperialviolet.org/darcsweb.cgi?r=binary-strict;a=headblob;f=/src/Data/Binary/Strict/IncrementalGet.hs


-- 
Adam Langley [EMAIL PROTECTED] http://www.imperialviolet.org
___
Haskell-Cafe mailing list
Haskell-Cafe@haskell.org
http://www.haskell.org/mailman/listinfo/haskell-cafe


Re: [Haskell-cafe] control-timeout with gtk

2008-09-18 Thread Adam Langley
On Thu, Sep 18, 2008 at 11:38 AM, Marco Túlio Gontijo e Silva
<[EMAIL PROTECTED]> wrote:
> Maybe it's better that they stay there, even with nobody maintaining
> them.

Except that I feel shitty everytime someone someone emails me with
questions or problems that I don't have the time to deal with. It's a
constant cost for my name to be associated with them and, if nobody
cares enough to take the maintainership, they should probably die -
Hackage has a lot of packages these days. Do you want control-timeout?
:)

AGL


-- 
Adam Langley [EMAIL PROTECTED] http://www.imperialviolet.org
___
Haskell-Cafe mailing list
Haskell-Cafe@haskell.org
http://www.haskell.org/mailman/listinfo/haskell-cafe


Re: [Haskell-cafe] control-timeout with gtk

2008-09-18 Thread Adam Langley
On Thu, Sep 18, 2008 at 10:05 AM, Judah Jacobson
<[EMAIL PROTECTED]> wrote:
> Just a guess, but this might be a problem with control-timeout's use
> of the unsafePerformIO global variables hack.  It's missing the
> standard NOINLINE annotations which prevent multiple copies of the
> global variable from being created.  See the haddock docs for
> System.IO.Unsafe.unsafePerformIO for more information.

That might well be it, should be an easy fix to try.

I'm afraid that I don't have enough time to do justice to my packages
on Hackage these days so I've just sent an email requesting that they
be removed. Anyone who wants to maintain any of them should grab it
now. The repos are all at http://darcs.imperialviolet.org

On the chopping block:

binary-strict
codec-libevent
control-timeout
ctemplate
LRU
network-connection
network-dns
network-minihttp
network-rpca
system-inotify


Cheers

-- 
Adam Langley [EMAIL PROTECTED] http://www.imperialviolet.org
___
Haskell-Cafe mailing list
Haskell-Cafe@haskell.org
http://www.haskell.org/mailman/listinfo/haskell-cafe


Re: [Haskell-cafe] Libevent FFI problems

2008-07-28 Thread Adam Langley
On Sat, Jul 26, 2008 at 4:34 AM, Levi Greenspan
<[EMAIL PROTECTED]> wrote:
> client: internal error: awaitEvent: descriptor out of range
>(GHC version 6.8.3 for i386_unknown_linux)
>Please report this as a GHC bug:
> http://www.haskell.org/ghc/reportabug
> Aborted
>
> This is caused by having more file descriptors than FD_SETSIZE (which is
> 1024) on my system. So clearly select() doesn't scale to the numbers I
> need :-(

Now, *thats* a good point. I'd still suggest that taking the time to
add epoll support to the threaded runtime would be better for you(,
and for everyone else!) than avoiding the use of GHC threads. But,
with that error, you're correct that the current select based system
is insufficient.



AGL

-- 
Adam Langley [EMAIL PROTECTED] http://www.imperialviolet.org
___
Haskell-Cafe mailing list
Haskell-Cafe@haskell.org
http://www.haskell.org/mailman/listinfo/haskell-cafe


Re: [Haskell-cafe] Libevent FFI problems

2008-07-25 Thread Adam Langley
On Fri, Jul 25, 2008 at 1:32 PM, Levi Greenspan
<[EMAIL PROTECTED]> wrote:
> Thank you (and Christopher) for the link. I have one question though -
> I read this ticket in the GHC trac:
> http://hackage.haskell.org/trac/ghc/ticket/635 which plans to use
> epoll instead of select. The reason I thought of libevent is exactly
> the support for epoll and other better-than-select mechanisms. Is
> there progress in GHC with regard to this? This is very important for
> me since I awill not write a web-server, but rather want to play with
> Comet (i.e. having thousands of open connections as it is common for
> long-polling in addition to many relatively short request/response
> based connections).

Bryan O'Sullivan make an off-the-cuff remark about doing this work at
a talk a few months back, however I haven't heard anything about him
starting. I think it's safe to assume that no one has taken it up yet.
Personally, some of my other projects are higher priorities at the
moment.

I'd suggest that you write your server on the select() based system
as-is for now. Then, when you need epoll you'll be sufficiently
motivated to hack up the RTS to include it ;)


AGL

-- 
Adam Langley [EMAIL PROTECTED] http://www.imperialviolet.org
___
Haskell-Cafe mailing list
Haskell-Cafe@haskell.org
http://www.haskell.org/mailman/listinfo/haskell-cafe


Re: [Haskell-cafe] Libevent FFI problems

2008-07-25 Thread Adam Langley
2008/7/25 Krzysztof Skrzętnicki <[EMAIL PROTECTED]>:
> "Developing a high-performance web server in Concurrent Haskell"
> http://www.haskell.org/~simonmar/papers/web-server-jfp.pdf (see page 15)
>
> Perhaps you might be interested in this paper also because of its topic.

That's a good reference. Also note that the paper is 6 years old and
GHC has come a long way since then. I'd suspect that the graph on page
15 would look much more favourable to Haskell these days.


AGL

-- 
Adam Langley [EMAIL PROTECTED] http://www.imperialviolet.org
___
Haskell-Cafe mailing list
Haskell-Cafe@haskell.org
http://www.haskell.org/mailman/listinfo/haskell-cafe


Re: [Haskell-cafe] Libevent FFI problems

2008-07-25 Thread Adam Langley
2008/7/23 Levi Greenspan <[EMAIL PROTECTED]>:
> I would be grateful for any advices, hints or comments. And I really
> look forward to the FFI section in the "Real World Haskell" book.

Generally it looks pretty good. I think I'm missing some C code
(function "wrapper"). However, libevent is an odd choice to wrap. The
RTS already multiplexes IO for Haskell programs. In order not to block
the RTS, the libevent using code would have to be in its own kernel
thread.

Really, the RTS needs to be ported to libevent rather than an FFI wrapping.

For your specific problem, I see you're allocating xptr, but I don't
see that you're ever poking a value into it. Indeed, I can't see that
createEvent ever uses 'x'.

Hope that helps.




AGL

-- 
Adam Langley [EMAIL PROTECTED] http://www.imperialviolet.org
___
Haskell-Cafe mailing list
Haskell-Cafe@haskell.org
http://www.haskell.org/mailman/listinfo/haskell-cafe


Re: [Haskell-cafe] Haskell, Microsoft, and interview questions

2008-06-26 Thread Adam Langley
On Thu, Jun 26, 2008 at 8:11 AM, Andrew Wagner <[EMAIL PROTECTED]> wrote:
> I did some reading up on this, and it seems interesting. It would be
> need to implement something like this in Haskell, but I can't seem to
> find any detailed specs on the data-structure. Got any
> recommendations?

Specialised for 2d only, but:

http://www.imperialviolet.org/binary/NearestNeighbour2D.hs

AGL

-- 
Adam Langley [EMAIL PROTECTED] http://www.imperialviolet.org
___
Haskell-Cafe mailing list
Haskell-Cafe@haskell.org
http://www.haskell.org/mailman/listinfo/haskell-cafe


Re: [Haskell-cafe] safe vs unsafe

2008-06-24 Thread Adam Langley
On Tue, Jun 24, 2008 at 12:08 PM, Galchin, Vasili <[EMAIL PROTECTED]> wrote:
> yes I am invoking a callback function written in Haskell.

Then you should just need to remove the "unsafe" from the foreign
import decl which leads to the callback getting called.


AGL

-- 
Adam Langley [EMAIL PROTECTED] http://www.imperialviolet.org
___
Haskell-Cafe mailing list
Haskell-Cafe@haskell.org
http://www.haskell.org/mailman/listinfo/haskell-cafe


Re: [Haskell-cafe] safe vs unsafe

2008-06-24 Thread Adam Langley
2008/6/23 Galchin, Vasili <[EMAIL PROTECTED]>:
> aio: schedule: re-entered unsafely.
>Perhaps a 'foreign import unsafe' should be 'safe'?
>
> I want to understand from an implementation viewpoint .. deeper. This is a
> GC issue yes?

Are you reentering Haskell on a path which involves an unsafe FFI
call? e.g. you enter C land via an unsafe FFI call and, on the same
call path, reenter Haskell with a callback?

http://www.cse.unsw.edu.au/~chak/haskell/ffi/ffi/ffise3.html#x6-130003.3 says:

"Optionally, an import declaration can specify, after the calling
convention, the safety level that should be used when invoking an
external entity. A safe call is less efficient, but guarantees to
leave the Haskell system in a state that allows callbacks from the
external code. In contrast, an unsafe call, while carrying less
overhead, must not trigger a callback into the Haskell system. If it
does, the system behaviour is undefined. The default for an invocation
is to be safe. Note that a callback into the Haskell system implies
that a garbage collection might be triggered after an external entity
was called, but before this call returns. Consequently, objects other
than stable pointers (cf. Section 5.6) may be moved or garbage
collected by the storage manager."

-- 
Adam Langley [EMAIL PROTECTED] http://www.imperialviolet.org
___
Haskell-Cafe mailing list
Haskell-Cafe@haskell.org
http://www.haskell.org/mailman/listinfo/haskell-cafe


Re: [Haskell-cafe] Re: "C" buffer suggestions??

2008-06-23 Thread Adam Langley
On Mon, Jun 23, 2008 at 2:27 PM, Don Stewart <[EMAIL PROTECTED]> wrote:
> So heap allocated and collected, but not moved.

My bad. In that case, you might want to work with ByteStrings all the
way since it might make building the visible interface (which probably
should use ByteStrings) easier.


AGL

-- 
Adam Langley [EMAIL PROTECTED] http://www.imperialviolet.org
___
Haskell-Cafe mailing list
Haskell-Cafe@haskell.org
http://www.haskell.org/mailman/listinfo/haskell-cafe


Re: [Haskell-cafe] Re: "C" buffer suggestions??

2008-06-23 Thread Adam Langley
On Mon, Jun 23, 2008 at 1:15 PM, Galchin, Vasili <[EMAIL PROTECTED]> wrote:
>  I tried to write a test value of Word8 without success. Can you give an 
> example?

I'm not quite sure what you would want in such an example. See [1] for
the malloc function. Then you take the result and wrap it with [2]:
 fp <- mallocBytes nbytes >>= newForeignPtr finalizerFree

When you want to use it, use [3] to unwrap fp and get a raw Ptr Word8.
You can pass these to foreign functions any place they expect a
uint8_t *, char * etc.


[1] 
http://www.haskell.org/hoogle/hoodoc.cgi?module=Foreign.Marshal.Alloc&name=mallocBytes&mode=func
[2] 
http://www.haskell.org/hoogle/hoodoc.cgi?module=Foreign.ForeignPtr&name=newForeignPtr&mode=func
[3] 
http://www.haskell.org/hoogle/hoodoc.cgi?module=Foreign.ForeignPtr&name=withForeignPtr&mode=func

-- 
Adam Langley [EMAIL PROTECTED] http://www.imperialviolet.org
___
Haskell-Cafe mailing list
Haskell-Cafe@haskell.org
http://www.haskell.org/mailman/listinfo/haskell-cafe


Re: [Haskell-cafe] IMAP and NNTP libraries

2008-06-23 Thread Adam Langley
On Sat, Jun 21, 2008 at 10:52 PM, Maurí­cio <[EMAIL PROTECTED]> wrote:
> Are there mature libraries for IMAP and NNTP
> available to Haskell?

Pretty much, if a Haskell library exist it will be listed at [1].
Since I don't see anything for IMAP, nor NNTP I can be pretty sure
that the answer is "no" I'm afraid. Perhaps you would care to write
such libraries if you have a need?


[1] http://hackage.haskell.org/packages/archive/pkg-list.html

-- 
Adam Langley [EMAIL PROTECTED] http://www.imperialviolet.org
___
Haskell-Cafe mailing list
Haskell-Cafe@haskell.org
http://www.haskell.org/mailman/listinfo/haskell-cafe


Re: [Haskell-cafe] Re: "C" buffer suggestions??

2008-06-23 Thread Adam Langley
2008/6/23 Galchin, Vasili <[EMAIL PROTECTED]>:
> Basically I want to model POSIX async io "aiocb"(
> http://uw714doc.sco.com/en/man/html.5/aiocb.5.html) in Haskell .. in
> particular the aiocb field "aio_buf"!

That's a mutable buffer, so this has to be pretty low level in your
wrapping of the AIO interface. I would keep it as a Ptr Word8 at that
point. ByteStrings are probably the correct type for the user visible
interface, but I would guess not here.

Plus, ByteString data lives in the Haskell heap so could get moved
about by the collector. Since an AIO interface is probably going to
return while the IO is still in progress, you don't want to stop the
collector, but nor do you want the data moving because the kernel's
pointer isn't going to move with it.

AGL

-- 
Adam Langley [EMAIL PROTECTED] http://www.imperialviolet.org
___
Haskell-Cafe mailing list
Haskell-Cafe@haskell.org
http://www.haskell.org/mailman/listinfo/haskell-cafe


Re: [Haskell-cafe] Re: Bit Streams

2008-06-19 Thread Adam Langley
On Wed, Jun 18, 2008 at 11:07 AM, Bryan O'Sullivan <[EMAIL PROTECTED]> wrote:
> If you provide Applicative and Alternative instances for your getter
> monads (perhaps you already do?)

I have Alternative instances, and they're used a fair bit in my HTTP
parser (network-minihttp). You're right that I should add Applicative
too; maybe this weekend.


AGL

-- 
Adam Langley [EMAIL PROTECTED] http://www.imperialviolet.org
___
Haskell-Cafe mailing list
Haskell-Cafe@haskell.org
http://www.haskell.org/mailman/listinfo/haskell-cafe


Re: [Haskell-cafe] Re: Bit Streams

2008-06-19 Thread Adam Langley
On Wed, Jun 18, 2008 at 11:00 AM, Evan Laforge <[EMAIL PROTECTED]> wrote:
> Inclusion in the official Data.Binary would be nice, especially if it
> could be a drop-in replacement, say by changing the Data.Binary
> instance definitions to Strict.Binary or something.

I have a class defined to abstract the various Get monads, i.e. you
can write generic code which will work over a strict get, or an
incremental (continuation based) get.

I don't have the same for put yet. The trick is to make it so that it
doesn't end up with any runtime cost. dons would not be happy if I
took his highly tuned binary library and made it slow :)

AGL

-- 
Adam Langley [EMAIL PROTECTED] http://www.imperialviolet.org
___
Haskell-Cafe mailing list
Haskell-Cafe@haskell.org
http://www.haskell.org/mailman/listinfo/haskell-cafe


Re: [Haskell-cafe] Re: Bit Streams

2008-06-18 Thread Adam Langley
On Wed, Jun 18, 2008 at 10:52 AM, Don Stewart <[EMAIL PROTECTED]> wrote:
> Would you recommend binary-strict over bitsyntax now?
> Or are none yet entirely satisfactory

Probably, yes. Bitsyntax was, after all, the first Haskell code I ever
wrote :) It works, but I think the monad style of binary-strict is
better.


AGL

-- 
Adam Langley [EMAIL PROTECTED] http://www.imperialviolet.org
___
Haskell-Cafe mailing list
Haskell-Cafe@haskell.org
http://www.haskell.org/mailman/listinfo/haskell-cafe


Re: [Haskell-cafe] blas bindings, why are they so much slower the C?

2008-06-18 Thread Adam Langley
On Wed, Jun 18, 2008 at 9:16 AM, Anatoly Yakovenko
<[EMAIL PROTECTED]> wrote:
> C doesn't work like that :)

Yes it can. You would have to check the disassembly to be sure, but C
compilers can, and do, perform dead code elimination.


AGL

-- 
Adam Langley [EMAIL PROTECTED] http://www.imperialviolet.org
___
Haskell-Cafe mailing list
Haskell-Cafe@haskell.org
http://www.haskell.org/mailman/listinfo/haskell-cafe


Re: [Haskell-cafe] Re: Bit Streams

2008-06-18 Thread Adam Langley
On Wed, Jun 18, 2008 at 12:46 AM, Dominic Steinitz
<[EMAIL PROTECTED]> wrote:
> OK, so today I tried to write my first program using the Binary library.
> And I've hit a snag: It appears the library will only handle data that
> is byte-aligned. So if I try to write three Bool values, it uses three
> bytes, not three bits.

There's a BitGet[1] and BitPut in the Hackage version of
binary-strict. However, these versions work on strict ByteStrings.
Putting, for one, is pretty inefficient. In the darcs version[2] of
binary-strict, there's a fully lazy BitPut[3].

There was a request for a lazy BitGet, but I never wrote it. Most of
the time, bit fields are small and you can use the lazy Get from
binary to extract a ByteString and parse that with strict BitGet.
However, the BitPut should be just fine (including doing things like
writing ByteStrings by shifting each byte etc).

If it's missing anything obvious, ping me and I'll see what I can do.

Cheers,


[1] 
http://darcs.imperialviolet.org/darcsweb.cgi?r=binary-strict;a=headblob;f=/src/Data/Binary/Strict/BitGet.hs
[2] http://darcs.imperialviolet.org/binary-strict/
[3] 
http://darcs.imperialviolet.org/darcsweb.cgi?r=binary-strict;a=headblob;f=/src/Data/Binary/BitPut.hs

-- 
Adam Langley [EMAIL PROTECTED] http://www.imperialviolet.org
___
Haskell-Cafe mailing list
Haskell-Cafe@haskell.org
http://www.haskell.org/mailman/listinfo/haskell-cafe


Re: [Haskell-cafe] who maintains the POSIX/Unix Unistd.hs?

2008-05-07 Thread Adam Langley
2008/5/7 Galchin, Vasili <[EMAIL PROTECTED]>:
> Hello,
>
> I am trying to find the definition for the data type
> "TimeSpec"/"CTimeSpec".


http://www.haskell.org/ghc/docs/latest/html/libraries/unix/src/System-Posix-Unistd.html

Doesn't look like it's exported from anywhere, just used internally
for c_nanosleep


AGL

-- 
Adam Langley [EMAIL PROTECTED] http://www.imperialviolet.org
___
Haskell-Cafe mailing list
Haskell-Cafe@haskell.org
http://www.haskell.org/mailman/listinfo/haskell-cafe


[Haskell-cafe] Re: RFC: A standardized interface between web servers and applications or frameworks (ala WSGI)

2008-04-14 Thread Adam Langley
On Mon, Apr 14, 2008 at 4:54 AM, Daniel Yokomizo
<[EMAIL PROTECTED]> wrote:
>  Both request and response accept any entity headers and 7.1 (of RFC
>  2616) says that a valid entity header is an extension header, which
>  can be any kind of header.

Is wasn't suggesting that other headers be dropped, just that they
remain as strings.

>  IMHO it would be better to create a two layered approach. The bottom
>  layer handles the request as a bunch of strings, just checks for
>  structural correctness (i.e. break the headers by line and such)
>  without checking if the headers are correct. The top layer provides a
>  bunch of parser combinators to validate, parse and sanitize the
>  request so a library can create its own contract:

Ok, I think I'm convinced by this argument. I'd hope that a standard
set of header parsers be defined, and that an application which only
cares about 2616 headers can do call a single function to parse them
all, but I no longer advocate that the base interface use parsed forms
of headers.

Also, parsing URLs seems to be pretty uncontroversial (maybe parsing
key, value pairs from the path, maybe not)

AGL

-- 
Adam Langley [EMAIL PROTECTED] http://www.imperialviolet.org
___
Haskell-Cafe mailing list
Haskell-Cafe@haskell.org
http://www.haskell.org/mailman/listinfo/haskell-cafe


[Haskell-cafe] Re: RFC: A standardized interface between web servers and applications or frameworks (ala WSGI)

2008-04-13 Thread Adam Langley
On Sun, Apr 13, 2008 at 6:32 PM, Chris Smith <[EMAIL PROTECTED]> wrote:
>  Does old code that handled these headers stop working, just because it
>  was looking in the "other" section, but now needs to check a field
>  dedicated to that header?

Yes, but it would be very sad if we couldn't do common header parsing
because of this.

I'd suggest that all the headers given in RFC 2616 be parsed and
nothing else. That leaves the question of how we would handle the
addition of any extra ones in the future. Firstly, packages could
depend on a given version of this interface and we declare that the
set of handled headers doesn't change within a major version.

Better would be some static assertion that the interface doesn't
handle some set of headers. Maybe there's a type trick to do this, but
I can't think of one, so we might have to settle for a non static:

checkUnparsedHeaders :: [String] -> IO ()

Which can be put in 'main' (or equivalent) and can call error if
there's a mismatch.


AGL

-- 
Adam Langley [EMAIL PROTECTED] http://www.imperialviolet.org
___
Haskell-Cafe mailing list
Haskell-Cafe@haskell.org
http://www.haskell.org/mailman/listinfo/haskell-cafe


[Haskell-cafe] Re: RFC: A standardized interface between web servers and applications or frameworks (ala WSGI)

2008-04-13 Thread Adam Langley
On Sun, Apr 13, 2008 at 4:59 AM, Johan Tibell <[EMAIL PROTECTED]> wrote:
>  * Using a different set of data types would work better.

Give that this is Haskell, I'd suggest more types ;)

HTTP headers aren't just strings and, at the risk of tooting my own
horn, I'll point to the Headers structure in [1]. Likewise, URLs have
lots of structure that should just be handled in one place [2]

[1] 
http://darcs.imperialviolet.org/darcsweb.cgi?r=network-minihttp;a=headblob;f=/Network/MiniHTTP/Marshal.hs
[2] 
http://darcs.imperialviolet.org/darcsweb.cgi?r=network-minihttp;a=headblob;f=/Network/MiniHTTP/URL.hs


AGL

-- 
Adam Langley [EMAIL PROTECTED] http://www.imperialviolet.org
___
Haskell-Cafe mailing list
Haskell-Cafe@haskell.org
http://www.haskell.org/mailman/listinfo/haskell-cafe


Re: [Haskell-cafe] ghc

2008-04-10 Thread Adam Langley
2008/4/10 Cetin Sert <[EMAIL PROTECTED]>:
> Is GHC required to be installed on the target OS I compile Haskell binaries
> for in order for these binaries to run? I need a quick answer on that!

Short answer: no. GHC can produce binaries which don't have any
runtime deps on GHC libs.

One thing that you might need on the target host is GMP, however
that's pretty common.


AGL

-- 
Adam Langley [EMAIL PROTECTED] http://www.imperialviolet.org
___
Haskell-Cafe mailing list
Haskell-Cafe@haskell.org
http://www.haskell.org/mailman/listinfo/haskell-cafe


Re: [Haskell-cafe] [GSoC] Student applications deadline extended one week

2008-03-31 Thread Adam Langley
On Mon, Mar 31, 2008 at 12:00 PM, Manlio Perillo
<[EMAIL PROTECTED]> wrote:
>  Since Nginx is asynchronous, how can be solved the producer-consumer
>  problem (that is, the Haskell program produces more data that Nginx can
>  send to the client without blocking)?

I assume that the Haskell process is connected to nginx over a pipe or
socket. In which case, nginx can use flow control to block the sending
side of the pipe and the Haskell code will backup on that.

If many connections need to be multiplexed over the same
flow-controlled entity (i.e. a pipe), without head-of-line blocking
then you can just suspend the current thread using an MVar or the STM
objects.

Alternatively, with Network.MiniHTTP the problem is turned inside out.
Request handlers give return a Source object, which can be asked to
generate more data on request. This would be similar to generators in
Python.


AGL

-- 
Adam Langley [EMAIL PROTECTED] http://www.imperialviolet.org
___
Haskell-Cafe mailing list
Haskell-Cafe@haskell.org
http://www.haskell.org/mailman/listinfo/haskell-cafe


Re: [Haskell-cafe] [GSoC] X Haskell bindings

2008-03-31 Thread Adam Langley
2008/3/31 Roman Cheplyaka <[EMAIL PROTECTED]>:
>  Especially I need help with "Milestones" section. I'm not quite sure how
>  it should look like for such project.

This would seem to be a commendable SoC project. I'm not sure that the
milestones are all that important - no plan ever survives contact with
reality. However, I'd suggest that these would be important aspects:

* A working parser for the XCB (this involves choosing an XML parser
and getting useful data structures from it)
* A working Haskell code generator which outputs bindings for a
minimal subset of the functions

(the last two need not be in Haskell, although I suspect that it would
be a good idea for them to be so)

* The first, minimal, Haskell X client using XCB
* Having sensible bindings for 25%/50%/75% and 100% of the XCB interface.



AGL

-- 
Adam Langley [EMAIL PROTECTED] http://www.imperialviolet.org
___
Haskell-Cafe mailing list
Haskell-Cafe@haskell.org
http://www.haskell.org/mailman/listinfo/haskell-cafe


Re: [Haskell-cafe] Re: HTTP client libraries

2008-03-29 Thread Adam Langley
On Fri, Mar 28, 2008 at 6:40 AM, John Goerzen <[EMAIL PROTECTED]> wrote:
> Your Source type is an interesting approach, too.

I'm still not sure if it's actually a good idea.

But if you do have any requests, I'm very open to them. I think that a
fetchBasic which walks redirects would be a good start but, other than
that, I'm rather distracted with elliptic curve groups at the moment.

AGL

-- 
Adam Langley [EMAIL PROTECTED] http://www.imperialviolet.org
___
Haskell-Cafe mailing list
Haskell-Cafe@haskell.org
http://www.haskell.org/mailman/listinfo/haskell-cafe


Re: [Haskell-cafe] HTTP client libraries

2008-03-27 Thread Adam Langley
On Thu, Mar 27, 2008 at 12:08 PM, John Goerzen <[EMAIL PROTECTED]> wrote:
>  * network-minihttp
>
>   Doesn't appear to actually be very useful as a client.
>
>  Also, as far as I have been able to deduce, none of these have
>  built-in support for https (SSL) URLs.

The client doesn't do a lot, but I don't know what you would want from
a client. Email me a list of use cases and ;)

It does support HTTPS, however. See examples/webcat.hs


AGL

-- 
Adam Langley [EMAIL PROTECTED] http://www.imperialviolet.org
___
Haskell-Cafe mailing list
Haskell-Cafe@haskell.org
http://www.haskell.org/mailman/listinfo/haskell-cafe


Re: [Haskell-cafe] AMQP framing layer: design help requested.

2008-03-21 Thread Adam Langley
On Fri, Mar 21, 2008 at 6:41 PM, Adam Langley <[EMAIL PROTECTED]> wrote:
>  Firstly, I would use the Get and Put monads directly, rather than
>  making them instances of Binary.

Also, while I'm at it - I believe that AMQP messages and small and
delineated. In which case, the Data.Binary.Strict.Get monad from the
binary-strict package might be a better match (shameless self-plug)


AGL

-- 
Adam Langley [EMAIL PROTECTED] http://www.imperialviolet.org
___
Haskell-Cafe mailing list
Haskell-Cafe@haskell.org
http://www.haskell.org/mailman/listinfo/haskell-cafe


Re: [Haskell-cafe] AMQP framing layer: design help requested.

2008-03-21 Thread Adam Langley
On Fri, Mar 21, 2008 at 11:02 AM, Paul Johnson <[EMAIL PROTECTED]> wrote:
>  This effectively duplicates the AmqpVariant solution.  It would work,
>  but it feels horrible.  I get the feeling that there ought to be a
>  better way, possibly using classes or existential types.  Any suggestions?

Firstly, I would use the Get and Put monads directly, rather than
making them instances of Binary. I think, if I were doing it, I would
start off with something like this:

class AMQPValue a where
  putAMQP :: a -> Put

instance AMQPValue Word8 where
  putAMQP v = putWord8 1 >> putWord8 v
... (many instances)

instance (AMQPValue a) => AMQPValue [a] where
 putAMQP vs = putWord8 (length vs) >> mapM_ putAMQP vs

You probably want a getAMQP member of the AMQPValue class too.

Also
>   getter <- fmap (amqpGetTable !)  getWord8
>   getter

is just

> fmap (amqpGetTable !)  getWord8

Hope that helps some


AGL



-- 
Adam Langley [EMAIL PROTECTED] http://www.imperialviolet.org
___
Haskell-Cafe mailing list
Haskell-Cafe@haskell.org
http://www.haskell.org/mailman/listinfo/haskell-cafe


[Haskell-cafe] ANN: Network.MiniHTTP 0.2

2008-03-20 Thread Adam Langley
Since many people are interested in a ByteString HTTP library, here's
network-minihttp 0.2 [1].
  * Has code for both clients and servers
  * HTTP and HTTPS support (the latter is a little shaky, but mostly works)
  * DNS lookups (in the client code) don't block the whole process

The Haddock documentation is at:
http://darcs.imperialviolet.org/minihttp-docs/index.html

Half the interfaces are ill-thought out, and the rest are just as bad.
However, if you send me your ideas for how it should look I'll
certainly consider them.

The major omission at the moment are defensive timeouts for talking to
the network. However, these shouldn't be too hard.

[1] 
http://hackage.haskell.org/cgi-bin/hackage-scripts/package/network-minihttp-0.2

-- 
Adam Langley [EMAIL PROTECTED] http://www.imperialviolet.org
___
Haskell-Cafe mailing list
Haskell-Cafe@haskell.org
http://www.haskell.org/mailman/listinfo/haskell-cafe


Re: [Haskell-cafe] Satnam Singh of Microsoft Research

2008-03-18 Thread Adam Langley
2008/3/18 Galchin Vasili <[EMAIL PROTECTED]>:
> Recently Satnam Singh of Microsoft Research gave a talk at Google about
> concurrency, Haskell STM, etc. Was there a transcript of this talk?

Do you have the exact date of this talk? I can't see that anyone
called Satnam has given a talk at Google.

(It might have been very informal, in which case there wouldn't be a
record of it, and nor would there be a video)


Cheers,

AGL

-- 
Adam Langley [EMAIL PROTECTED] http://www.imperialviolet.org
___
Haskell-Cafe mailing list
Haskell-Cafe@haskell.org
http://www.haskell.org/mailman/listinfo/haskell-cafe


Re: [Haskell-cafe] Re: all threads are blocked by recvFrom

2008-03-18 Thread Adam Langley
On Tue, Mar 18, 2008 at 3:41 AM, Vitaliy Akimov
<[EMAIL PROTECTED]> wrote:
>  Unfortunally this way seems to be wrong. Error codes for winsockets
>  and BSD-sockets are different.

Hmm, "Networking broken on Windows" would seem to be a pretty big
issue. One of the Windows peeps like to speak up here?


AGL

-- 
Adam Langley [EMAIL PROTECTED] http://www.imperialviolet.org
___
Haskell-Cafe mailing list
Haskell-Cafe@haskell.org
http://www.haskell.org/mailman/listinfo/haskell-cafe


Re: [Haskell-cafe] Re: all threads are blocked by recvFrom

2008-03-17 Thread Adam Langley
On Mon, Mar 17, 2008 at 2:29 AM, Vitaliy Akimov
<[EMAIL PROTECTED]> wrote:
> Hi Adam, sorry for late answer. Here is my example [1], but yours
>  doesn't work on my PC too. And it's strange it works on yours.

Are you running Windows? Because you're hpaste example works,
nonblocking for me too.

>  And after changing in the network package FFI declaration for
>  c_recvfrom from unsafe to safe both examples start working.

The important point here is that the recvFrom calls in
Network.Socket[1] don't block. They are non-blocking calls so,
although other threads may be suspended while in the FFI call, the FFI
call returns -EAGAIN if it would block and blocking is passed to the
RTS to handle. While sleeping in the RTS (with threadWaitRead etc),
other Haskell threads can read. See the code in [1] (esp
throwErrnoIfMinus1Retry_repeatOnBlock)

So, the only reason that other threads would be blocked is if the
socket wasn't marked as non-blocking. Here's a snippet of strace
output from compiling your hpaste example:

socket(PF_INET, SOCK_DGRAM, IPPROTO_IP) = 5
fcntl64(5, F_GETFL) = 0x2 (flags O_RDWR)
fcntl64(5, F_SETFL, O_RDWR|O_NONBLOCK)  = 0

You can see that the socket is set to non-blocking immediately.

[1] 
http://haskell.org/ghc/docs/latest/html/libraries/network/src/Network-Socket.html#recvFrom

-- 
Adam Langley [EMAIL PROTECTED] http://www.imperialviolet.org
___
Haskell-Cafe mailing list
Haskell-Cafe@haskell.org
http://www.haskell.org/mailman/listinfo/haskell-cafe


Re: [Haskell-cafe] GHC threaded RTS will call the finaliser of a ForeignPtr while references still exist

2008-03-16 Thread Adam Langley
On Sun, Mar 16, 2008 at 8:45 PM, Don Stewart <[EMAIL PROTECTED]> wrote:
>  Create a ticket on the bug tracker, and I'm sure Ian or Simon M will have a 
> look at it.
>
>http://hackage.haskell.org/trac/ghc/newticket?type=bug

Done, thanks:

http://hackage.haskell.org/trac/ghc/ticket/2161



AGL

-- 
Adam Langley [EMAIL PROTECTED] http://www.imperialviolet.org
___
Haskell-Cafe mailing list
Haskell-Cafe@haskell.org
http://www.haskell.org/mailman/listinfo/haskell-cafe


[Haskell-cafe] Re: GHC threaded RTS will call the finaliser of a ForeignPtr while references still exist

2008-03-16 Thread Adam Langley
On Sun, Mar 16, 2008 at 8:40 PM, Adam Langley <[EMAIL PROTECTED]> wrote:
> I believe that I've managed to distill a crash which has been driving
>  me crazy for a few days into a short enough test case (22 lines) that
>  it might be useful.

Cale made a suggestion which shortens it:
  fptest.hs: http://hpaste.org/6423


-- 
Adam Langley [EMAIL PROTECTED] http://www.imperialviolet.org
___
Haskell-Cafe mailing list
Haskell-Cafe@haskell.org
http://www.haskell.org/mailman/listinfo/haskell-cafe


[Haskell-cafe] GHC threaded RTS will call the finaliser of a ForeignPtr while references still exist

2008-03-16 Thread Adam Langley
I believe that I've managed to distill a crash which has been driving
me crazy for a few days into a short enough test case (22 lines) that
it might be useful.

In short: the threaded RTS will call the finialiser of a ForeignPtr
while an exception handler still holds a reference to it:


% ghc -make fptest.hs cbits.c -threaded
[1 of 1] Compiling Main ( fptest.hs, fptest.o )
Linking fptest ...
% ./fptest
New object getting created: 66f010
Finaliser getting called for 66f010
Use called for 66f010


cbits.c: http://hpaste.org/6420
fptest.hs: http://hpaste.org/6421

I'm hoping that this is useful to someone who knows the RTS.

Cheers,


-- 
Adam Langley [EMAIL PROTECTED] http://www.imperialviolet.org
___
Haskell-Cafe mailing list
Haskell-Cafe@haskell.org
http://www.haskell.org/mailman/listinfo/haskell-cafe


Re: [Haskell-cafe] Re: all threads are blocked by recvFrom

2008-03-14 Thread Adam Langley
On Fri, Mar 14, 2008 at 10:13 AM, Adam Langley <[EMAIL PROTECTED]> wrote:
>  See [1] for an example which works for me.

(If you're on Windows, you probably need to wrap main in withSocketsDo)


AGL


-- 
Adam Langley [EMAIL PROTECTED] http://www.imperialviolet.org
___
Haskell-Cafe mailing list
Haskell-Cafe@haskell.org
http://www.haskell.org/mailman/listinfo/haskell-cafe


Re: [Haskell-cafe] Re: all threads are blocked by recvFrom

2008-03-14 Thread Adam Langley
On Fri, Mar 14, 2008 at 8:51 AM, Vitaliy Akimov
<[EMAIL PROTECTED]> wrote:
> > I assume that you're binding the libc function directly here:
>
>  I'm using Network.Socket. Sory if it's not clear from my previous posts.

Then everything should Just Work(tm). You might need to paste in code
in order to figure out why this wouldn't be so.

See [1] for an example which works for me. It starts a thread which
prints "working..." once a second and, in another thread, listens for
UDP packets on port 4112. I can use `nc -u 127.0.0.1 4112` to get
this:
"working..."
"working..."
("testing\n",8,127.0.0.1:36179)
"working..."
"working..."
("testing two\n",12,127.0.0.1:36179)
"working..."



[1] http://hpaste.org/6362

-- 
Adam Langley [EMAIL PROTECTED] http://www.imperialviolet.org
___
Haskell-Cafe mailing list
Haskell-Cafe@haskell.org
http://www.haskell.org/mailman/listinfo/haskell-cafe


Re: [Haskell-cafe] Re: all threads are blocked by recvFrom

2008-03-14 Thread Adam Langley
On Fri, Mar 14, 2008 at 7:43 AM, Vitaliy Akimov
<[EMAIL PROTECTED]> wrote:
> Rebuilding of the network package with changed safety helped but I
>  don't think this is the solution. BTW accept is declared as safe. What
>  is the reason of declaring recvFrom as unsafe?  I think this breaks
>  highly required feature. Apparently it's impossible to make concurrent
>  server for non connection based protocols.

I assume that you're binding the libc function directly here:

In that case, you need to have the RTS manage sleeping your thread for
you. You should make the socket non-blocking and handle the EAGAIN and
EWOULDBLOCK cases by calling threadWaitRead[1] to block the current
Haskell thread only, until the fd is readable. Note that
threadWaitRead is GHC only.

If you download the source to network-bytestring[2], you can see a
very similar pattern in the code for send and recv in there.

Alternatively, you can use the functions in Network.Socket, which
should work fine.

[1] 
http://haskell.org/ghc/docs/latest/html/libraries/base/Control-Concurrent.html#v%3AthreadWaitRead
[2] 
http://hackage.haskell.org/cgi-bin/hackage-scripts/package/network-bytestring-0.1.1.2


-- 
Adam Langley [EMAIL PROTECTED] http://www.imperialviolet.org
___
Haskell-Cafe mailing list
Haskell-Cafe@haskell.org
http://www.haskell.org/mailman/listinfo/haskell-cafe


Re: [Haskell-cafe] Problem making a program in ghc

2008-03-13 Thread Adam Langley
2008/3/13 Maverick <[EMAIL PROTECTED]>:
> I have writed an application that parses xml files and returns a result
> (using HaXml), this is published as a service, I implemented a server
> (withSocketsDo), the server listen the port 5760 in a forever fashion, when
> a message arrives, then the program creates a new thread (forkIO) for attend
> the request. This works fine when I run it from ghci, when I make the
> program and run the binary (in Windows), the requests arrive to the program
> but I think that the response can´t be sent because the front end can´t
> receive the response (I am calling the service from a .Net web application),
> I have a log that confirms that the response arrives correctly.

I hate to see any requests for help go unanswered here, but this one
might be tough. I think you need to give some more information,
otherwise the suggestions are going to be very general. Can you put
the Haskell source code on a website somewhere and link to it. Since
it's a network service, an example request and reply might be good to
include.

In general, you should check that you are correctly flushing your
connection. If you are using Handles to interface to the network, they
can buffer the response. hFlush[1] may need to be called when you have
finished generating it.

[1] 
http://haskell.org/ghc/docs/latest/html/libraries/base/System-IO.html#v%3AhFlush

AGL

-- 
Adam Langley [EMAIL PROTECTED] http://www.imperialviolet.org
___
Haskell-Cafe mailing list
Haskell-Cafe@haskell.org
http://www.haskell.org/mailman/listinfo/haskell-cafe


Re: [Haskell-cafe] Starting Haskell with a web application

2008-03-12 Thread Adam Langley
On Wed, Mar 12, 2008 at 3:49 PM, Adam Langley <[EMAIL PROTECTED]> wrote:
>  The OpenID example is running in EC2[4] at the moment if anyone wants to 
> play.

Well, thanks to all the people who hit it, there's nothing like users
to find the stupid bugs ;)

  * Caching was wrong on the front page, so it would act funky if you
were behind a proxy
  * Serving from the filesystem (i.e. the CSS) was broken because I
missed an action in that code

All fixed...

AGL

-- 
Adam Langley [EMAIL PROTECTED] http://www.imperialviolet.org
___
Haskell-Cafe mailing list
Haskell-Cafe@haskell.org
http://www.haskell.org/mailman/listinfo/haskell-cafe


Re: [Haskell-cafe] Starting Haskell with a web application

2008-03-12 Thread Adam Langley
On Wed, Mar 5, 2008 at 12:25 PM, Adam Langley <[EMAIL PROTECTED]> wrote:
>  I'm
>  working towards, is an OpenID consumer. Once I have that working, I'll
>  do a second release. It's not that far off, it's just a question of
>  time.

The darcs release of minihttp[1] can now do this. It's not a Hackage
release because it needs a darcs version of HsOpenSSL[2] and
network-connection[3].

The OpenID example is running in EC2[4] at the moment if anyone wants to play.



[1] http://darcs.imperialviolet.org/network-minihttp
[2] http://darcs.imperialviolet.org/HsOpenSSL
[3] http://darcs.imperialviolet.org/network-connection
[4] http://ec2-67-202-22-226.compute-1.amazonaws.com:4112/


AGL

-- 
Adam Langley [EMAIL PROTECTED] http://www.imperialviolet.org
___
Haskell-Cafe mailing list
Haskell-Cafe@haskell.org
http://www.haskell.org/mailman/listinfo/haskell-cafe


Re: [Haskell-cafe] Re: Constructing Data.Map from ByteString

2008-03-12 Thread Adam Langley
On Tue, Mar 11, 2008 at 4:13 AM, Dave Tapley <[EMAIL PROTECTED]> wrote:
>  I've upgraded to bytestring-0.9.0.5 from Darcs, no improvement.
>  Also this morning I tried using Data.HashMap with Bytestring's readInt
>  and HashMap's hashInt.. The result was a Stack space overflow :(

Map is probably a better choice than HashMap. Exactly how large is the
input? Like I said, I got about 2x bloat (measured by RSS), maybe you
can send a fuller example code?


AGL

-- 
Adam Langley [EMAIL PROTECTED] http://www.imperialviolet.org
___
Haskell-Cafe mailing list
Haskell-Cafe@haskell.org
http://www.haskell.org/mailman/listinfo/haskell-cafe


Re: [Haskell-cafe] Constructing Data.Map from ByteString

2008-03-12 Thread Adam Langley
On Mon, Mar 10, 2008 at 4:33 PM, Dave Tapley <[EMAIL PROTECTED]> wrote:
>  Concerning this minimal example:
>  http://hpaste.org/6268

Change the Data.ByteString.Char8 to Data.ByteString.Lazy.Char8, also
the takeWhile, dropWhile pair is better expressed using span.

With those two changes, and a 150MB test file, a compiled binary is
using about 220MB of resident space. That seems pretty reasonable to
me.

As a purely subjective atheistic point: importing some modules
qualified will help your sanity. Otherwise you have lots of common
function names (readFile etc) and you have to try and figure out which
module they are going to resolve into all the time!


Cheers,


AGL

-- 
Adam Langley [EMAIL PROTECTED] http://www.imperialviolet.org
___
Haskell-Cafe mailing list
Haskell-Cafe@haskell.org
http://www.haskell.org/mailman/listinfo/haskell-cafe


Re: [Haskell-cafe] STM example code

2008-03-09 Thread Adam Langley
2008/3/8 Galchin Vasili <[EMAIL PROTECTED]>:
>   I am playing around with the STM API. I would like to see examples of
> STM other than the Santa.hs as I am having problems with STM vs IO.


The STM papers were the best documentation for me:

http://research.microsoft.com/~simonpj/papers/stm/index.htm


-- 
Adam Langley [EMAIL PROTECTED] http://www.imperialviolet.org
___
Haskell-Cafe mailing list
Haskell-Cafe@haskell.org
http://www.haskell.org/mailman/listinfo/haskell-cafe


[Haskell-cafe] Re: Difficulties implementing an incremental parser using Oleg-style left fold enumerator

2008-03-08 Thread Adam Langley
On Sat, Mar 8, 2008 at 12:56 AM, Johan Tibell <[EMAIL PROTECTED]> wrote:
>  The problem -- maybe there are others too -- is that when a parser such as
>
>  many (byte 65)
>
>  is run it will always return a 'Partial' result waiting for more input
>  even though the enumerator is exhausted. In other words, there's no
>  way to detect end of input.

It appears that you need some way to distinguish the end of input
from, "that's all I have for now". You could use an empty Bytestring
in S if you were careful that you maintained that, in normal
processing, such a state doesn't arise. Otherwise, have a Maybe in
your state and set it to Nothing when the input is exhausted. Then
have combinators, like many, handle the EOF case sensibly.


AGL


-- 
Adam Langley [EMAIL PROTECTED] http://www.imperialviolet.org
___
Haskell-Cafe mailing list
Haskell-Cafe@haskell.org
http://www.haskell.org/mailman/listinfo/haskell-cafe


Re: [Haskell-cafe] Re: Network.Socket Woes

2008-03-07 Thread Adam Langley
On Fri, Mar 7, 2008 at 11:04 AM, Benedikt Schmidt <[EMAIL PROTECTED]> wrote:
>  The PortNumber type is an instance of Num, so it works  if you do not
>  use the PortNum constructor.
>
>  Prelude Network.Socket> case 10162 :: PortNumber of PortNum p -> p
>  45607

And there was the trick that I've been missing all this time ;) I'll
probably submit a documentation patch for this since I'm a Bear of
Little Brain and this wasn't obvious to me.

-- 
Adam Langley [EMAIL PROTECTED] http://www.imperialviolet.org
___
Haskell-Cafe mailing list
Haskell-Cafe@haskell.org
http://www.haskell.org/mailman/listinfo/haskell-cafe


Re: [Haskell-cafe] Network.Socket Woes

2008-03-07 Thread Adam Langley
On Fri, Mar 7, 2008 at 8:10 AM, Scott Bell <[EMAIL PROTECTED]> wrote:
>  my test program. The Haskell version, however, does not
>  return from recvFrom. I've also tried wrapping this in a
>  withSocketsDo, with no effect.

So this is a long standing, ah, issue with the Network modules.

Try sending a UDP packet to port 45607 and you'll find that the
Haskell code gets it.

hex(45607) = 0xb227
0x27b2 = 10162

In short, PortNum doesn't do the endian conversion for you. And I
don't know a good way to figure out the endianness of the underlying
system from Haskell I'm afraid. I usually end up FFIing out to htons
or just assuming that the system is little-endian.

We should really fix this unless there's some trick that I've been
missing all this time.


AGL

-- 
Adam Langley [EMAIL PROTECTED] http://www.imperialviolet.org
___
Haskell-Cafe mailing list
Haskell-Cafe@haskell.org
http://www.haskell.org/mailman/listinfo/haskell-cafe


Re: [Haskell-cafe] Starting Haskell with a web application

2008-03-06 Thread Adam Langley
On Thu, Mar 6, 2008 at 3:08 AM, Bayley, Alistair
<[EMAIL PROTECTED]> wrote:
>  Do you (both) have repos that I could download from? I quite interested
>  in both projects, esp. the WSGI clone.

There was a Hackage release of network-minihttp[1], which I think
would serve files from the filesystem quite happily. The darcs repo is
just a mess at the moment. (darcs.imperialviolet.org/network-minihttp)

[1] 
http://hackage.haskell.org/cgi-bin/hackage-scripts/package/network-minihttp-0.1

AGL

-- 
Adam Langley [EMAIL PROTECTED] http://www.imperialviolet.org
___
Haskell-Cafe mailing list
Haskell-Cafe@haskell.org
http://www.haskell.org/mailman/listinfo/haskell-cafe


Re: [Haskell-cafe] Starting Haskell with a web application

2008-03-05 Thread Adam Langley
On Wed, Mar 5, 2008 at 11:07 AM, Bryan O'Sullivan <[EMAIL PROTECTED]> wrote:
>  Indeed.  In addition to the code you mention, people like Adam Langley
>  and Johan Tibbell are taking on corners of the web app problem space in
>  a more modern context.

I should probably speak up then ;)

I'm (slowly) writing Network.MiniHTTP. What it does have is a modern,
ByteString based HTTP parser and serialiser (in decent shape), SSL
support (only in darcs at the moment) and my test case, that I'm
working towards, is an OpenID consumer. Once I have that working, I'll
do a second release. It's not that far off, it's just a question of
time.


AGL

-- 
Adam Langley [EMAIL PROTECTED] http://www.imperialviolet.org
___
Haskell-Cafe mailing list
Haskell-Cafe@haskell.org
http://www.haskell.org/mailman/listinfo/haskell-cafe


Re: [Haskell-cafe] Connection helpers: for people interested in network code

2008-03-04 Thread Adam Langley
On Tue, Mar 4, 2008 at 7:31 AM, Bjorn Bringert <[EMAIL PROTECTED]> wrote:
>  you may want to have a look at the socket abstraction used in the HTTP 
> package:
>
>  
> http://hackage.haskell.org/packages/archive/HTTP/3001.0.4/doc/html/Network-Stream.html
>
>  It would be great to get HTTPS support going!

I should have mentioned that I had seen this in the original email. I
think the major problem with this interface was that it was written in
the time before ByteStrings. Now that we have ByteStrings I think that
it makes a lot of sense for networking to use them.

However, it shouldn't be too hard to wrap HsOpenSSL in this interface.
I might try this this week. Then HTTPS should Just Work (maybe ;)

AGL


-- 
Adam Langley [EMAIL PROTECTED] http://www.imperialviolet.org
___
Haskell-Cafe mailing list
Haskell-Cafe@haskell.org
http://www.haskell.org/mailman/listinfo/haskell-cafe


Re: [Haskell-cafe] .cabal problem

2008-03-01 Thread Adam Langley
2008/3/1 Galchin Vasili <[EMAIL PROTECTED]>:
> I am trying to install a package on my Linux Ubuntu machine. It chokes
> build-depends:  base, directory because directory dependency is
> unresolvable. Do I have to specify "extra-libs" so that correct library
> space is searched? Actually I tried this and it didn't help. ??

So, here's[1] the package "directory", however it ships with GHC, so
you very probably already have it installed. Try `ghc-pkg describe
directory` to check. Also, run `ghc --version` to see what version of
GHC you are running. If it's < 6.8, you may have problems because the
directory package didn't exist before then, I believe the same modules
were in the base package. In that case, the easy solution is probably
to upgrade GHC.


[1] http://hackage.haskell.org/cgi-bin/hackage-scripts/package/directory-1.0.0.0


AGL

-- 
Adam Langley [EMAIL PROTECTED] http://www.imperialviolet.org
___
Haskell-Cafe mailing list
Haskell-Cafe@haskell.org
http://www.haskell.org/mailman/listinfo/haskell-cafe


[Haskell-cafe] Connection helpers: for people interested in network code

2008-02-29 Thread Adam Langley
I generally find that I'm wrapping sockets in the same functions a lot
and now I'm looking writings code which works with both Sockets and
SSL connections. So I wrote a module, presumptuously called
Network.Connection, although I'm not actually going to try and take
that name (even in Hackage) unless I get a general agreement that this
is a good thing.

So, any comments on the interface, similar things that I should look at etc?

http://www.imperialviolet.org/binary/network-connection/Network-Connection.html

I made the BaseConnection an ADT, rather than a class because I wanted
to avoid hitting the monomorphism restriction in code. That might have
been a mistake, I'm not sure yet.

If it doesn't excite anyone enough to reply, I'll change the name and
put it in Hackage, mostly as is. Then I'll tie HsOpenSSL into it so
that SSL connections work transparently.

Cheers,


AGL

-- 
Adam Langley [EMAIL PROTECTED] http://www.imperialviolet.org
___
Haskell-Cafe mailing list
Haskell-Cafe@haskell.org
http://www.haskell.org/mailman/listinfo/haskell-cafe


Re: [Haskell-cafe] parsing c

2008-02-24 Thread Adam Langley
On Sun, Feb 24, 2008 at 4:08 PM, Evan Martin <[EMAIL PROTECTED]> wrote:
>  So two questions: Is there an alternative C-parsing library?

You could use gcc-xml[1] with one of the various XML parsers.

[1] http://www.gccxml.org/HTML/Index.html

-- 
Adam Langley [EMAIL PROTECTED] http://www.imperialviolet.org
___
Haskell-Cafe mailing list
Haskell-Cafe@haskell.org
http://www.haskell.org/mailman/listinfo/haskell-cafe


Re: [Haskell-cafe] A beginners question

2008-02-23 Thread Adam Langley
On Sat, Feb 23, 2008 at 8:00 AM, Harri Kiiskinen <[EMAIL PROTECTED]> wrote:
>  then why does
>
>  let i = fmap (^4) [1,2,3] in shows i " "
>
>  give
>
>  "[1,16,81] "

I'll probably mess this up somewhere, but if I do, reset assured that
someone else here will correct me ;)

fmap (^4) [1,2,3] == [1,16,81] so shows " " of that == "[1,16,81] "
(note the trailing space), obvious I hope.

However, [1,16,81] >>= \i -> shows i " " operates in the list monad. I
can't find the actual instance of Monad [] right now, but it will
apply 'shows i " "' to each element in the first list and concat the
results. Since strings are just [Char], concating them is string
concatenation. And so you have "1 16 81 " (again, note the trailing
space)


AGL

-- 
Adam Langley [EMAIL PROTECTED] http://www.imperialviolet.org
___
Haskell-Cafe mailing list
Haskell-Cafe@haskell.org
http://www.haskell.org/mailman/listinfo/haskell-cafe


[Haskell-cafe] ANN: ctemplate, SSL server support in HsOpenSSL, network-minihttp, binary parsing, network-dns

2008-02-14 Thread Adam Langley
Binary parsing:

binary-strict now has support for combinator parsing[1] with <|> and
friends. This works both for the strict Get and the IncrementalGet.
(The latter was more complex than I expected)

binary-strict also gained a very fast byte set module[2] (used for
scanning ranges of valid bytes) and a surprisingly useful hexDump[3]
function (because I kept having to write it in lots of places)

[1] 
http://hackage.haskell.org/packages/archive/binary-strict/0.3.0/doc/html/Data-Binary-Strict-Class.html
[2] 
http://hackage.haskell.org/packages/archive/binary-strict/0.3.0/doc/html/Data-Binary-Strict-ByteSet.html
[3] 
http://hackage.haskell.org/packages/archive/binary-strict/0.3.0/doc/html/Data-Binary-Strict-Util.html

DNS:

network-dns contains a DNS client library[4] in pure Haskell. The ADNS
library (for which a wrapper already exists) is GPL, not BSD. This
code is also halfway to being a DNS server should anyone so wish. This
is very similar to the DNS client library in libevent (because I wrote
them both).

HTTP:

ctemplate is a wrapping of the template library which Google uses for
most of their sites. There are several template systems already in
Hackage, but I like ctemplate because escaping is so easy and well
supported. Generally the power of a template system doesn't interest
me (in fact, it can be a bad thing), but stopping XSS attacks is a big
deal.

HsOpenSSL now has rudimentary support for writing SSL servers[5]
(clients coming soon). In terms of blocking IO, this /should/ work the
way you would hope (i.e. like a Haskell function w.r.t. forkIO).

network-minihttp is a small HTTP server. Currently it serves files
from the disk (with caching and range support), but it's most of the
way to being a HTTP client also. Don't do anything too serious with it
yet, however. It needs a few limits to stop DoS attackers from, for
example, sending an infinite header and using up all the memory :)

[4] 
http://hackage.haskell.org/packages/archive/network-dns/0.1.1/doc/html/Network-DNS-Client.html

-- 
Adam Langley [EMAIL PROTECTED]
http://www.imperialviolet.org 650-283-9641
___
Haskell-Cafe mailing list
Haskell-Cafe@haskell.org
http://www.haskell.org/mailman/listinfo/haskell-cafe


Re: [Haskell-cafe] getSymbolicLinkStatus completely broken on some 6.8.2 systems

2008-02-14 Thread Adam Langley
Filed as:
  http://hackage.haskell.org/trac/ghc/ticket/2096

-- 
Adam Langley [EMAIL PROTECTED]
http://www.imperialviolet.org 650-283-9641
___
Haskell-Cafe mailing list
Haskell-Cafe@haskell.org
http://www.haskell.org/mailman/listinfo/haskell-cafe


Re: [Haskell-cafe] getSymbolicLinkStatus completely broken on some 6.8.2 systems

2008-02-12 Thread Adam Langley
On Feb 12, 2008 11:04 PM, Adam Langley <[EMAIL PROTECTED]> wrote:
> structure filled in. HsUnix.h has a wrapper around lstat for exactly
> this reason, however ltrace shows it calling the wrong one.

So (finally!) the real issue:

hsc2hs has a C preprocessor prelude (utils/hsc2hs/template-hsc.h)
which includes some GHC header files. As a result, hsc processes files
in 64-bit mode:
% cpp -I../../includes -dM template-hsc.h | grep FILE_OFFSET_BITS
#define _FILE_OFFSET_BITS 64

However, when building HsUnix.c (or any c file with cabal), the
environment is different and it's built in 32-bit mode. Thus the lstat
wrapper called the 32-bit glibc function, but the structure sizes and
offsets in Files.hs (from Files.hsc) are all for 64-bit mode.

The fix is not quite obvious. Probably hsc2hs should try harder not to
change the environment so much. I'm not quite sure why the hsc2hs
template pulls in the HsFFI header - it doesn't seem to need it.

Although you can remove the #include and rebuild the unix package, I
can safely say that you'll be reinstalling ghc if you do. What might
work is changing hsc2hs and then rebuilding all of GHC. However,
that's an overnight build so I don't know yet.


AGL

-- 
Adam Langley  [EMAIL PROTECTED]
http://www.imperialviolet.org   650-283-9641
___
Haskell-Cafe mailing list
Haskell-Cafe@haskell.org
http://www.haskell.org/mailman/listinfo/haskell-cafe


Re: [Haskell-cafe] getSymbolicLinkStatus completely broken on some 6.8.2 systems

2008-02-12 Thread Adam Langley
On Feb 12, 2008 10:44 PM, Adam Langley <[EMAIL PROTECTED]> wrote:
> Currently I'm looking at hsc2hs for this bug. On a 32 bit box here:

hsc2hs is forgiven; if you build with #define _FILE_OFFSET_BITS 32
then the structure is 96 bytes and the 64-bit offset is, indeed, at
offset 88.

However, if you end up calling the wrong libc lstat (and it's tough to
tell, because both lstat and lstat64 in libc call the system call
lstat64, so strace can't tell you) you'll only get an 88 byte
structure filled in. HsUnix.h has a wrapper around lstat for exactly
this reason, however ltrace shows it calling the wrong one.


-- 
Adam Langley  [EMAIL PROTECTED]
http://www.imperialviolet.org   650-283-9641
___
Haskell-Cafe mailing list
Haskell-Cafe@haskell.org
http://www.haskell.org/mailman/listinfo/haskell-cafe


Re: [Haskell-cafe] getSymbolicLinkStatus completely broken on some 6.8.2 systems

2008-02-12 Thread Adam Langley
On Feb 12, 2008 8:38 PM, Jeremy Shaw <[EMAIL PROTECTED]> wrote:
> I get funny results when I call getSymbolicLinkStatus in ghc 6.8.2 on
> Ubuntu 7.10. This happens on several systems and has been confirmed by
> other people.

Currently I'm looking at hsc2hs for this bug. On a 32 bit box here:

int main() {
  struct stat st;
  printf("size of structure is %d\n", sizeof(st));
  printf("inode is at offset %d\n", __builtin_offsetof(struct stat, st_ino));
}

Gives:

size of structure is 88
inode is at offset 12

However, building Files.hsc gives:

fileID (FileStatus stat) =
  unsafePerformIO $ withForeignPtr stat $ ((\hsc_ptr -> peekByteOff hsc_ptr 88))
{-# LINE 315 "Files.hsc" #-}


e.g. it's reading random memory at the end of the structure. Looking
at the code from the documentation:

http://haskell.org/ghc/docs/latest/html/libraries/unix/src/System-Posix-Files.html#fileID

fileID (FileStatus stat) =
  unsafePerformIO $ withForeignPtr stat $ ((\hsc_ptr -> peekByteOff hsc_ptr 8))
{-# LINE 315 "System/Posix/Files.hsc" #-}

Seems like it might work, but that offset is too small. It appears
that it might be a 64-bit build (because the size of the stat
structure which when it gets malloced is 144), but a 64-bit build
should have larger offsets, not smaller.

You can't write your own fileID function, because FileStatus isn't
exported from Posix.Files. You can, however, write
getSymbolicLinkInode a c function (which does both the lstat and
extraction from the structure) and FFI it.


AGL


-- 
Adam Langley  [EMAIL PROTECTED]
http://www.imperialviolet.org   650-283-9641
___
Haskell-Cafe mailing list
Haskell-Cafe@haskell.org
http://www.haskell.org/mailman/listinfo/haskell-cafe


Re: [Haskell-cafe] example packages that use Storable in Hackage ??

2008-02-11 Thread Adam Langley
2008/2/11 Galchin Vasili <[EMAIL PROTECTED]>:
>  http://hackage.haskell.org/packages/archive/pkg-list.html .. what are
> some packages that use Storable?

binary and binary-strict at least.


AGL

-- 
Adam Langley  [EMAIL PROTECTED]
http://www.imperialviolet.org   650-283-9641
___
Haskell-Cafe mailing list
Haskell-Cafe@haskell.org
http://www.haskell.org/mailman/listinfo/haskell-cafe


Re: [Haskell-cafe] ANN: nano-hmac 0.2.0

2008-02-11 Thread Adam Langley
On Feb 11, 2008 12:54 AM, Hitesh Jasani <[EMAIL PROTECTED]> wrote:
> nano-hmac provides bindings to OpenSSL's HMAC interface.  With this release 
> the
> set of hashing functions supported is: MD5, SHA, SHA1, SHA224, SHA256, SHA384,
> SHA512.

Just a heads up; PHO has written nice bindings to much of OpenSSL:

http://hackage.haskell.org/cgi-bin/hackage-scripts/package/HsOpenSSL-0.3.1


AGL

-- 
Adam Langley  [EMAIL PROTECTED]
http://www.imperialviolet.org   650-283-9641
___
Haskell-Cafe mailing list
Haskell-Cafe@haskell.org
http://www.haskell.org/mailman/listinfo/haskell-cafe


Re: [Haskell-cafe] question concerning ANSI "void *"

2008-02-08 Thread Adam Langley
On Feb 8, 2008 10:57 AM, Galchin Vasili <[EMAIL PROTECTED]> wrote:
> basically I am trying to implement ioctl for the Posix library .. so a
> possible signtaure would be:
>
> fdIoctl :: Fd -> Int -> Ptr Word 8 -> IO( Ptr Word8) 

Ah, ok. You could cover many of the ioctls (the ones which only take a
single primitive) by using the Storable class and the types CInt etc.
Define the FFI call to ioctl with a type of CInt -> Int -> Ptr () ->
IO CInt and then

ioctlCInt :: CInt -> CInt -> CInt -> IO CInt
ioctlCInt fd call arg = do
  allocaBytes $ \ptr -> do
poke ptr arg
result <- ioctl fd call (cast ptr)
when (result < 0) $ fail "ioctl error"
peek ptr

(untested, might work ;)

... and likewise for the other C types. However, for those ioctls which take a
complex structure (e.g. many of the networking ones), you'll need to marshal
yourself:

data SomeIOCtlStruct = CInt CInt CInt

ioctlSomeIOCtlStruct :: CInt -> CInt -> SomeIOCtlStruct -> IO ()
ioctlSomeIOCtlStruct = do
  ...  (see the above linked to pointers to hsc2hs and c2hs about how to write
  this function)



AGL

--
Adam Langley  [EMAIL PROTECTED]
http://www.imperialviolet.org   650-283-9641
___
Haskell-Cafe mailing list
Haskell-Cafe@haskell.org
http://www.haskell.org/mailman/listinfo/haskell-cafe


Re: [Haskell-cafe] question concerning ANSI "void *"

2008-02-08 Thread Adam Langley
On Feb 8, 2008 9:13 AM, Galchin Vasili <[EMAIL PROTECTED]> wrote:
> Let's take a concrete but "made up" case .. suppose we want to call through
> to pthread_create and pass the (void *) argument to pthread_create which in
> turn gets interpreted by the pthread that is launched. How would one
> populate the C struct that is passed to the launched pthread keeping in mind
> that this C struct is variable in length? From the FFI how would one model
> this C struct?

It tough to be helpful with such a generic request. Here are some
options that you can consider:

1) Write a wrapper function in C which has a nicer FFI interface to
call from Haskell. Using cabal this is pretty painless and, if the
interface suits it, it probably the easiest solution.
2) Call pthread_create directly with the FFI. You can give the FFI
function a Haskell type with 'Ptr ()' or 'Ptr X', it doesn't really
matter. However the type system serves you best, do it that way. This
means that you need to populate the struct yourself in Haskell. The
issue with this is that the local system defines lots of things like
padding and alignment which mean that the layout of the structure in
memory is complex and platform specific. Tools like hsc2hs[1] or c2hs
will be very helpful here. Dealing with the variable length probably
isn't an issue. Usually variable length structures have a fixed header
and a variable tail, where the tail is an array of primitives. You can
malloc the correct sized region, use one of the previous tools to fill
in the fixed header and then use poke to complete the tail.

I might be able to be more helpful if you give the actual struct and
function prototype that you're trying to wrap.

Cheers



[1] http://therning.org/magnus/archives/tag/hsc2hs
[2] http://www.cse.unsw.edu.au/~chak/haskell/c2hs/

-- 
Adam Langley  [EMAIL PROTECTED]
http://www.imperialviolet.org   650-283-9641
___
Haskell-Cafe mailing list
Haskell-Cafe@haskell.org
http://www.haskell.org/mailman/listinfo/haskell-cafe


Re: [Haskell-cafe] question concerning ANSI "void *"

2008-02-07 Thread Adam Langley
2008/2/7 Galchin Vasili <[EMAIL PROTECTED]>:
>  Ok .. I am writing a Haskell function that will call down into the ANSI
> C library .. blah ::  -> Ptr Word8 ->  The underlying C function
> that "blah" is calling has a "void *" so I am using "Ptr Word  8" to model
> the "void *".

Depending on the context, "void *" is generally either taken as a Ptr
() (for an opaque pointer) or Ptr Word8 (for calls like memcpy).

> I propose to have the callers of function "blah" to populate a
> data structure something like "Ptr Buz" where "data Buz = { } and
> then do a "recast :: Ptr Word 8 -> Ptr Buz" when invoking function "blah".
> Does this seem reasonable? Is there a better way?

Generally, Ptr x is only used where x is either a shadow type or a
Bits type. Having a Ptr Baz where Baz is an ADT seems a little odd. If
you need to translate a structure from Haskell to C code, probably you
are better off having callers pass in a Baz then, internal to the
wrapping, fill out the C structure and call the FFI function with a
Ptr CBaz (where CBaz is a shadow type).


AGL

-- 
Adam Langley  [EMAIL PROTECTED]
http://www.imperialviolet.org   650-283-9641
___
Haskell-Cafe mailing list
Haskell-Cafe@haskell.org
http://www.haskell.org/mailman/listinfo/haskell-cafe


Re: [Haskell-cafe] reading from the cdrom drive on Red Hat

2008-02-01 Thread Adam Langley
On Feb 1, 2008 1:42 PM, Bryan O'Sullivan <[EMAIL PROTECTED]> wrote:
> No, it's the Haskell runtime turning a -1 return from read into an
> exception.  You need to call hIsEOF to check whether you've hit EOF,
> then break out of the loop.

(assuming you meant 0, not -1)

My bad, I thought that the Posix.IO stuff was a closer wrapping than
that. It does, indeed, throw an exception on 0. How unfortunate.


AGL


-- 
Adam Langley  [EMAIL PROTECTED]
http://www.imperialviolet.org   650-283-9641
___
Haskell-Cafe mailing list
Haskell-Cafe@haskell.org
http://www.haskell.org/mailman/listinfo/haskell-cafe


Re: [Haskell-cafe] reading from the cdrom drive on Red Hat

2008-02-01 Thread Adam Langley
2008/2/1 Galchin Vasili <[EMAIL PROTECTED]>:
>   I wrote a simple program to read the contents of a cdrom:

(Note that this is a terribly inefficient way of reading large amounts
of binary data. Of course, if this is just meant as an example, that's
fine. Otherwise, see the struff about ByteStrings at
http://www.haskell.org/haskellwiki/DealingWithBinaryData)

The error you are seeing comes from the operating system. You could
run the resulting Haskell program under strace to see the exact error,
but what's happening is that the kernel is getting an error from the
CD drive itself. This is pretty much unique to CDs I guess. If you
were reading a hard drive, the kernel knows exactly how large the disk
is (see /proc/partitions) and would return EOF (zero bytes).

As an aside, hitting EOF in your code would result in an infinite
loop, since you don't handle actualByteCount == 0.

You can catch and handle the resulting exception with the functions in:
  http://haskell.org/ghc/docs/latest/html/libraries/base/Control-Exception.html



AGL


-- 
Adam Langley  [EMAIL PROTECTED]
http://www.imperialviolet.org   650-283-9641
___
Haskell-Cafe mailing list
Haskell-Cafe@haskell.org
http://www.haskell.org/mailman/listinfo/haskell-cafe


Re: [Haskell-cafe] Simple network client

2008-01-31 Thread Adam Langley
On Jan 31, 2008 5:07 AM, Gary Bickford <[EMAIL PROTECTED]> wrote:
> Wasn't there a Linux file system (possibly a FUSE user-space one) that
> worked on writable CDs?  IIRC it worked by marking the previous copy of
> the file as erased, and writing a new copy.

Probably you're thinking of layering a unionfs over the top. It works
well in my experience.



AGL

-- 
Adam Langley  [EMAIL PROTECTED]
http://www.imperialviolet.org   650-283-9641
___
Haskell-Cafe mailing list
Haskell-Cafe@haskell.org
http://www.haskell.org/mailman/listinfo/haskell-cafe


Re: [Haskell-cafe] Simple network client

2008-01-30 Thread Adam Langley
On Jan 30, 2008 1:07 PM, Adam Langley <[EMAIL PROTECTED]> wrote:
> So, if I don't hear otherwise soon, I'll probably push a new version
> of binary-strict later on today with the interface above.

It's in the darcs now, http://darcs.imperialviolet.org/binary-strict


AGL

-- 
Adam Langley  [EMAIL PROTECTED]
http://www.imperialviolet.org   650-283-9641
___
Haskell-Cafe mailing list
Haskell-Cafe@haskell.org
http://www.haskell.org/mailman/listinfo/haskell-cafe


Re: [Haskell-cafe] Simple network client

2008-01-30 Thread Adam Langley
On Jan 30, 2008 12:04 PM, Bryan O'Sullivan <[EMAIL PROTECTED]> wrote:
> Adam Langley wrote:
> I'd have expected it to look more like this:
>
> data Result a = Failed String
>   | Finished B.ByteString a
>   | Partial (B.ByteString -> Result a)
>
> (The change here is from a list to a singleton.)  I don't think I care
> much for the extra boxing and reversing this involves.

Well, since you're probably the /only/ user you can pretty much say
how it works ;) The original interface was designed so that you can
yield a list of results as you parse. I guess that, since you get the
remainder anyway, you can chain these together if you like anyway.

So, if I don't hear otherwise soon, I'll probably push a new version
of binary-strict later on today with the interface above.



AGL


-- 
Adam Langley  [EMAIL PROTECTED]
http://www.imperialviolet.org   650-283-9641
___
Haskell-Cafe mailing list
Haskell-Cafe@haskell.org
http://www.haskell.org/mailman/listinfo/haskell-cafe


Re: [Haskell-cafe] ffi and pass by value structs

2008-01-30 Thread Adam Langley
On Jan 30, 2008 5:16 AM, Federico Squartini
<[EMAIL PROTECTED]> wrote:
> foreign import ccall "print_couple_ptr" printCouple::Ptr Couple->IO Int
>
> But not the first, as a struct it's not an atomic type.
> So how does one solve this issue?

(This applies to x86-64 only!)

In the case of your struct couple, I believe that it's small enough
that it will be passed in registers:

http://www.x86-64.org/documentation/abi.pdf
> 1. If the size of an object is larger than two eightbytes, or it contains 
> unaligned
>   fields, it has class MEMORY.

However, it's larger than this the C compiler will actually call
memcpy to make a copy before calling the function with a pointer to
the copy on the parent's stack.

With the Haskell FFI, the portable solution is probably to write a
very simple wrapper .c file which takes a pointer and calls the real
function with the contents of that pointer. Then you just call your
wrapper function. You can make these functions by the dozen with the
preprocessor.


AGL

-- 
Adam Langley  [EMAIL PROTECTED]
http://www.imperialviolet.org   650-283-9641
___
Haskell-Cafe mailing list
Haskell-Cafe@haskell.org
http://www.haskell.org/mailman/listinfo/haskell-cafe


Re: [Haskell-cafe] Simple network client

2008-01-30 Thread Adam Langley
On Jan 30, 2008 4:32 AM, Jules Bean <[EMAIL PROTECTED]> wrote:
> The third, but more sophisticated answer is to use non-blocking IO, read
>   'only what is available', decide if it's enough to process, if not
> store it in some local buffer until next time. This is much more work
> and easy to implement bugs in, but you need it for true streaming
> protocols.  In that case hGetBufNonBlocking is your friend.

If using bytestrings from the network, the network-bytestring package
is what you need.

If you're parsing small lumps from the network, Data.Binary's failures
will probably make life harder than it should be for you since they
can only be caught in IO. See
http://www.haskell.org/haskellwiki/DealingWithBinaryData about a
strict Get which might work for you.

Also, if you want the above approach (read a bit, see if it's enough),
see IncrementalGet in the binary-strict package which is a Get with a
continuation monad that stops when it runs out of bytes and returns a
continuation that you can give more data to in the future.


AGL

-- 
Adam Langley  [EMAIL PROTECTED]
http://www.imperialviolet.org   650-283-9641
___
Haskell-Cafe mailing list
Haskell-Cafe@haskell.org
http://www.haskell.org/mailman/listinfo/haskell-cafe


Re: [Haskell-cafe] Performance Issues

2008-01-29 Thread Adam Langley
> This computes 100!. This version takes 8m29.189s to execute.
> Replace foldr1 with foldr and that goes down to 7m4.315s. Replace
> product' with the Prelude product and it takes only 6m17.685s. Why is
> that so? I'm using ghc 6.8.1 on Mac OS X.

I'm guessing that the speedup with the Prelude product is because it's
a left fold. foldr goes to the end of the list and starts applying
backwards, up the thunks. A left fold just keeps an accumulator as it
walks down.

The foldr1 to foldr speedup might be because foldr1 has three cases,
to foldr's two:
http://haskell.org/ghc/docs/latest/html/libraries/base/src/GHC-List.html#foldr1

AGL

-- 
Adam Langley  [EMAIL PROTECTED]
http://www.imperialviolet.org   650-283-9641
___
Haskell-Cafe mailing list
Haskell-Cafe@haskell.org
http://www.haskell.org/mailman/listinfo/haskell-cafe


Re: [Haskell-cafe] Simple network client

2008-01-29 Thread Adam Langley
On Jan 29, 2008 6:28 AM, Timo B. Hübel <[EMAIL PROTECTED]> wrote:
> Hm, unfortunately not for me (Linux, GHC 6.8.2) ...

That's odd, because it works for me on the exact same setup.

There was a similar bug in lazy bytestring's hGetContents a while back
which involve it waiting for a whole chunk and not returning short
reads, but from watching the strace of this code, GHC is reading
byte-by-byte (which is actually pretty dumb, but functions).

Can you compile both with:
  % ghc --make file.hs

And run them with:
  % strace -o /tmp/trace ./file

(obviously, you're running strace twice, with different binaries and
output files)

and send me the resulting traces? (They'll be quite big, so I don't
know if you want to spam that whole list with them)

Cheers


AGL

-- 
Adam Langley  [EMAIL PROTECTED]
http://www.imperialviolet.org   650-283-9641
___
Haskell-Cafe mailing list
Haskell-Cafe@haskell.org
http://www.haskell.org/mailman/listinfo/haskell-cafe


[Haskell-cafe] ANN: RPCA and fec

2008-01-27 Thread Adam Langley
Doubling up an announcement email again to reduce the traffic on the
list. First:

RPCA is an RPC system. This first release is pretty lacking, simple
client-server interactions work but there's no lame-duck handling,
load balancing, security etc. RPCA uses codec-libevent's tagged data
structures for serialisation. Tagged structures allow you to grow you
RPCs incrementally, which is very important in large programs. Very
simple tests should that 1000's of RPCs a second isn't a problem.
There's also the start of a C library for the protocol, written on
libevent and glib.

To give a taste, here's what you need to write on the server side:

addNumbers :: TestRPC.Addnumbersrequest
   -> ((Maybe TestRPC.Addnumbersreply) -> STM ())
   -> IO ()
addNumbers (TestRPC.Addnumbersrequest
   { TestRPC.addnumbersrequest_a = a
   , TestRPC.addnumbersrequest_b = b }) cb = do
  atomically $ cb $ Just $ TestRPC.addnumbersreplyEmpty {
TestRPC.addnumbersreply_c = a + b }


and on the client side...

rpc chan "add" (TestRPC.Addnumbersrequest 400 300) 0.1

That's a synchronous call with a timeout. You can, of course, call
them asynchronously as well (and from multiple threads etc)

It's highly experimental and can be found in Hackage and at
http://darcs.imperialviolet.org/darcsweb.cgi?r=network-rpca;a=summary



FEC is an erasure coding library, based on zfec by Zooko, based on
code by Luigi Rizzo. The most widely known example of an erasure code
is the RAID-5 algorithm which makes it so that in the event of the
loss of any one hard drive, the stored data can be completely
recovered. The algorithm in the fec package has a similar effect, but
instead of recovering from the loss of only a single element, it can
be parameterized to choose in advance the number of elements whose
loss it can tolerate.

Hackage: 
http://hackage.haskell.org/packages/archive/fec/0.1.1/doc/html/Codec-FEC.html

It's new code, but the library that it's based on is well tested so
should be sound.



AGL

-- 
Adam Langley  [EMAIL PROTECTED]
http://www.imperialviolet.org   650-283-9641
___
Haskell-Cafe mailing list
Haskell-Cafe@haskell.org
http://www.haskell.org/mailman/listinfo/haskell-cafe


Re: [Haskell-cafe] Upgrading ByteString causes (seemingly) impossible RTS linker errs

2008-01-27 Thread Adam Langley
On Jan 27, 2008 12:24 PM, Don Stewart <[EMAIL PROTECTED]> wrote:
> It should be possible to specify that your lib depends on exactly
> 0.9.0.1 in the .cabal file.

In the same general area. When you upgrade something like bytestring
and have to update everything that was built with it (otherwise you'll
get type errors because the ByteString types from the two different
versions are different) cabal-install is of no help. Since it already
believes that I have those packages installed (correctly) if refuses
to install them again so I have to do it all by hand.

I realise that I should be sending darcs patches, not bug reports, but
a --force to reinstall an already installed package would be great. I
guess, ideally, cabal-install would know to rebuild everything that
depends on x when I upgrade x.

darcs patches might follow if I can get round to it (or the next time
I update bytestring...)

Cheers,


-- 
Adam Langley  [EMAIL PROTECTED]
http://www.imperialviolet.org   650-283-9641
___
Haskell-Cafe mailing list
Haskell-Cafe@haskell.org
http://www.haskell.org/mailman/listinfo/haskell-cafe


Re: [Haskell-cafe] CPP and INLINE pragmas

2008-01-20 Thread Adam Langley
Thanks Isaac and Malcolm. That neatly solves all my problems!


AGL

-- 
Adam Langley  [EMAIL PROTECTED]
http://www.imperialviolet.org   650-283-9641
___
Haskell-Cafe mailing list
Haskell-Cafe@haskell.org
http://www.haskell.org/mailman/listinfo/haskell-cafe


Re: [Haskell-cafe] Re: ANN: A triple of new packages for talking tothe outside world

2008-01-20 Thread Adam Langley
On Jan 15, 2008 7:33 PM, Adam Langley <[EMAIL PROTECTED]> wrote:
> Ok, no TH ;)

I've just uploaded binary-strict 0.2.2 to Hackage which factors most
of the common code out via CPP. Hopefully I didn't break anything.



AGL

-- 
Adam Langley  [EMAIL PROTECTED]
http://www.imperialviolet.org   650-283-9641
___
Haskell-Cafe mailing list
Haskell-Cafe@haskell.org
http://www.haskell.org/mailman/listinfo/haskell-cafe


[Haskell-cafe] CPP and INLINE pragmas

2008-01-20 Thread Adam Langley
Since CPP mode removes newlines in the out macro expansion. It appears
to be impossible to have a macro expand to a function with an INLINE
pragma since it appears to need to be in its own line.

For example:

#define GETHOSTWORD(name, m, type) \
{-# INLINE name #-} \
name :: m type ; \
name = getPtr (sizeOf (undefined :: type)) \

Does work (since you can't follow the INLINE pragma with anything else
on the same line.

Likewise:

#define GETHOSTWORD(name, m, type) \
name :: m type ; \
name = getPtr (sizeOf (undefined :: type)) \
{-# INLINE name #-}

Also doesn't work since the INLINE needs to start at the beginning of a line.

Has anyone a workaround for this, or a way to get the preprocessor to
output a newline?


Cheers

AGL

-- 
Adam Langley  [EMAIL PROTECTED]
http://www.imperialviolet.org   650-283-9641
___
Haskell-Cafe mailing list
Haskell-Cafe@haskell.org
http://www.haskell.org/mailman/listinfo/haskell-cafe


Re: [Haskell-cafe] bytea decoding

2008-01-20 Thread Adam Langley
2008/1/20 Bryan Green <[EMAIL PROTECTED]>:
> Does anyone know of a library that will handle bytea encodings from
> postgres?  The bytea field that I need to access contains a jpg file.  I
> want to retrieve it from the database and write it out for an image display
> program.

I'd love to see Don give the correct ByteString solution, but here's a
slow version I knocked up (probably buggy):

import qualified Data.ByteString as B
import Data.ByteString.Internal (c2w)
import Data.Maybe (catMaybes)
import Data.List (mapAccumL)

byteaDecode :: B.ByteString -> B.ByteString
byteaDecode = B.pack . catMaybes . snd . mapAccumL f initState . B.unpack where
  initState = (0, 0)
  f (0, _) 92 = ((1, 0), Nothing)
  f (0, _) x = ((0, 0), Just x)
  f (1, _) 92 = ((0, 0), Just 92)
  f (3, n) x = ((0, 0), Just (n * 8 + (x - 48)))
  f (c, n) x = ((c + 1, n * 8 + (x - 48)), Nothing)


AGL

-- 
Adam Langley  [EMAIL PROTECTED]
http://www.imperialviolet.org   650-283-9641
___
Haskell-Cafe mailing list
Haskell-Cafe@haskell.org
http://www.haskell.org/mailman/listinfo/haskell-cafe


Re: [Haskell-cafe] Re: ANN: A triple of new packages for talking tothe outside world

2008-01-17 Thread Adam Langley
On Jan 17, 2008 11:05 AM, Dominic Steinitz
<[EMAIL PROTECTED]> wrote:

> I'm not clear what happens if you
> are reading from a socket and not all the input has arrived but I'll think
> about that over the weekend.

At the moment, BitGet deals with strict ByteStrings only. One could use it
within a standard Get monad by getting a strict ByteString from the lazy input.
I believe that lazy ByteStrings got fixed a while back so that reading from a
socket doesn't block reading a whole block. (i.e. if you trickle data, byte by
byte, to a socket a lazy ByteString should return a spine of 1 byte strict
ByteStrings.)

A fully lazy BitGet would also be possible, of course, I've just not written it
yet ;)

> Adam Langley  imperialviolet.org> writes:
> Another thought: could e.g. getRightByteString be in the IO monad and then I
> don't have to run the Get(?) monad? Or is that a really stupid question?

If it were in the IO monad, I guess that you're suggesting that it read from a
handle? If that were the case, the remainder of the last byte would have to be
discarded because one can only read whole bytes from a Handle and there's no
mechanism for pushing back into it.

It's certainly possible to do, but I think a quick wrapper around a BitGet
would be the way to do it. If it's particually desirable I can add it, although
I'll admit that it seems a bit odd and I'm wondering what your use case is.

Cheers


AGL

--
Adam Langley  [EMAIL PROTECTED]
http://www.imperialviolet.org   650-283-9641
___
Haskell-Cafe mailing list
Haskell-Cafe@haskell.org
http://www.haskell.org/mailman/listinfo/haskell-cafe


Re: [Haskell-cafe] Re: ANN: A triple of new packages for talking tothe outside world

2008-01-16 Thread Adam Langley
On Jan 16, 2008 2:41 PM, Dominic Steinitz
<[EMAIL PROTECTED]> wrote:
> tlv_ bin =
>do tagValueVal <- getBits bin 5
>   tagConstructionVal <- getBits bin 1
>   tagTypeVal <- getBits bin 2
>
> I'm sure I'm wrong but putting bits into [Bool] doesn't look very efficient. 
> Of
> course, NewBinary didn't address what happened for n >= 8. Some possibilities
> are a) not allowing more than 8 b) returning [Word8] or c) (which I thought 
> was
> where we'd go) a ByteString with some sort of padding.

BitGet is just an API RFC at the moment, so I'm just describing it
here - not trying to justify it.

In BitGet there's getAsWord[8|16|32|64] which take a number of bits ($n$) and
returns the next $n$ bits in the bottom of a Word$x$. Thus, getAsWord8 is what
you call getBits and, if you had a 48 bit number, you could use getAsWord64 and
the bottom 48-bits of the resulting Word64 would be what you want.

Equally, asking for more than $x$ bits when calling getAsWord$x$ is a mistake,
however I don't check for it in the interest of speed.

There are also get[Left|Right]ByteString which return the next $n$ bits in a
ByteString of Word8's. The padding is either at the end of the last byte (left
aligned) or at the beginning of the first byte (right aligned).

If you did want a [Bool], you could use:
  bits <- sequence $ take n $ repeat getBit


AGL

--
Adam Langley  [EMAIL PROTECTED]
http://www.imperialviolet.org   650-283-9641
___
Haskell-Cafe mailing list
Haskell-Cafe@haskell.org
http://www.haskell.org/mailman/listinfo/haskell-cafe


Re: [Haskell-cafe] Re: ANN: A triple of new packages for talking tothe outside world

2008-01-15 Thread Adam Langley
On Jan 15, 2008 5:01 PM, Neil Mitchell <[EMAIL PROTECTED]> wrote:
> That would be bad. Then you'll have gone from Data.Binary being
> portable code, to being GHC specific code, and I will cry :'-(

Ok, no TH ;)


AGL

-- 
Adam Langley  [EMAIL PROTECTED]
http://www.imperialviolet.org   650-283-9641
___
Haskell-Cafe mailing list
Haskell-Cafe@haskell.org
http://www.haskell.org/mailman/listinfo/haskell-cafe


Re: [Haskell-cafe] Re: ANN: A triple of new packages for talking tothe outside world

2008-01-15 Thread Adam Langley
On Jan 15, 2008 3:26 PM, Don Stewart <[EMAIL PROTECTED]> wrote:
> Ok. That's awesome. I guess if you do all the TODOs for Binary like
> this, we should merge the code back in :)

Well, at the moment I'm pretty unhappy with the amount of code
duplication required both within binary-strict and between binary and
binary-strict. I think this code needs a whole lot of restructuring
(maybe a bit of TH for generating the common bits). I'll get to that
when it appears that the API seems reasonable.

AGL

-- 
Adam Langley  [EMAIL PROTECTED]
http://www.imperialviolet.org   650-283-9641
___
Haskell-Cafe mailing list
Haskell-Cafe@haskell.org
http://www.haskell.org/mailman/listinfo/haskell-cafe


Re: [Haskell-cafe] Re: ANN: A triple of new packages for talking tothe outside world

2008-01-15 Thread Adam Langley
On Jan 10, 2008 10:45 AM, Don Stewart <[EMAIL PROTECTED]> wrote:
> That's pretty much what we envisaged as the approach to take.
> Monad transformers adding some bit-buffer state over Get/Put.

For anyone who's still reading this thread...

I've just uploaded[1] binary-strict 0.2.1 which includes
Data.Binary.Strict.BitGet - a Get like monad which works by the bit.
I'm afraid that Haddock 2 is choaking on {-# UNPACK #-}, so I don't
have the HTML documentation to point to. (And I thought that Haddock 2
was going to fix all those parsing issues :( - hopefully I'm just
doing something stupid).

[1] 
http://hackage.haskell.org/cgi-bin/hackage-scripts/package/binary-strict-0.2.1

AGL

-- 
Adam Langley  [EMAIL PROTECTED]
http://www.imperialviolet.org   650-283-9641
___
Haskell-Cafe mailing list
Haskell-Cafe@haskell.org
http://www.haskell.org/mailman/listinfo/haskell-cafe


Re: [Haskell-cafe] Re: ANN: A triple of new packages for talking tothe outside world

2008-01-09 Thread Adam Langley
On Jan 9, 2008 5:01 PM, David Roundy <[EMAIL PROTECTED]> wrote:
> But I can't imagine an implementation in which this change wouldn't slow
> down getBytes for the normal case.  Perhaps the slowdown would be small,
> but it seems unwise to enforce that slowness at the API level, when we've
> already got a perfectly good API for fast binary IO.  Maybe there's some
> type hackery you could do to avoid a speed penalty, but that's a lot to add
> for a somewhat dubious benefit.

I believe that it would be an additional if statement in the fast path at least.

How about a BitGet monad which get be run in the Get monad?

> test :: Get ()
> test = do
>  runBitGet 2 (do
>getBitField 2)

So the first argument to runBitGet is the number of bytes to parse for
bit fields and then functions in BitGet can extract bit-length ints
etc.

Anyone like that idea?

AGL


-- 
Adam Langley  [EMAIL PROTECTED]
http://www.imperialviolet.org   650-283-9641
___
Haskell-Cafe mailing list
Haskell-Cafe@haskell.org
http://www.haskell.org/mailman/listinfo/haskell-cafe


Re: [Haskell-cafe] Data.Binary and serialising doubles in IEEE format

2008-01-09 Thread Adam Langley
On Jan 9, 2008 5:18 AM, allan <[EMAIL PROTECTED]> wrote:
> Essentially then, is there anyone that can write my 'putDouble' function
> or give some hint as to how I might do it.
> For the moment assume that I'm not really concerned with portability
> across platforms at least for the time being (I certainly don't think
> that the C program I'm attempting to communicate with is particularly
> portable anyway).
>

I happen to have the reverse functions lying around which might be
some help in doing this. They aren't terribly beautiful, but they'll
take an IEEE double or float from a Bytestring and return the value +
the remaining ByteString. (They aren't very well tested, but they have
been found to mostly work).

ddouble :: BS.ByteString -> (Double, BS.ByteString)
ddouble bytes = (encodeFloat fraction $ fromIntegral exp, rest) where
  fraction = sign * fromIntegral ((n .&. 0xf) .|. (if
rawExp > 0 then 0x10 else 0))
  sign = if n `shiftR` 63 == 0 then 1 else -1
  exp = rawExp - (1023 + 52)
  rawExp = (n `shiftR` 52) .&. 0x7ff
  (b, rest) = BS.splitAt 8 bytes
  n :: Word64
  n = foldl1 (.|.) $ map (\(s, v) -> (fromIntegral v) `shiftL` s) $
zip [0,8..56] $ BS.unpack b

dfloat :: BS.ByteString -> (Float, BS.ByteString)
dfloat bytes = (encodeFloat fraction $ fromIntegral exp, rest) where
  fraction = sign * fromIntegral ((n .&. 0x7f) .|. (if rawExp > 0
then 0x80 else 0))
  sign = if n `shiftR` 31 == 0 then 1 else -1
  exp = rawExp - (127 + 23)
  rawExp = (n `shiftR` 23) .&. 0xff
  (b, rest) = BS.splitAt 4 bytes
  n :: Word32
  n = foldl1 (.|.) $ map (\(s, v) -> (fromIntegral v) `shiftL` s) $
zip [0, 8, 16, 24] $ BS.unpack b


AGL

-- 
Adam Langley  [EMAIL PROTECTED]
http://www.imperialviolet.org   650-283-9641
___
Haskell-Cafe mailing list
Haskell-Cafe@haskell.org
http://www.haskell.org/mailman/listinfo/haskell-cafe


Re: [Haskell-cafe] ANN: A triple of new packages for talking to the outside world

2008-01-08 Thread Adam Langley
On Jan 7, 2008 11:17 AM, Bryan O'Sullivan <[EMAIL PROTECTED]> wrote:
> > It would seem that there would be three possible outcomes from an
> > incremental Get:
> >   - Failure: some bitstreams are just invalid and no amount of extra
> > data will ever fix that
> >   - Complete [Result]: the last chunk of data has been processed.
> > Maybe this should also include the remainder of the data?
> >   - Partial Cont [Result]: needs more data, but here's a (possibly
> > empty) list of results so far.
>
> Yes, that's more or less exactly what I had in mind.

Ok, see http://www.imperialviolet.org/IncrementalGet.hs

Hot off the keyboard and lacking documentation and comments ;)

There's a small test function at the bottom which demonstrates an
infinite parser which extracts 16-bit ints from whatever you feed it.

Questions:
  1) Should Finished include the remainder of the ByteString (e.g.
that which wasn't used by that parser)
  2) I've no idea what I've done to the parse speed
  3) I removed all the lookahead functions because, if the look ahead
runs out of data it's not clean what should be done. We can request
more with a partial result, but if the user doesn't have any more data
to give there's no way, currently, to signal this and cause the
lookahead to backtrack. Also, if we let the lookahead request more
data via a Partial, then there's no way for the user to tell that
Partial apart from one which has "real" values.

But if this is useful to you, make any requests. I'll (hopefully) do
them, clean it up and push a new release of binary-strict.

AGL


-- 
Adam Langley  [EMAIL PROTECTED]
http://www.imperialviolet.org   650-283-9641
___
Haskell-Cafe mailing list
Haskell-Cafe@haskell.org
http://www.haskell.org/mailman/listinfo/haskell-cafe


Re: [Haskell-cafe] ANN: A triple of new packages for talking to the outside world

2008-01-07 Thread Adam Langley
On Jan 6, 2008 9:13 PM, Bryan O'Sullivan <[EMAIL PROTECTED]> wrote:
> Ooh, nice.  We could really do with an incremental version, too, which
> could be spoonfed chunks of bytes, and dole out values as
> deserialisation completes.
>
> Passing back a Left String is in some sense not much of an improvement
> over calling error, as if I merely doesn't have enough bytes accumulated
> yet, I can't restart parsing from the point the bytes ran out.  Any
> chance you'd like to hand back a continuation instead?

The Left String was just a way to return an error string, otherwise a
parse failure can be a little oblique.

I think an incremental parser would be sufficiently different that it
would be a different monad. Of course, the code duplication (again)
makes me a sad panda, but avoiding that involves a lot of work on the
binary package and the maintainers were uninterested last time I tried
that.

It would seem that there would be three possible outcomes from an
incremental Get:
  - Failure: some bitstreams are just invalid and no amount of extra
data will ever fix that
  - Complete [Result]: the last chunk of data has been processed.
Maybe this should also include the remainder of the data?
  - Partial Cont [Result]: needs more data, but here's a (possibly
empty) list of results so far.

Applying a ByteString to the Cont from the Partial result would result
in one of the same three outcomes etc.

What do you think of that? It wouldn't be too hard to make up
something like that using ContT if it would be useful to you.

AGL

-- 
Adam Langley  [EMAIL PROTECTED]
http://www.imperialviolet.org   650-283-9641
___
Haskell-Cafe mailing list
Haskell-Cafe@haskell.org
http://www.haskell.org/mailman/listinfo/haskell-cafe


[Haskell-cafe] ANN: A triple of new packages for talking to the outside world

2008-01-06 Thread Adam Langley
Over the Xmas break I made some headway on writing an RPC package
since many of the ideas that I want to play with involve such a thing
as a basic building block. However, some might find some of the
spin-off packages that I wrote along the way useful.

* binary-strict:

This is mostly a cut-n-paste job from the excellent binary package
which provides Data.Binary.Strict.Get - a monad which is a drop in
replacement for Get, but which parses strict ByteStrings and returns
an Either, thus one can parse binary data without triggering an
exception on failure.

* codec-libevent:

This parses the "tagging" IDL files introduced in libevent[1]
1.4.0-beta and later and can generate the Haskell code for them. These
are platform-independent "structs" (in the C sense) and libevent
provides the complementary C infrastructure for them. You should be
able to understand almost everything these is to know about them from
a quick example:

struct msg {
string from_name = 1;
string to_name = 2;
optional struct[kill] attack = 3;
array struct[run] run = 4;
}

[1] http://monkey.org/~provos/libevent/

* control-timeout:

This module provides simple timeouts - e.g. actions which occur after
a given number of seconds and which can be canceled. It does it in a
not-totally-stupid fashion so that you don't need to worry about
setting hundreds of them.


AGL

-- 
Adam Langley  [EMAIL PROTECTED]
http://www.imperialviolet.org   650-283-9641
___
Haskell-Cafe mailing list
Haskell-Cafe@haskell.org
http://www.haskell.org/mailman/listinfo/haskell-cafe


[Haskell-cafe] Getting links to Haddock documentation in Hackage

2007-12-29 Thread Adam Langley
Some packages[1] have links on their Hackage pages to the haddock
generated documentation for each exported module[2]. However, many[3]
don't.

What's the secret to getting this generated documentation to work with
Hackage? Even packages for which `runhaskell Setup.lhs haddock` runs
don't (always) get the magic links.


Cheers,

AGL


[1]
http://hackage.haskell.org/cgi-bin/hackage-scripts/package/bzlib-0.4.0.1
[2] 
http://hackage.haskell.org/packages/archive/bzlib/0.4.0.1/doc/html/Codec-Compression-BZip.html
[3] http://hackage.haskell.org/cgi-bin/hackage-scripts/package/tar-0.1.1.1
-- 
Adam Langley  [EMAIL PROTECTED]
http://www.imperialviolet.org   650-283-9641
___
Haskell-Cafe mailing list
Haskell-Cafe@haskell.org
http://www.haskell.org/mailman/listinfo/haskell-cafe


Re: [Haskell-cafe] very elementary "import" problem .. help

2007-11-23 Thread Adam Langley
On Nov 23, 2007 1:43 PM, Galchin Vasili <[EMAIL PROTECTED]> wrote:
> I am loading the above script in ghci. However, System.Posix, System.Process
> and STM cannot be found. Network and Directory are found. I assume when ghc
> is installed on Windows that the Windows registry get populated with paths
> to GHC 'library" directory paths. Please someone tell me why I am having
> trouble importing these modules/libraries.

Well, NT used to pretend to have a POSIX layer because of some DOD
regulations I believe. I'm not sure if that still stands and wouldn't
be surprised if System.Posix just doesn't exist for Windows.
System.Process is marked as portable, so I'm not sure why that one
wouldn't be found. Finally, I believe that you want
Control.Concurrent.STM.


AGL

-- 
Adam Langley  [EMAIL PROTECTED]
http://www.imperialviolet.org   650-283-9641
___
Haskell-Cafe mailing list
Haskell-Cafe@haskell.org
http://www.haskell.org/mailman/listinfo/haskell-cafe


Re: [Haskell-cafe] Distributed haskell using Hadoop

2007-10-16 Thread Adam Langley
On 10/16/07, Brad Clow <[EMAIL PROTECTED]> wrote:
> I would prefer a more Haskell orientated solution and welcome any
> suggestions. If not maybe this will be of use to others.

Well, Hadoop is aiming towards a Google style of cluster processing
and the path towards that is pretty clear:

1) An XDR like serialisation scheme with support for backwards
compatibility (which involves unique-for-all-time ids in the IDL and
"required", "optional" etc tag). Data.Binary would be a great start
for this, but it's sadly lazy in parsing and they never applied my
patch for optional strictness so one would probably have to start from
scratch.

2) An RPC system which handles the most common use case: arguments and
replies are serialised using the above system, TCP transport, simple
timeouts, STM for concurrency.

Then you can start doing cool stuff like using the GHC API for code
motion and building a simple MapReduce like framework etc.

-- 
Adam Langley  [EMAIL PROTECTED]
http://www.imperialviolet.org   650-283-9641
___
Haskell-Cafe mailing list
Haskell-Cafe@haskell.org
http://www.haskell.org/mailman/listinfo/haskell-cafe


Re: [Haskell-cafe] ANNOUNCE: binary 0.4: high performance, pure binary parsing and serialisation

2007-10-06 Thread Adam Langley
On 10/6/07, Felipe Almeida Lessa <[EMAIL PROTECTED]> wrote:
> May I ask what are the changes? I didn't find some sort of changelog anywhere.

There's the darcs changes list. The descriptions there in are .. terse :)

But here's a selection:
  * Add getLazyByteStringNul.

-- | Get a lazy ByteString that is terminated with a NUL byte. Fails
-- if it reaches the end of input without hitting a NUL.
getLazyByteStringNul :: Get L.ByteString

  * Fix strictness bug in runGetState that led to runtime errors.

  * Port binary to ghc 6.8

  * add parallel driver  (appears to be a parallel quicktest - which
is cool because the tests took quite a while previously).

-- 
Adam Langley  [EMAIL PROTECTED]
http://www.imperialviolet.org   650-283-9641
___
Haskell-Cafe mailing list
Haskell-Cafe@haskell.org
http://www.haskell.org/mailman/listinfo/haskell-cafe


Re: [Haskell-cafe] GHC optimisations

2007-08-23 Thread Adam Langley
On 8/23/07, Neil Mitchell <[EMAIL PROTECTED]> wrote:
> It's not even referentially transparent in all cases. a == b may fail
> while the double's are in the high precision registers, and then
> succeed later on in the program once they are truncated. I think you
> have to specify -fexcess-precision with GHC to get this behaviour.

Also, if you tell GCC to use SSE (there are a number of flags, like
-march and -msse, -msse2 etc which can do this). It can stop using the
80-bit x87 registers for floating point and switch to 64-bit SSE
registers. Then you *don't* get this affect, but I'm not sure you can
be sure that you don't ever get it.

Floating point numbers make me sad :(


AGL

-- 
Adam Langley  [EMAIL PROTECTED]
http://www.imperialviolet.org   650-283-9641
___
Haskell-Cafe mailing list
Haskell-Cafe@haskell.org
http://www.haskell.org/mailman/listinfo/haskell-cafe


Re: [Haskell-cafe] Parsing binary data.

2007-08-20 Thread Adam Langley
On 8/19/07, Matthew Sackman <[EMAIL PROTECTED]> wrote:
> But it's vastly harder to do that for floats / non-integers. Now I know
> that the number classes in the Prelude are basically broken anyway and
> all really need rewriting, but it does seem completely arbitrary that
> Words somehow are only allowed to contain whole numbers!

Well, see the attached patch to Data.Binary to add
putFloat[32|64][be|le]. I got bored, so adding the Get functions is an
exercise for the reader :)

(And so because I think it needs unsafeSomethingIO and I'm a little
unsure about that).

If these functions would be useful for you, you should bug the binary
team to add something similar.


AGL

-- 
Adam Langley  [EMAIL PROTECTED]
http://www.imperialviolet.org   650-283-9641


data-binary-float.darcs
Description: Binary data
___
Haskell-Cafe mailing list
Haskell-Cafe@haskell.org
http://www.haskell.org/mailman/listinfo/haskell-cafe


Re: [Haskell-cafe] Parsing binary data.

2007-08-19 Thread Adam Langley
On 8/18/07, Matthew Sackman <[EMAIL PROTECTED]> wrote:
> Also, one thing to watch out for is the fact the existing Get and Put
> instances may not do anything like what you expect. For example, for
> some reason I expected that the instances of Get and Put for Float and
> Double would send across the wire Floats and Doubles in IEEE floating
> point standard. How wrong I was...

Ah, those aren't instances of Get and Put, but of Binary[1]. You use
the Binary instances via the functions 'get' and 'put' (case is
important).

Get and Put provide actions like "putWord32be", for which the
resulting bits are pretty much universally accepted. Binary has
default instances which uses Get and Put to serialise Haskell types
like [Int], or (Float, Float). Here the resulting bits aren't
documented, but you can read the code and I have some C code for
dealing with them somewhere if anyone is interrested. The
serialisation of Float is, indeed, nothing like IEEE in either
endianness.

(* and, although Get isn't currently a class, I have sent patches to
dons to make it so, with a default instance which matches current
behaviour and speed, and an alternative which returns a Maybe,
removing a little bit of lazyness in cases where you want to handle
parse failures in pure code. Hopefully something will happen with this
at the next sprint ;) )

[1] http://www.cse.unsw.edu.au/~dons/binary/Data-Binary.html#1

-- 
Adam Langley  [EMAIL PROTECTED]
http://www.imperialviolet.org   650-283-9641
___
Haskell-Cafe mailing list
Haskell-Cafe@haskell.org
http://www.haskell.org/mailman/listinfo/haskell-cafe


Re: [Haskell-cafe] Parsing binary data.

2007-08-18 Thread Adam Langley
On 8/18/07, Peter Cai <[EMAIL PROTECTED]> wrote:
> As the old version of this component is written in C, it's very natural that
> this protocol is base on C structure definitions, which are, unfortunately,
> very complicated.  And the worse is that every field in every structure must
> be converted to Network Endian.

You could certainly try Data.Binary[1] for this. It has a nice Get
monad with methods such as getWord32be which sounds like it might be
what you want. One caveat is that it's fully lazy - you get the result
immediately and parse errors can only be caught as exceptions when you
actually come to using the result. This is perfect for very large
messages, but might be slightly wrong for you.

[1] http://hackage.haskell.org/cgi-bin/hackage-scripts/package/binary-0.3

-- 
Adam Langley  [EMAIL PROTECTED]
http://www.imperialviolet.org   650-283-9641
___
Haskell-Cafe mailing list
Haskell-Cafe@haskell.org
http://www.haskell.org/mailman/listinfo/haskell-cafe


  1   2   >