Re: [Haskell-cafe] Re: Go Haskell! - array libraries
On 30/11/2008, at 11:36, Don Stewart wrote: Should mutable arrays have list-like APIs? All the usual operations, just in-place and destructive where appropriate? I don't know. To be honest, I don't think that the term mutable array describes a single data structure. For instance, one of the central questions which unveils a whole bunch of design possibilities is: can mutable arrays be concatenated and how does that work if yes? Roman ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] Re: Go Haskell! - array libraries
Should mutable arrays have list-like APIs? All the usual operations, just in-place and destructive where appropriate? I don't know. To be honest, I don't think that the term mutable array describes a single data structure. For instance, one of the central questions which unveils a whole bunch of design possibilities is: can mutable arrays be concatenated and how does that work if yes? wrt the first part: within a pure functional language, mutable arrays are something I (am forced to) choose when the implementation is unable to see some essential optimization opportunities in the way I am using immutable arrays; whenever I am forced to go down that route, I'd at least like to have an equivalent set of operations. The move from immutable arrays to mutable arrays is painful enough as it is. So the suggested design criterion would be: as uniform an API as possible, just that immutable arrays cannot make some essential performance guarantees and some operations sit a little uncomfortably in a librart where those guarantees are important. wrt the second part: I'm not sure whether you'll be able to choose a single one of those design possibilities as the only one (perhaps the concatenation is about simpler code, so virtual concatenation would be sufficient and copying possible, but perhaps the goal was locality, so copying would be needed, or perhaps a mixture of both, and the limited disruption in locality is less expensive than the cost of copying, ..), so you might want to expose several of them, with a separate way of selecting a default or I-don't-care choice (imilar to why you have several array libs at present, just that there should be a common API and documentation of design alternatives/rationales;-). -- Btw, it would really be nice if packages took a hint from academic publishing: good papers are expected not only (a) to provide new content, but also (b) to position that content wrt related work and (c) to make explicit what the new contributions/goals are. As forks have become more common on hackage, perhaps .cabal files could be extended with two fields, pointing to related packages (this is more specific than category, linking to packages that might be used instead or in combination with the current package) and giving the rationale/high-lights for the package in text form. In the meantime, it would be great if all packages came with good old README files, which should be linked from the .cabal-based package descriptions on hackage, covering all those useful bits of information that are not yet covered in .cabal files, and giving an overview of what the package is about if the package does not have its own webspace (as is often the case). That would improve on the current situation, where sometimes all one has to go on is the package name and a one-liner description. Random example: looking at hackage package categories, I'd look for array libraries under 'data structures', but they are actually spread over 'data structures', 'data', and 'maths' (perhaps others?). Once I've found one, say 'vector' or 'uvector', what do the package descriptions tell me about the packages and their relation, or their relative advantages? The descriptions are brief, the home pages are actually home repositories, the haddocks seem to have details only, no overview, and from the references/authors/uvector README, one might think these are actually the same library, sequential libs spun off from the same data-parallelism project; only that the haddock details suggest that these libraries are quite different. How? Why? What other alternatives are there? Note that I'm not attacking any specific packages, I would just encourage all package authors to try and see their packages from the perspective of a hackage user: what information does the user look for, what information does the package description offer? -- While we're on the topic: hackage has grown so much and so rapidly, that it might be worthwhile to have a hackage editor, not in the sense of accepting/rejecting packages (not wanted at present), nor in the sense of objective quality measurements (that is in the process of being automated, according to Duncan), but in the sense of trying to guarantee a subjectively useful overall presentation (category organisation, finding and putting side-by-side related packages despite different names/categories, suitability of package descriptions, getting package authors to talk, etc.). And no, I'm not volunteering!-) But I would imagine that someone might find this a useful way of contributing. Such a hackage editor would also be able to give the Haskell Community Report editor a hand by summarizing hackage activity/trends and highlighting interesting projects that the HCAR editor might want to solicit reports for. From my own time as HCAR editor, I recall finding and chasing interesting projects as well as deciding how to structure the toolslibraries sections as
Re: [Haskell-cafe] Re: Go Haskell! - array libraries
On Sunday 30 November 2008 6:28:29 am Roman Leshchinskiy wrote: On 30/11/2008, at 11:36, Don Stewart wrote: Should mutable arrays have list-like APIs? All the usual operations, just in-place and destructive where appropriate? I don't know. To be honest, I don't think that the term mutable array describes a single data structure. For instance, one of the central questions which unveils a whole bunch of design possibilities is: can mutable arrays be concatenated and how does that work if yes? I don't know about concatenation, but it is useful for them to be able to be zipped. For instance: schwartz f arr = do keys - cloneWith f arr pairs - zip keys arr sortBy (comparing fst) pairs (cloneWith should construct a new array where new[i] = f (old[i])) is a mutable sort of arr using the comparison compare e e' = compare (f e) (f e') caching the results of f for each element. This, obviously, takes advantage of the fact that sorting the zipped array mutably updates the underlying (original) arrays. On the other hand, I could see such an operation easily leading to bugs if it slips your mind that changing one array can effect another (maybe it should be called unsafeZip :)). -- Dan ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] Re: Go Haskell! - array libraries
Lennart Augustsson wrote: But I don't want Perl, I want a well designed language and well designed libraries. I think it's find to let libraries proliferate, but at some point you also need to step back and abstract. I agree. On Fri, Nov 28, 2008 at 9:46 PM, Don Stewart [EMAIL PROTECTED] wrote: andrewcoppin: What *I* propose is that somebody [you see what I did there?] should sit down, take stock of all the multitudes of array libraries, what features they have, what obvious features they're missing, and think up a good API from scratch. Once we figure out what the best way to arrange all this stuff is, *then* we attack the problem of implementing it for real. It seems lots of people have written really useful code, but we need to take a step back and look at the big picture here before writing any more of it. No. My view would be to let the free market of developers decide what is best. No bottlenecks -- there's too many Haskell libraries already (~1000 now). And this approach has yielded more code than ever before, more libraries than ever before, and library authors are competing. So let the market decide. We're a bazaar, not a cathedral. I find this kind of attitude disturbing. Are you seriously asserting that it's bad for people to stop and think about their designs before building? That it's bad for people to get together and coordinate their efforts? Would you really prefer each and every developer to reinvent the wheel until we have 50,000 equivilent but slightly different wheel implementations? Certainly you seem obsessed with the notion that more packages on Hackage == better. Well in my book, quantity /= quality. (The latter being vastly more important than the former - while admittedly far harder to measure objectively.) I would far prefer to see one well-written library that solves the problem properly than see 25 incompatible libraries that all solve small fragments of the problem poorly. In the latter case, there will be no competition between libraries; everybody will just give up and not use *any* of them. You _can_ have too much choice! I really hope I'm not the only person here who sees it this way... ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] Re: Go Haskell! - array libraries
On 29/11/2008, at 11:49, Claus Reinke wrote: Yes, it is very difficult. A sensible API for a standard array library is something that needs more research. FWIW, I don't know of any other language that has what I'd like to see in Haskell. C+ + probably comes closest but they have it easy - they don't do fusion. I assume you've looked at SAC? http://www.sac-home.org/ Yes. They have it even easier - they don't have polymorphism, they don't have higher-order functions, they don't have boxing and laziness and in a sense, they don't even do general-purpose programming, just scientific algorithms. And they have no existing language to integrate their stuff with. This is not to imply that their work isn't interesting and valuable; IMO, it just doesn't really help us when it comes to designing a Haskell array library. Roman ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] Re: Go Haskell! - array libraries
On Fri, 2008-11-28 at 22:20 +, Lennart Augustsson wrote: But I don't want Perl, I want a well designed language and well designed libraries. I think it's find to let libraries proliferate, but at some point you also need to step back and abstract. Yes, let the ideas simmer and when we can identify a consensus then we can standardise something by including it into the Haskell platform. There's obviously judgement involved to decide when it's right to standardise. We can see all around us the results of standardising too early or too late. Duncan ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] Re: Go Haskell! - array libraries
On Fri, 28 Nov 2008 19:00:38 -0500, Roman Leshchinskiy [EMAIL PROTECTED] wrote: On 29/11/2008, at 10:47, Claus Reinke wrote: [...] And would it be difficult for you all to agree on a standard API, to make switching between the alternatives easy (if it is indeed impossible to unify their advantages in a single library, the reasons for which should also be documented somewhere)? Yes, it is very difficult. A sensible API for a standard array library is something that needs more research. FWIW, I don't know of any other language that has what I'd like to see in Haskell. C++ probably comes closest but they have it easy - they don't do fusion. [...] Would you elaborate on what you'd like to see in an array library? And perhaps which C++ array library you are thinking of? Your C++ comment caught my attention, and now I'm curious. Surely you don't mean C-style arrays. :-D Regards, Brad Larsen ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] Re: Go Haskell! - array libraries
John Lato wrote: I would love to see a perfect, unified array library in Haskell. I think everyone would. However, the problem Don, Roman, and others have raised is that there is no single consensus on what that library would look like, or how it would be implemented. It might be impossible for one library to fill the entire design space. Don's point is that, since this isn't yet a solved problem, having multiple implementations available to see what works well (and what doesn't) is a necessary step in finding a solution. Interesting. I thought this was more or less a solved problem, it's just that nobody has yet had time to implement it all. I mean, the Haskell '98 array libraries are OK, they're just rather incomplete. We could do with the ability to have unboxed arrays of arbitrary types. (Remember, this is the default position in Pascal / C / C++ / most normal programming languages - although admittedly they don't have ADTs.) It would be useful to have the option to dispence with Ix. (And bounds checks, if you're sure you don't need them.) It would also be useful to be able to slice arrays. And we have the new parallel-array stuff coming now, but (AFAIK) it's fairly nontrivial to switch between normal arrays and parallel ones. Plus there are lots and lots of obvious and useful functions that aren't in the libraries. (E.g., in-place map for mutable arrays.) It seems silly to reimplement these every time you want to do something; they should be in the libraries. I doubt there will ever be a perfect library for anything. But that doesn't mean we can't take a few steps in the right direction. ;-) I also think this is a exactly why Hackage needs a way to indicate that packages are deprecated/superceded by other packages. There was some talk about this recently, and IIRC even a submitted patch. I hope that gets adopted soon. Agreed. This goes beyond array libraries; do you have any idea how many binary packages there are? ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] Re: Go Haskell! - array libraries
andrewcoppin: My view would be to let the free market of developers decide what is best. No bottlenecks -- there's too many Haskell libraries already (~1000 now). And this approach has yielded more code than ever before, more libraries than ever before, and library authors are competing. So let the market decide. We're a bazaar, not a cathedral. I find this kind of attitude disturbing. Are you seriously asserting that it's bad for people to stop and think about their designs before building? That it's bad for people to get together and coordinate their efforts? Would you really prefer each and every developer to reinvent the wheel until we have 50,000 equivilent Strawman, Andrew, and rather silly too. I'm aggressively in favour of reuse. That's why I advocate everyone use and contribute to Hackage, so that they can reuse other's work, and they can collaborate on existing code. The current approach of /people who do things/ is working very well. They're designing and implementing new ideas in the language, leading to interesting collaborations, and new designs, and more code, that does more things, than ever before. And I'm in favour of that. I never thought I'd say the day when people complained about there being too many libraries for Haskell. Mwhahaha! -- Don ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] Re: Go Haskell! - array libraries
Austin Seipp wrote: Excerpts from Andrew Coppin's message of Sat Nov 29 03:37:58 -0600 2008: Are you seriously asserting that it's bad for people to stop and think about their designs before building? To be fair, I don't think you're in a position to say whether the authors of these libraries took careful consideration in their design or not; unless, of course, you wrote one of them and *didn't* think about the design? I said I think we should take a step back and work out a plan and Don said no, I think we should just keep blindly hacking away instead. So I said that seems like a very bad idea to me... ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] Re: Go Haskell! - array libraries
andrewcoppin: Austin Seipp wrote: Excerpts from Andrew Coppin's message of Sat Nov 29 03:37:58 -0600 2008: Are you seriously asserting that it's bad for people to stop and think about their designs before building? To be fair, I don't think you're in a position to say whether the authors of these libraries took careful consideration in their design or not; unless, of course, you wrote one of them and *didn't* think about the design? I said I think we should take a step back and work out a plan and Don said no, I think we should just keep blindly hacking away instead. So I said that seems like a very bad idea to me... I didn't say that - you just made it up! And you even added fake quotes! Andrew, seriously, it's about time you contributed some code, rather than just empty noise on the list? -- Don ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] Re: Go Haskell! - array libraries
Henning Thielemann wrote: I suspect that this particular function is less useful than you think. It safes one allocation and might be faster since it uses less cache, but on the other hand, it cannot be fused. If the array is seriously large, you don't want to have five or six versions of it sitting in memory until the GC comes along to remove it. (OTOH, presumably an unboxed array can be allocated or freed quite quickly...) At a minimum, you've going to end up with two copies in memory - the existing one, and the one being built. I think in-place array updates are only sensible for writing array elements in really random order. As long as you can formulate your algorithm the way read from random indices, but write a complete array from left to right, there is almost no need for mutable arrays. Does quick-sort fit that pattern? (I'm guessing yes, since all you need to do is filtering and merging...) Yeah, generally you only need arrays if you really want random access. Except in Haskell, where an array also happens to be the only way of storing large numbers of values unboxed. So sometimes you'll use an array because you want to save space. (E.g., parsing text is usually a sequential process with no random access, but you probably want to use ByteString all the same!) I sometimes also use unboxed arrays for the increase in strictness. I guess the thing to do would be to measure the difference... ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] Re: Go Haskell! - array libraries
On 30/11/2008, at 02:43, Brad Larsen wrote: On Fri, 28 Nov 2008 19:00:38 -0500, Roman Leshchinskiy [EMAIL PROTECTED] wrote: On 29/11/2008, at 10:47, Claus Reinke wrote: [...] And would it be difficult for you all to agree on a standard API, to make switching between the alternatives easy (if it is indeed impossible to unify their advantages in a single library, the reasons for which should also be documented somewhere)? Yes, it is very difficult. A sensible API for a standard array library is something that needs more research. FWIW, I don't know of any other language that has what I'd like to see in Haskell. C++ probably comes closest but they have it easy - they don't do fusion. [...] Would you elaborate on what you'd like to see in an array library? I'd like to have a library which is efficient (in particular, implements aggressive fusion), is roughly equivalent to the current standard list library in power and supports strict/unboxed/mutable arrays. It should also provide a generic framework for implementing new kinds of arrays. And eventually, it should also be usable in high- performance and parallel algorithms. And perhaps which C++ array library you are thinking of? Your C++ comment caught my attention, and now I'm curious. Surely you don't mean C-style arrays. :-D No, I meant vector, basic_string and deque which are provided by the standard library. Roman ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] Re: Go Haskell! - array libraries
On 30/11/2008, at 08:32, Andrew Coppin wrote: Henning Thielemann wrote: I suspect that this particular function is less useful than you think. It safes one allocation and might be faster since it uses less cache, but on the other hand, it cannot be fused. Hmm, I haven't seen your original message but I suspect you are talking about in-place map. In that case, this is not entirely true. Shameless plug: http://www.cse.unsw.edu.au/~rl/publications/recycling.html I think in-place array updates are only sensible for writing array elements in really random order. As long as you can formulate your algorithm the way read from random indices, but write a complete array from left to right, there is almost no need for mutable arrays. Many array algorithms cannot really be written in this way. I think we do need mutable arrays and they should provide much more than just read/write. How to integrate them nicely with immutable arrays is not really clear, though. Roman ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] Re: Go Haskell! - array libraries
rl: On 30/11/2008, at 08:32, Andrew Coppin wrote: Henning Thielemann wrote: I suspect that this particular function is less useful than you think. It safes one allocation and might be faster since it uses less cache, but on the other hand, it cannot be fused. Hmm, I haven't seen your original message but I suspect you are talking about in-place map. In that case, this is not entirely true. Shameless plug: http://www.cse.unsw.edu.au/~rl/publications/recycling.html I think in-place array updates are only sensible for writing array elements in really random order. As long as you can formulate your algorithm the way read from random indices, but write a complete array from left to right, there is almost no need for mutable arrays. Many array algorithms cannot really be written in this way. I think we do need mutable arrays and they should provide much more than just read/write. How to integrate them nicely with immutable arrays is not really clear, though. Should mutable arrays have list-like APIs? All the usual operations, just in-place and destructive where appropriate? -- Don ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] Re: Go Haskell! - array libraries
Henning Thielemann wrote: On Fri, 28 Nov 2008, Simon Marlow wrote: Manuel M T Chakravarty wrote: Claus Reinke: What do those folks working on parallel Haskell arrays think about the sequential Haskell array baseline performance? You won't like the answer. We are not happy with the existing array infrastructure and hence have our own. Roman recently extracted some of it as a standalone package: http://hackage.haskell.org/cgi-bin/hackage-scripts/package/vector In the longer run, we would like to factor our library into DPH-specific code and general-purpose array library that you can use independent of DPH. So we have two vector libraries, vector and uvector, which have a lot in common - they are both single-dimension array types that support unboxed instances and have list-like operations with fusion. They ought to be unified, really. It's worse: http://hackage.haskell.org/cgi-bin/hackage-scripts/package/storablevector :-) What *I* propose is that somebody [you see what I did there?] should sit down, take stock of all the multitudes of array libraries, what features they have, what obvious features they're missing, and think up a good API from scratch. Once we figure out what the best way to arrange all this stuff is, *then* we attack the problem of implementing it for real. It seems lots of people have written really useful code, but we need to take a step back and look at the big picture here before writing any more of it. IMHO, anyway... ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] Re: Go Haskell! - array libraries
andrewcoppin: What *I* propose is that somebody [you see what I did there?] should sit down, take stock of all the multitudes of array libraries, what features they have, what obvious features they're missing, and think up a good API from scratch. Once we figure out what the best way to arrange all this stuff is, *then* we attack the problem of implementing it for real. It seems lots of people have written really useful code, but we need to take a step back and look at the big picture here before writing any more of it. No. My view would be to let the free market of developers decide what is best. No bottlenecks -- there's too many Haskell libraries already (~1000 now). And this approach has yielded more code than ever before, more libraries than ever before, and library authors are competing. So let the market decide. We're a bazaar, not a cathedral. -- Don ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] Re: Go Haskell! - array libraries
But I don't want Perl, I want a well designed language and well designed libraries. I think it's find to let libraries proliferate, but at some point you also need to step back and abstract. -- Lennart On Fri, Nov 28, 2008 at 9:46 PM, Don Stewart [EMAIL PROTECTED] wrote: andrewcoppin: What *I* propose is that somebody [you see what I did there?] should sit down, take stock of all the multitudes of array libraries, what features they have, what obvious features they're missing, and think up a good API from scratch. Once we figure out what the best way to arrange all this stuff is, *then* we attack the problem of implementing it for real. It seems lots of people have written really useful code, but we need to take a step back and look at the big picture here before writing any more of it. No. My view would be to let the free market of developers decide what is best. No bottlenecks -- there's too many Haskell libraries already (~1000 now). And this approach has yielded more code than ever before, more libraries than ever before, and library authors are competing. So let the market decide. We're a bazaar, not a cathedral. -- Don ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] Re: Go Haskell! - array libraries
But I don't want Perl, I want a well designed language and well designed libraries. I think it's find to let libraries proliferate, but at some point you also need to step back and abstract. -- Lennart Especially so if the free marketeers claim there is something fundamentally wrong with the standard libraries and language, as in the case of arrays. When someone did that nice little survey of the last bunch of array libraries (Bulat, I think? now in the wiki book), I was hoping there would be a grand unification soon. Instead, it seems that those who have worked most with Haskell arrays recently have simply abandoned all of the standard array libraries. Okay, why not, if there are good reasons. But can't you document those reasons, for each of your alternative proposals, so that people have some basis on which to choose (other than who has the loudest market voice;-)? And would it be difficult for you all to agree on a standard API, to make switching between the alternatives easy (if it is indeed impossible to unify their advantages in a single library, the reasons for which should also be documented somewhere)? And what is wrong about Simon's suggestion, to use the standard array lib APIs on top of your implementations? Not that I see Haskell' coming soon, but I'd certainly not want it to continue standardising a kind of array that appears to have no backing among the Haskell array user/library author community. Nor would I like something as central as arrays to remain outside the standard, where it won't remain stable enough for Haskell programmers to rely on in the long run. bazaar, yes; mayhem, no. Claus On Fri, Nov 28, 2008 at 9:46 PM, Don Stewart [EMAIL PROTECTED] wrote: andrewcoppin: What *I* propose is that somebody [you see what I did there?] should sit down, take stock of all the multitudes of array libraries, what features they have, what obvious features they're missing, and think up a good API from scratch. Once we figure out what the best way to arrange all this stuff is, *then* we attack the problem of implementing it for real. It seems lots of people have written really useful code, but we need to take a step back and look at the big picture here before writing any more of it. No. My view would be to let the free market of developers decide what is best. No bottlenecks -- there's too many Haskell libraries already (~1000 now). And this approach has yielded more code than ever before, more libraries than ever before, and library authors are competing. So let the market decide. We're a bazaar, not a cathedral. -- Don ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] Re: Go Haskell! - array libraries
On 29/11/2008, at 08:43, Andrew Coppin wrote: What *I* propose is that somebody [you see what I did there?] should sit down, take stock of all the multitudes of array libraries, what features they have, what obvious features they're missing, and think up a good API from scratch. Once we figure out what the best way to arrange all this stuff is, *then* we attack the problem of implementing it for real. That is the idea behind vector. I don't know how good it is but it's the best I could come up with (or will be once it's finished). That said, I don't think there is such a thing as a perfect array API. Different libraries serve different purposes. Roman ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] Re: Go Haskell! - array libraries
claus.reinke: But I don't want Perl, I want a well designed language and well designed libraries. I think it's find to let libraries proliferate, but at some point you also need to step back and abstract. -- Lennart Especially so if the free marketeers claim there is something fundamentally wrong with the standard libraries and language, as in the case of arrays. When someone did that nice little survey of the last bunch of array libraries (Bulat, I think? now in the wiki book), I was hoping there would be a grand unification soon. Instead, it seems that those who have worked most with Haskell arrays recently have simply abandoned all of the standard array libraries. I don't think Bulat uploaded his libraries to hackage in the end. And if it's not on hackage, then no one will use it, so it may as well not exist. Okay, why not, if there are good reasons. But can't you document those reasons, for each of your alternative proposals, so that people have some basis on which to choose (other than who has the loudest market voice;-)? And would it be difficult for you all to agree on a standard API, to make switching between the alternatives easy (if it is indeed impossible to unify their advantages in a single library, the reasons for which should also be documented somewhere)? And what is wrong about Simon's suggestion, to use the standard array lib APIs on top of your implementations? Nothing. This would be great. Who's volunteering to write the code? The new 'list-like' fusible array libraries are still in alpha, anyway. Not that I see Haskell' coming soon, but I'd certainly not want it to continue standardising a kind of array that appears to have no backing among the Haskell array user/library author community. Nor would I like something as central as arrays to remain outside the standard, where it won't remain stable enough for Haskell programmers to rely on in the long run. Hence the Haskell Platform. To provide the stability that people rely on in the long run. Haskell' is not the process by which new libraries will be standardised -- there's simply too many libraries being produced. The platform let's us: * Take libraries out of the standardisation process. * Let developers develop in competition, and converge on agreed designs. * Let users decide what to use, rather than waste time standardising things when the developer community has already moved on. * And then publish a list of blessed code. Since arrays are just another (non-obvious) data structure, there's a huge design space: * flat and/or nested arrays? * strict or lazy or eager? * callable from C or Fortran? * mutable/immutable * polymorphic in what dimensions? * mmap-able? * read / write API, or list-like API? We've not yet found the perfect implementation, but we're learning a lot. And since it is not yet clear what the optimal solution looks like, I say let the developers keep hacking for a while, until we get an idea of what works. And by all means, if someone thinks they know what the best API is, step up and show us the implementation! -- Don ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] Re: Go Haskell! - array libraries
On 29/11/2008, at 10:47, Claus Reinke wrote: But I don't want Perl, I want a well designed language and well designed libraries. I think it's find to let libraries proliferate, but at some point you also need to step back and abstract. -- Lennart Especially so if the free marketeers claim there is something fundamentally wrong with the standard libraries and language, as in the case of arrays. When someone did that nice little survey of the last bunch of array libraries (Bulat, I think? now in the wiki book), I was hoping there would be a grand unification soon. Instead, it seems that those who have worked most with Haskell arrays recently have simply abandoned all of the standard array libraries. Okay, why not, if there are good reasons. But can't you document those reasons, for each of your alternative proposals, so that people have some basis on which to choose (other than who has the loudest market voice;-)? I think so far, it's always been the same two reasons: efficiency and ease of use. H98 arrays are inherently inefficient and IMO not very easy to use, at least not for the things that I'm doing. And would it be difficult for you all to agree on a standard API, to make switching between the alternatives easy (if it is indeed impossible to unify their advantages in a single library, the reasons for which should also be documented somewhere)? Yes, it is very difficult. A sensible API for a standard array library is something that needs more research. FWIW, I don't know of any other language that has what I'd like to see in Haskell. C++ probably comes closest but they have it easy - they don't do fusion. And what is wrong about Simon's suggestion, to use the standard array lib APIs on top of your implementations? Again, IMO H98 arrays are only suitable for a very restricted set of array algorithms. Roman ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] Re: Go Haskell! - array libraries
Yes, it is very difficult. A sensible API for a standard array library is something that needs more research. FWIW, I don't know of any other language that has what I'd like to see in Haskell. C++ probably comes closest but they have it easy - they don't do fusion. I assume you've looked at SAC? http://www.sac-home.org/ Their main research and development focus was/has been on arrays (fusion/layout/padding/tiling/slicing/data-parallelism/shape-invariance (source algorithms parameterized over array dimensionality/shape)/ whole-array ops/list-like ops/lots of surface operations reducable to a minimal set of core operations that need implementation/cache behaviour/performance/performance/performance/..). When they started out, I tried to make the point that I would have liked to have their wonderful array ideas in our otherwise wonderful language, but they preferred to follow their own way towards world domination (*). Does that sound familiar?-) Claus (*) ok, they did have a good motive: they came out of a research group that had done non-sequential functional programming in the 1980s, with all the things we see today: great speedups, shame about the sequential baseline; creating parallel threads is easy, load balancing slightly harder, but pumping (creating thread hierarchies recursively, only to see them fold into a small result, for the process to begin again) is a waste, etc.; so they decided to start from a fast sequential baseline instead of full functional language, and designed their language around arrays instead of trying to add arrays to an existing language. ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
