Re: [Haskell-cafe] Efficient mutable arrays in STM

2011-10-30 Thread Ben Franksen
Benjamin Franksen wrote: David Barbour wrote: Create an extra TVar Int for every `chunk` in the array (e.g every 256 elements, tuned to your update patterns). Read-write it (increment it, be sure to force evaluation) just before every time you write an element or slice it or slice the array

Re: [Haskell-cafe] Efficient mutable arrays in STM

2011-10-30 Thread David Barbour
On Sun, Oct 30, 2011 at 6:20 PM, Ben Franksen ben.frank...@online.dewrote: According to the original STM paper the implementation does an equality test, albeit only for pointer equality. It strikes me as bad form to depend on characteristics of `the implementation`. An incremented integer

Re: [Haskell-cafe] Efficient mutable arrays in STM

2011-10-28 Thread Benjamin Franksen
David Barbour wrote: Create an extra TVar Int for every `chunk` in the array (e.g every 256 elements, tuned to your update patterns). Read-write it (increment it, be sure to force evaluation) just before every time you write an element or slice it or slice the array element. Incrementing and

Re: [Haskell-cafe] Efficient mutable arrays in STM

2011-10-27 Thread Ryan Ingram
On Tue, Oct 25, 2011 at 1:46 PM, Ben Franksen ben.frank...@online.dewrote: IME, there are (at least) two possible problems here, 1) transactions scale (quadratically, I think) with the number of TVars touched, Ouch! What would be the reason for that? I thought it would be linear... I

Re: [Haskell-cafe] Efficient mutable arrays in STM

2011-10-27 Thread Alberto G. Corona
The main question is: does the STM transaction actually see that I changed part of the underlying array, so that the transaction gets re-tried? Or do I have to implement this manually, and if yes: how? The transaction does not detect anything inside the unsafeIOtoSTM. But to implement

[Haskell-cafe] Efficient mutable arrays in STM

2011-10-25 Thread Ben Franksen
I have an application in mind where concurrent access to large arrays (up to millions of elements) of mostly small elements (Int or Double) is common. Typical access patterns would be chunk-wise, i.e. reads or writes from index n up to index m. Imagine stuff like images, scientific data, etc.

Re: [Haskell-cafe] Efficient mutable arrays in STM

2011-10-25 Thread Antoine Latter
As far as I lnow the function 'unsafeIOToSTM' is not transactional in nature - IO actions will be performed immediately and are not rolled back, and are then re-performed on retry. On Oct 25, 2011 12:49 PM, Ben Franksen ben.frank...@online.de wrote: I have an application in mind where concurrent

Re: [Haskell-cafe] Efficient mutable arrays in STM

2011-10-25 Thread David Barbour
On Tue, Oct 25, 2011 at 10:47 AM, Ben Franksen ben.frank...@online.dewrote: The main question is: does the STM transaction actually see that I changed part of the underlying array, so that the transaction gets re-tried? Or do I have to implement this manually, and if yes: how? Create an

Re: [Haskell-cafe] Efficient mutable arrays in STM

2011-10-25 Thread Ben Franksen
David Barbour wrote: On Tue, Oct 25, 2011 at 10:47 AM, Ben Franksen ben.frank...@online.dewrote: The main question is: does the STM transaction actually see that I changed part of the underlying array, so that the transaction gets re-tried? Or do I have to implement this manually, and

Re: [Haskell-cafe] Efficient mutable arrays in STM

2011-10-25 Thread Ketil Malde
Ben Franksen ben.frank...@online.de writes: An array of TVars is certainly *much* too inefficient for what I have in mind w.r.t. both memory and cpu time. You must be a lot more confident than I if you say this without benchmarking first. :-) IME, there are (at least) two possible problems

Re: [Haskell-cafe] Efficient mutable arrays in STM

2011-10-25 Thread Bryan O'Sullivan
On Tue, Oct 25, 2011 at 1:24 PM, Ketil Malde ke...@malde.org wrote: You must be a lot more confident than I if you say this without benchmarking first. :-) IME, there are (at least) two possible problems here, 1) transactions scale (quadratically, I think) with the number of TVars touched, so

Re: [Haskell-cafe] Efficient mutable arrays in STM

2011-10-25 Thread Ben Franksen
Ketil Malde wrote: Ben Franksen ben.frank...@online.de writes: An array of TVars is certainly *much* too inefficient for what I have in mind w.r.t. both memory and cpu time. You must be a lot more confident than I if you say this without benchmarking first. :-) Ok, not science, but an