I split the string in lines, then map some functions on each line to
parse the data, and finally calling toU, for converting to an UArr.
Just to make sure (code fragments or, better, reduced examples
would make it easier to see what the discussion is about): are you
forcing the UArr to be constructed before putting it into the Map?
parse handle =
contents <- S.hGetContents handle
let v = map singleton' $ ratings contents
let m = foldl1' (unionWith appendU) v
v `seq` return $! m
where
-- Build a Map with a single movie rating
singleton' :: (Word32, Word8) -> MovieRatings
singleton' (id, rate) =
singleton (fromIntegral $ id) (singletonU $ pairS (id, rate))
That helps to make things clearer, I think. One issue is
the nature of Maps (strict in keys, non-strict in values).
- neither singleton nor unionWith are strict in the Map values, so
nothing here forces the evaluation of rate or construction
of UArr
Prelude Data.IntMap> (unionWith (++) (singleton 1 undefined) (singleton 2
undefined)) `seq` ()
()
- singletonU is strict, but that only means that it will evaluate its
parameter if it is evaluated itself (which it isn't, because
singleton isn't strict)
- seq on a list only forces the first node of the list ((:),[],_|_),
so (v `seq`) isn't likely to help much. Also, you probably
do not want to force the whole list of singletons before
builing the Map, you want the singletons to be constructed
and consumed incrementally.
- forcing a Map doesn't force any of the values, nor does
it force more than the top-level node of whatever the
internal Map representation is, so (return $! m) isn't much
help, either (by nature of unionWith and foldl1', it'll force
all keys before it can say anything much about the Map,
but the values remain untouched, burried further under
unevaluated (++)s)
type Rating = Word32 :*: Word8
type MovieRatings = IntMap (UArr Rating) -- UArr from uvector
A standard trick to keep Map values evaluated by construction
is to make the availability of keys dependent on their values, eg
(singleton key) $! value. That won't help with unionWith and
the appendUs, but it should allow the source string references
to be dropped early, as the singletons are constructed.
Hth,
Claus
_______________________________________________
Haskell-Cafe mailing list
[email protected]
http://www.haskell.org/mailman/listinfo/haskell-cafe
