Hi,

i currently use this code to divide work between worker threads:
    
    
    var linePtr: seq[pointer]
      
      for line in memSlices(memfiles.open($args["--inputFile"], mode = fmRead)):
        linePtr.add(line.data)
      
      let batchSize = (linePtr.len / parseInt($args["--cores"])).ceil().toInt()
      let f = system.open($args["--inputFile"])
      
      {.push experimental: "parallel".}
      parallel:
        for i in 0..(parseInt($args["--cores"])-1):
          spawn worker(f, linePtr.at((batchSize * i)), batchSize)
      {.pop.}
    
    
    Run

and this worker function:
    
    
    proc worker(fd: File, offset, size: int) {.thread.} =
      let fs = newFileStream(fd)
      
      fs.setPosition(offset)
      
      var i = 0
      while i < size:
        echo i, " ", size, " ", fs.atEnd(), " ", fs.readLine()
        inc(i)
    
    
    Run

now the thing is: the program throws after a random amount of lines an IOError; 
when i wrap the while-loop as follows:
    
    
    while i < size:
      try:
        echo i, " ", size, " ", fs.atEnd(), " ", fs.readLine()
      except IOError:
        echo i, " ", size, " ", fs.atEnd()
      
      inc(i)
    
    
    Run

i see that fs.atEnd() returns true even if i shuld have lines left. Why is this 
happening and how do i crcumvent it?

Also a little side question: i currently divide into chunks of size 
ceil(total_lines/core_count) this means the last chunk is bigger or smaller 
than the previous ones. Is it possible to use fs.atEnd() as in while i<size and 
not fs.atEnd()? Because i didn't mange to get this to work either :(.

Thx in advance, ff

Reply via email to