RFC: Recursive iterator macro

[jyapayne]

Is anyone interested in a macro that could automatically transform an iterator 
into the factory proc version recommended in [the 
manual](https://nim-lang.github.io/Nim/manual.html#iterators-and-the-for-statement-firstminusclass-iterators)?

I've been tinkering with one since [this 
discussion](https://forum.nim-lang.org/t/8127) and have come up with something 
that I think is reasonable. It transforms your code into a proc + macro pair 
that allows the transformation to be transparent to the caller. One limitation 
is that you can't use `auto` as a return type (might be a compiler bug, or 
might be by design).

It can transform something like this, which doesn't work currently:
    
    
    iterator recCountDown*(n: int): int =
      if n > 0:
        yield n
        for e in recCountDown(n - 1):
            yield e
    
    
    Run

Into something like this:
    
    
    macro recCountDown*(x: ForLoopStmt): untyped =
      let expr`gensym2 = x`gensym2[0]
      let call`gensym2 = x`gensym2[1]
      let body`gensym2 = x`gensym2[2]
      call`gensym2[0] = ident(":tmp_939524152")
      
      template toItr(expr`gensym2, call`gensym2, body`gensym2) =
        let itr`gensym2 = call`gensym2
        for expr`gensym2 in itr`gensym2():
          body`gensym2
      
      result = getAst(toItr(expr`gensym2, call`gensym2, body`gensym2))
    
    proc :tmp_939524152*(n: int): iterator (): int =
      result = iterator (): int =
        if n > 0:
          yield n
          for e in recCountDown(n - 1):
            yield e
    
    
    Run

and then you can simply call it like a normal iterator
    
    
    for i in recCountDown(100):
      echo i
    # Outputs 100 down to 1
    
    
    Run

And all you have to do to get this functionality is annote your iterator like 
so:
    
    
    iterator recCountDown*(n: int): int {.recursive.} =
      if n > 0:
        yield n
        for e in recCountDown(n - 1):
            yield e
    
    
    Run

The proc generated is random, so there won't be any conflicts.

Here is the macro:
    
    
    import macros
    
    macro recursive*(iterDef: untyped): untyped =
      iterDef.expectKind nnkIteratorDef
      
      let
        body = iterDef.body
        iterDefProto = iterDef.copy()
      
      iterDefProto.body = newEmptyNode()
      
      let
        iterDefName = iterDefProto[0]
        iterRetType = iterDefProto.params[0]
      
      # Create a new proc def and copy everything to it
      let procDef = newNimNode(nnkProcDef)
      for i in 0..<iterDefProto.len:
        procDef.add(iterDef[i])
      
      procDef.params[0] = nnkIteratorTy.newTree(
        nnkFormalParams.newTree(
          iterRetType
        ),
        newEmptyNode()
      )
      
      # Use gensym to generate a random name
      let
        symName = genSym(nskProc)
        procName = ident($symName.toStrLit)
        procStringName = newLit($procName.toStrLit)
      
      let bodyComment = if body[0].kind == nnkCommentStmt: body[0] else: 
nnkCommentStmt.newTree()
      
      procDef.name = procName
      
      let procDefProto = procDef.copy()
      procDefProto.body = newEmptyNode()
      
      procDef.body = quote do:
        result = iterator(): `iterRetType` =
          `body`
      
      result = quote do:
        macro `iterDefName`(x: ForLoopStmt): untyped =
          `bodyComment`
          let expr = x[0]
          let call = x[1]
          let body = x[2]
          call[0] = ident(`procStringName`)
          
          template toItr(expr, call, body) =
            let itr = call
            for expr in itr():
                body
          
          result = getAst(toItr(expr, call, body))
        
        `procDef`
    
    
    Run

Nim playground: <https://play.nim-lang.org/#ix=3qyK>

Advantages:

  * No need to do much extra to your code
  * hides the complexity from the user



Disadvantages:

  * hides the complexity from the user (could be a disadvantage as well)
  * can't use auto return type (not a big deal IMO)
  * goto definition doesn't work, since the definition is hidden in the macro 
ast



What do people think? Should this be added to something like fusion? Could more 
improvements be made?