Re: [netlogo-devel] Procedure call graph

[Steve Upton]

Thanks again, Seth!!

steve

On Monday, June 29, 2020 at 1:30:54 PM UTC-4, Seth Tisue wrote:
>
> On Thu, Jun 25, 2020 at 7:46 AM Steve Upton  > wrote: 
> > BTW, what is __ignore? I don't see it in the Netlogo dictionary. 
>
> see https://github.com/NetLogo/NetLogo/wiki/Unofficial-features 
>
> Seth 
>

-- 
You received this message because you are subscribed to the Google Groups 
"netlogo-devel" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to netlogo-devel+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/netlogo-devel/aabd1633-dea3-460c-9380-590ca320886fo%40googlegroups.com.

Re: [netlogo-devel] Procedure call graph

[Seth Tisue]

On Thu, Jun 25, 2020 at 7:46 AM Steve Upton  wrote:
> BTW, what is __ignore? I don't see it in the Netlogo dictionary.

see https://github.com/NetLogo/NetLogo/wiki/Unofficial-features

Seth

-- 
You received this message because you are subscribed to the Google Groups 
"netlogo-devel" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to netlogo-devel+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/netlogo-devel/CAOoO2vgK8pWGteXpnyuESo4LB-2Opx7ar%2BjEma4PKMYm1eJocw%40mail.gmail.com.

Re: [netlogo-devel] Procedure call graph

[Steve Upton]

Seth,

Thank you for following up! Learning something new about Netlogo and Scala 
everyday!  

You are correct - I am only looking at the first level of the tree. I had 
assumed I only needed to go one level to capture all the procedures that 
called a set of procedures within its code block, and that all procedures 
would be included in the workspace.procedures map (and interested in 
user-defined procedures mainly). For your test case (I made a slight mod to 
make an edge from a procedure that didn't make any calls to a "Nothing" 
node), I get:
FOO -> Nothing
BAR -> Nothing
BAZ -> FOO

so clearly missed the call to Bar. 

BTW, what is __ignore? I don't see it in the Netlogo dictionary.

Also, thanks for the nitpick - always good to clean up code.

steve

On Wednesday, June 17, 2020 at 2:00:46 PM UTC-4, Seth Tisue wrote:
>
> This looks it will produce correct output in many cases, but I'm not 
> convinced it's correct in general.
>
> Does it work on a case like this, and on even more deeply nested cases?
>
> to-report foo [x] report x end
> to-report bar [x] report x end
> to baz __ignore foo bar 0 end
>
> I'm looking at your logic and it isn't clear to me that all possible 
> nestings (including recursive nestings) of _call and _callreport are 
> handled properly, hence the test case. I don't see anywhere in your code 
> that recursively processes entire argument trees, and not just the first 
> level of the tree.
>
> Also a nitpick that won't usually affect correctness: you shouldn't need 
> to resort to looking for "_call" and "_callreport' as strings — _call and 
> _callreport are classes, you should be able to detect instances of them via 
> pattern matching (or `isInstanceOf`).
>
> Seth
>

-- 
You received this message because you are subscribed to the Google Groups 
"netlogo-devel" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to netlogo-devel+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/netlogo-devel/7bb92c80-6dd6-4255-98ee-300a25b7b198o%40googlegroups.com.

Re: [netlogo-devel] Procedure call graph

[Seth Tisue]

This looks it will produce correct output in many cases, but I'm not 
convinced it's correct in general.

Does it work on a case like this, and on even more deeply nested cases?

to-report foo [x] report x end
to-report bar [x] report x end
to baz __ignore foo bar 0 end

I'm looking at your logic and it isn't clear to me that all possible 
nestings (including recursive nestings) of _call and _callreport are 
handled properly, hence the test case. I don't see anywhere in your code 
that recursively processes entire argument trees, and not just the first 
level of the tree.

Also a nitpick that won't usually affect correctness: you shouldn't need to 
resort to looking for "_call" and "_callreport' as strings — _call and 
_callreport are classes, you should be able to detect instances of them via 
pattern matching (or `isInstanceOf`).

Seth

-- 
You received this message because you are subscribed to the Google Groups 
"netlogo-devel" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to netlogo-devel+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/netlogo-devel/9d8d89fe-66e8-494f-9786-154217a5b94fn%40googlegroups.com.

Re: [netlogo-devel] Procedure call graph

[Steve Upton]

In case anyone is interested, I came up with this simple code to generate 
the call graph:
import org.nlogo.headless.HeadlessWorkspace
import org.nlogo.nvm._

case class GraphEdge(from : String, to: String, isolate : Boolean) {
override def toString = s"$from -> $to"
def toCSV = s"$from,$to,$isolate"
}

object GraphEdge {
def apply(from: String, to: String) : GraphEdge = {
val isolate = to match {
case "" => true
case _ => false
}
apply(from, to, isolate)
}

}

 object GenerateCallGraph {
  def extractCallGraph(procedures : Procedure.ProceduresMap) : 
Set[GraphEdge] = {
val procNames = procedures.keySet
println("printing the procNames")
procNames foreach println
println("***printing the calls")
val edges = procedures.map{case(parent,child) => {
println(s"working procedure: $parent")
val (gcall,gargs) = child.code.partition(_.toString.contains("_call"))
val topEdges = gcall.map(c => GraphEdge(from=parent,to=c.displayName))
// greports : Array[Array[org.nlogo.nvm.Reporter]] 
val greports = gargs.map(_.args).filter(a => 
a.exists(_.toString.contains("_callreport")))
val argEdges = greports.map(r => {
val edge = GraphEdge(from=parent, to=r(0).displayName)
println(s"making edge in proc: $parent => $edge")
edge
})
argEdges match {
case e if e.isEmpty => topEdges.toList
case _ => topEdges.toList ::: argEdges.toList
}
}}  
val edgeSet = edges.toList.flatten.toSet
edgeSet foreach println
edgeSet

  }

def run(modelFileName : String, callGraphOutFileName : String) : Unit = {
val workspace = HeadlessWorkspace.newInstance
workspace.open(modelFileName)
val procs = workspace.procedures 
val callGraph = extractCallGraph(procs)
val output = callGraph.map(_.toCSV).toList
Utils.writeFile(callGraphOutFileName, output)
workspace.dispose()
}


} 

On Thursday, June 11, 2020 at 11:41:03 AM UTC-4, Steve Upton wrote:
>
> Shoot, has it been that long??
>
> Your answers, and references, are enough to get me going. I'll parse the 
> Commands since I don't see where to get the FrontEndResults - maybe they 
> have been discarded. Or maybe someone else will pipe in.
>
> thanx again
> steve
>
> On Thursday, June 11, 2020 at 11:20:30 AM UTC-4, Seth Tisue wrote:
>>
>> It's now been more than five years since I worked on NetLogo; there's a 
>> limit to how much I still remember.
>>
>> In particular, I don't remember whether it's possible to get back to the 
>> ASTs after the entire compiler has run, or whether they have already been 
>> discarded and it's too late.
>>
>> I found some prior discussions in this area in the NetLogo room on Gitter:
>>
>> * https://gitter.im/NetLogo/NetLogo?at=5be1a45dda57ff676c9d4305  — "you 
>> want `FrontEndResults` and not `CompilerResults`. "FrontEndResults contains 
>> ProcedureDefinition objects, these have the ASTs inside them. see 
>> AstNode.scala"
>> * https://gitter.im/NetLogo/NetLogo?at=5bba30ccc7bf7c3662cda279
>>
>> But anyway, although my first thought was to walk the ASTs, on second 
>> thought it might be easier and just as effective for your use case to walk 
>> the fully compiled model (the `Array[Command]` inside each procedure) and 
>> look for the `_call` and `_callreport` nodes there. Any `_call`s will be in 
>> the `Array[Command]` directly, but to find the `_callreport`s you'll need 
>> recursively walk the `args` tree under each `Command`.
>>
>> Seth
>>
>

-- 
You received this message because you are subscribed to the Google Groups 
"netlogo-devel" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to netlogo-devel+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/netlogo-devel/af675a50-601e-4a20-8d54-4deb5338c211o%40googlegroups.com.

Re: [netlogo-devel] Procedure call graph

[Steve Upton]

Shoot, has it been that long??

Your answers, and references, are enough to get me going. I'll parse the 
Commands since I don't see where to get the FrontEndResults - maybe they 
have been discarded. Or maybe someone else will pipe in.

thanx again
steve

On Thursday, June 11, 2020 at 11:20:30 AM UTC-4, Seth Tisue wrote:
>
> It's now been more than five years since I worked on NetLogo; there's a 
> limit to how much I still remember.
>
> In particular, I don't remember whether it's possible to get back to the 
> ASTs after the entire compiler has run, or whether they have already been 
> discarded and it's too late.
>
> I found some prior discussions in this area in the NetLogo room on Gitter:
>
> * https://gitter.im/NetLogo/NetLogo?at=5be1a45dda57ff676c9d4305  — "you 
> want `FrontEndResults` and not `CompilerResults`. "FrontEndResults contains 
> ProcedureDefinition objects, these have the ASTs inside them. see 
> AstNode.scala"
> * https://gitter.im/NetLogo/NetLogo?at=5bba30ccc7bf7c3662cda279
>
> But anyway, although my first thought was to walk the ASTs, on second 
> thought it might be easier and just as effective for your use case to walk 
> the fully compiled model (the `Array[Command]` inside each procedure) and 
> look for the `_call` and `_callreport` nodes there. Any `_call`s will be in 
> the `Array[Command]` directly, but to find the `_callreport`s you'll need 
> recursively walk the `args` tree under each `Command`.
>
> Seth
>

-- 
You received this message because you are subscribed to the Google Groups 
"netlogo-devel" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to netlogo-devel+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/netlogo-devel/e5c504e1-ca20-43b0-a0a4-488ec5f6f633o%40googlegroups.com.

Re: [netlogo-devel] Procedure call graph

[Seth Tisue]

It's now been more than five years since I worked on NetLogo; there's a 
limit to how much I still remember.

In particular, I don't remember whether it's possible to get back to the 
ASTs after the entire compiler has run, or whether they have already been 
discarded and it's too late.

I found some prior discussions in this area in the NetLogo room on Gitter:

* https://gitter.im/NetLogo/NetLogo?at=5be1a45dda57ff676c9d4305  — "you 
want `FrontEndResults` and not `CompilerResults`. "FrontEndResults contains 
ProcedureDefinition objects, these have the ASTs inside them. see 
AstNode.scala"
* https://gitter.im/NetLogo/NetLogo?at=5bba30ccc7bf7c3662cda279

But anyway, although my first thought was to walk the ASTs, on second 
thought it might be easier and just as effective for your use case to walk 
the fully compiled model (the `Array[Command]` inside each procedure) and 
look for the `_call` and `_callreport` nodes there. Any `_call`s will be in 
the `Array[Command]` directly, but to find the `_callreport`s you'll need 
recursively walk the `args` tree under each `Command`.

Seth

-- 
You received this message because you are subscribed to the Google Groups 
"netlogo-devel" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to netlogo-devel+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/netlogo-devel/46c6d304-5223-44ff-a512-bda439e19977n%40googlegroups.com.

Re: [netlogo-devel] Procedure call graph

[Steve Upton]

Maybe a teensy bit more help :-)

I found this post "analysis/visualization of internal representations?" but 
still not grokking how to get the FrontEnd.

object ExtractProcedures2 {
  def extract(filename : String) : Unit = {
val workspace = HeadlessWorkspace.newInstance
workspace.open("models/Fire.nlogo")
val frontend : FrontEndInterface = workspace.compiler.frontEnd

/* val procNames = procs.keySet
println("printing the procNames")
procNames foreach println
println("***printing the calls")
procs.foreach{case(k,p) => {
println(s"working procedure: $k")
p.code foreach println
}}  */ 

workspace.dispose()
  }
}

but the FrontEndInterface isn't getting me the AST (i'm sure I'm missing 
something simple).

steve


On Wednesday, June 10, 2020 at 10:28:16 PM UTC-4, Seth Tisue wrote:
>
> I think you will need to walk the ASTs representing the procedure bodies 
> and look for _call and _callreport nodes.
>
> Seth
>

-- 
You received this message because you are subscribed to the Google Groups 
"netlogo-devel" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to netlogo-devel+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/netlogo-devel/43fb1917-cc45-4c27-919c-9f39501f245fo%40googlegroups.com.

Re: [netlogo-devel] Procedure call graph

[Seth Tisue]

I think you will need to walk the ASTs representing the procedure bodies
and look for _call and _callreport nodes.

Seth

-- 
You received this message because you are subscribed to the Google Groups 
"netlogo-devel" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to netlogo-devel+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/netlogo-devel/CAOoO2vgQfVZa9Fjm4M_t_-VP5vcEzOMP1zqjsH28fgw2Xak6sg%40mail.gmail.com.

Re: [netlogo-devel] Procedure call graph

[Steve Upton]

Thank you for the pointer. Hadn't looked at that yet. The profiler 
extension is a start, but it only gives me the calls to the procedures, not 
what other procedures called that procedure, etc., in a graph. From your 
pointer, I found the Reflection-Extension, which looked like it produce 
what I want, but if I'm reading it correctly, I'd have to put calls to 
"reflection:procedure" and "reflection:callers" in each procedure. I was 
hoping for something more like a static analysis of the netlogo code.

Seth Tissue had posted code back in 2012 that got me started, and I had 
been looking at the procedures map that is returned. I can get a dump but 
don't see how I can just get the children's names (i.e., the callers). With 
the dump,  I can at least parse that to get what I'm looking for.

thanx
steve

On Tuesday, June 9, 2020 at 8:45:40 PM UTC-4, Forrest wrote:
>
> You might take a look at the profiler extension (or source code thereof), 
> if you haven't already...
>
> https://github.com/NetLogo/Profiler-Extension
>
> Cheers,
>
> Dr. Forrest Stonedahl
> Computer Science
> Augustana College
>
> On Tue, Jun 9, 2020, 11:39 AM Steve Upton > 
> wrote:
>
>> Hi All,
>>
>> Fairly new to Netlogo. We have a good sized model and would like to 
>> create a procedure call graph, i.e., for each procedure, identify what 
>> procedures it calls, and so forth. I've found a post on reflection and can 
>> get the procedures (as well as globals, turtles, breeds), so about to 
>> explore further, but wondered if this wasn't already available.
>>
>> thank you
>> steve upton
>>
>> -- 
>> You received this message because you are subscribed to the Google Groups 
>> "netlogo-devel" group.
>> To unsubscribe from this group and stop receiving emails from it, send an 
>> email to netlog...@googlegroups.com .
>> To view this discussion on the web visit 
>> https://groups.google.com/d/msgid/netlogo-devel/4cbab5a6-b52a-48d5-87fa-e272397c4e7bo%40googlegroups.com
>>  
>> 
>> .
>>
>

-- 
You received this message because you are subscribed to the Google Groups 
"netlogo-devel" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to netlogo-devel+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/netlogo-devel/e35605f1-2fa9-488c-9675-fa540073d462o%40googlegroups.com.

Re: [netlogo-devel] Procedure call graph

[Forrest Stonedahl]

You might take a look at the profiler extension (or source code thereof),
if you haven't already...

https://github.com/NetLogo/Profiler-Extension

Cheers,

Dr. Forrest Stonedahl
Computer Science
Augustana College

On Tue, Jun 9, 2020, 11:39 AM Steve Upton  wrote:

> Hi All,
>
> Fairly new to Netlogo. We have a good sized model and would like to create
> a procedure call graph, i.e., for each procedure, identify what procedures
> it calls, and so forth. I've found a post on reflection and can get the
> procedures (as well as globals, turtles, breeds), so about to explore
> further, but wondered if this wasn't already available.
>
> thank you
> steve upton
>
> --
> You received this message because you are subscribed to the Google Groups
> "netlogo-devel" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to netlogo-devel+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/netlogo-devel/4cbab5a6-b52a-48d5-87fa-e272397c4e7bo%40googlegroups.com
> 
> .
>

-- 
You received this message because you are subscribed to the Google Groups 
"netlogo-devel" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to netlogo-devel+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/netlogo-devel/CAG_mutPoXYOAdiZUr3-WAi9Bp%3DQy6MXp5r0fxju5jTtUUC_QLg%40mail.gmail.com.