Re: [GSOC 2013] Program analysis suite, based on Rich Hickey's Codeq
Hi Ambrose, Agreed, using CinC to analyze would be the best way. However given the wish list [1], I do not feel its suitable for me. Btw, I meant extending Codeq's analyzer rather than rewriting jvm.tools.analyzer. [1]: http://dev.clojure.org/display/design/Compiler+in+Clojure Cheers Navgeet -- -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups Clojure group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out.
Re: [GSOC 2013] Program analysis suite, based on Rich Hickey's Codeq
Hi Navgeet, Yes CinC is a big job. Ah, I read the proposal more closely. I think you're on the right track. You should only need one more AST node for unexpanded macros, or maybe even just extra metadata on the :invoke :op. As a quick and dirty approach I like it. Thanks, Ambrose On Wed, May 1, 2013 at 3:29 PM, Navgeet Agrawal navgeet.agra...@gmail.comwrote: Hi Ambrose, Agreed, using CinC to analyze would be the best way. However given the wish list [1], I do not feel its suitable for me. Btw, I meant extending Codeq's analyzer rather than rewriting jvm.tools.analyzer. [1]: http://dev.clojure.org/display/design/Compiler+in+Clojure Cheers Navgeet -- -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups Clojure group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out. -- -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups Clojure group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out.
Re: [GSOC 2013] Program analysis suite, based on Rich Hickey's Codeq
I'm not sure how useful unexpanding analyzer will actually be useful for Codeq, especially if you want to do analysis like who calls this function from it. Forms passed to a macro aren't necessarily going to be useful as unexpanded forms. eg. a macro that rewrites (+ 1 2) to (plus 1 2) might count as a call to clojure.core/+. Have you considered these issues? Will you have a tolerance for false positives? Thanks, Ambrose On Wed, May 1, 2013 at 3:47 PM, Ambrose Bonnaire-Sergeant abonnaireserge...@gmail.com wrote: Hi Navgeet, Yes CinC is a big job. Ah, I read the proposal more closely. I think you're on the right track. You should only need one more AST node for unexpanded macros, or maybe even just extra metadata on the :invoke :op. As a quick and dirty approach I like it. Thanks, Ambrose On Wed, May 1, 2013 at 3:29 PM, Navgeet Agrawal navgeet.agra...@gmail.com wrote: Hi Ambrose, Agreed, using CinC to analyze would be the best way. However given the wish list [1], I do not feel its suitable for me. Btw, I meant extending Codeq's analyzer rather than rewriting jvm.tools.analyzer. [1]: http://dev.clojure.org/display/design/Compiler+in+Clojure Cheers Navgeet -- -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups Clojure group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out. -- -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups Clojure group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out.
Re: [GSOC 2013] Program analysis suite, based on Rich Hickey's Codeq
If I understand correctly, a macro invocation of the form `(expand '(+ 1 2)) = (plus 1 2)` will be analyzed as invoking the macro 'expand' on the list (+ 1 2). The list is never analyzed to be a call to clojure.core/+. As far as analysis like who calls this function are concerned, I think its best to limit them to function invocations, those not from macro expansions. That said, I think versioned analysis of macro expansions would be useful. I will add it to the gist. Cheers Navgeet -- -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups Clojure group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out.
Re: [GSOC 2013] Program analysis suite, based on Rich Hickey's Codeq
What about calls to `let`, `defn`, `loop`, `cond` and `fn`, which are all macros? Thanks, Ambrose On Wed, May 1, 2013 at 4:34 PM, Navgeet Agrawal navgeet.agra...@gmail.comwrote: If I understand correctly, a macro invocation of the form `(expand '(+ 1 2)) = (plus 1 2)` will be analyzed as invoking the macro 'expand' on the list (+ 1 2). The list is never analyzed to be a call to clojure.core/+. As far as analysis like who calls this function are concerned, I think its best to limit them to function invocations, those not from macro expansions. That said, I think versioned analysis of macro expansions would be useful. I will add it to the gist. Cheers Navgeet -- -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups Clojure group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out. -- -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups Clojure group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out.
Re: [GSOC 2013] Program analysis suite, based on Rich Hickey's Codeq
Ah seems like I did not think that through. For macros it makes sense to do analysis on their expansions. For example, for analyzing a defn form, codeq should examine the expansion's ast. Since I wanted to build all analysis functionality as plugins, therefore source analysis plugins should have access to a defn's full expanded ast, which would adequately allow them to answer questions like Which functions are called by this function?. This should also be able to deal with false positives, since analysis is run on expanded code. Thanks for the feedback, Navgeet -- -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups Clojure group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out.
Re: [GSOC 2013] Program analysis suite, based on Rich Hickey's Codeq
As a user of Codeq with Navgeet's proposed additions, I would definitely want to see macro usages along with runtime var references. So you're right that you'd want the result of analyzing the subject code with all macros expanded, but it would also be valuable to see the macros that got expanded along the way. On May 1, 2013 5:53 AM, Navgeet Agrawal navgeet.agra...@gmail.com wrote: Ah seems like I did not think that through. For macros it makes sense to do analysis on their expansions. For example, for analyzing a defn form, codeq should examine the expansion's ast. Since I wanted to build all analysis functionality as plugins, therefore source analysis plugins should have access to a defn's full expanded ast, which would adequately allow them to answer questions like Which functions are called by this function?. This should also be able to deal with false positives, since analysis is run on expanded code. Thanks for the feedback, Navgeet -- -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups Clojure group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out. -- -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups Clojure group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out.
Re: [GSOC 2013] Program analysis suite, based on Rich Hickey's Codeq
That's the benefit of a custom analyzer, its possible to control the process. Its best to harvest both the expanded ast and the macro invocation ast, since both are useful. Cheers Navgeet -- Navgeet Agrawal -- -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups Clojure group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out.
Re: [GSOC 2013] Program analysis suite, based on Rich Hickey's Codeq
Hi all, I've started with the application, though it has gotten quite late. I've explained what I intend to do in this gist - https://gist.github.com/Navgeet/5490202 I'm kinda stuck in the last part - analysis techniques. I would appreciate suggestions that are not too heavy to deal with, given my tight schedule in the later half of coding period. Cheers Navgeet -- -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups Clojure group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out.
Re: [GSOC 2013] Program analysis suite, based on Rich Hickey's Codeq
Hi Navgeet, I'm not sure if rewriting jvm.tools.analyzer is a good idea, given that there is CinC project for this year's GSoC. Perhaps you could contribute to the CinC analyzer, and then follow with integration with Codeq? Thanks, Ambrose On Wed, May 1, 2013 at 1:45 AM, Navgeet Agrawal navgeet.agra...@gmail.comwrote: Hi all, I've started with the application, though it has gotten quite late. I've explained what I intend to do in this gist - https://gist.github.com/Navgeet/5490202 I'm kinda stuck in the last part - analysis techniques. I would appreciate suggestions that are not too heavy to deal with, given my tight schedule in the later half of coding period. Cheers Navgeet -- -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups Clojure group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out. -- -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups Clojure group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out.
Re: [GSOC 2013] Program analysis suite, based on Rich Hickey's Codeq
Hi all, thanks for these suggestions and sorry for the late reply. I have been looking into jvm.tools.analyze for the past few days and it looks as a better choice to build a comprehensive analyzer. Of course the biggest problem in using it is it likes to expand all macros fully, which is bad for codeq. I think that can be handled by patching some parts of Compiler.java and using the patched methods. I am currently trying to write a demonstration fork of jvm.tools.analyze. I have been thinking for some time that tracking the inputs and outputs through every function in a program over time can be useful. We could allow the user to call some function with test input, then for all functions down the call chain, store their input and output in the datomic database. Do this for every commit (every time the function changes). The results can provide feedback about data flow in the program across commits. Any comments? That's all for now till my exams end. -- Navgeet -- -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups Clojure group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out.
Re: [GSOC 2013] Program analysis suite, based on Rich Hickey's Codeq
On Apr 16, 2013, at 04:54, Navgeet Agrawal wrote: Hi all, thanks for these suggestions and sorry for the late reply. No problem; I'm happy that we didn't scare you away (:-). I have been looking into jvm.tools.analyze for the past few days and it looks as a better choice to build a comprehensive analyzer. Of course the biggest problem in using it is it likes to expand all macros fully, which is bad for codeq. I think that can be handled by patching some parts of Compiler.java and using the patched methods. I am currently trying to write a demonstration fork of jvm.tools.analyze. From a documentation point of view, macros have two operational contexts. In one, they generate code; in the other, the generated code is run. I'm not precisely sure how this should be handled, but it _is_ interesting. I have been thinking for some time that tracking the inputs and outputs through every function in a program over time can be useful. We could allow the user to call some function with test input, then for all functions down the call chain, store their input and output in the datomic database. Do this for every commit (every time the function changes). The results can provide feedback about data flow in the program across commits. Any comments? One use case for this sort of tracking would be to collect signatures for the inputs and outputs seen in the test suite. If a production run generates diverging signatures, either the test suite is incomplete or the production code is using the function in an unexpected fashion. That's all for now till my exams end. Good luck in the exams! -r -- http://www.cfcl.com/rdmRich Morin http://www.cfcl.com/rdm/resume r...@cfcl.com http://www.cfcl.com/rdm/weblog +1 650-873-7841 Software system design, development, and documentation -- -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups Clojure group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out.
Re: [GSOC 2013] Program analysis suite, based on Rich Hickey's Codeq
One thing that is very useful is an answer to the question Who calls
function X?.
A literate program using Latex can generate this. By adding
\calls{me}{somebody}
at the head of each function it adds two entries in the index,
one of the form
me
calls somebody
and one of the form
somebody
called by me
This inverse lookup is especially useful for code factoring.
Tim Daly
--
--
You received this message because you are subscribed to the Google
Groups Clojure group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
---
You received this message because you are subscribed to the Google Groups
Clojure group.
To unsubscribe from this group and stop receiving emails from it, send an email
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
Re: [GSOC 2013] Program analysis suite, based on Rich Hickey's Codeq
On Apr 13, 2013, at 08:24, u1204 wrote: One thing that is very useful is an answer to the question Who calls function X?. ... This inverse lookup is especially useful for code factoring. Indeed. A developer working on a function X might want to know the names of all other functions that call it. Alternatively, it can be useful to see examples of a function's usage, possibly in proximity to other functions (eg, Who calls functions X and Y). A literate program using Latex can generate this. By adding ... Although it's possible to add annotations (in whatever format) to a code header, this is problematic for several reasons. Annotations: - require time and effort to research and add - introduce visual noise to the code base - may be erroneous, incomplete, outdated, etc. So, it's better to limit annotations to things that a program (eg, Codeq) can't determine. For example, This function uses Greeble's algorithm to calculate the rotation of the lunar wainshaft. Many useful annotations have to do with connascence: In software engineering, two components are connascent if a change in one would require the other to be modified in order to maintain the overall correctness of the system. -- http://en.wikipedia.org/wiki/Connascence_(computer_programming) Jim Weirich has given some great talks on this topic, including: Grand Unified Theory of Software Design - Jim Weirich http://vimeo.com/10837903 Getting back to the immediate question, determining who calls whom can be a problem in dynamic languages such as Clojure and Ruby. So, the programmer may need to provide hints. Alternatively, it may be possible to get this information by monitoring program operation in a testing or limited production environment. -r -- http://www.cfcl.com/rdmRich Morin http://www.cfcl.com/rdm/resume r...@cfcl.com http://www.cfcl.com/rdm/weblog +1 650-873-7841 Software system design, development, and documentation -- -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups Clojure group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out.
Re: [GSOC 2013] Program analysis suite, based on Rich Hickey's Codeq
Getting back to the immediate question, determining who calls whom can be a problem in dynamic languages such as Clojure and Ruby. So, the programmer may need to provide hints. Alternatively, it may be possible to get this information by monitoring program operation in a testing or limited production environment. In my experience, there are 2 cases. Either the call is explicit or the call is constructed (e.g. funcall). In the explicit case, it is easy to write a program to read code. Code walkers exist for common lisp. I'm not sure if there is a code walker for Clojure (does one exist? If not, is this a GSOC?) Code walkers can macro expand code to find embedded calls. In the constructed case the architecture needs documentation. I'm advocating literate programming because this is obviously the way to do that. The cross-references come for free. Axiom (common lisp) uses a parser driven by alist on symbols (maps, in Clojure speak). Thus it is not obvious where a parser function is called. Indeed, it is not obvious that it is ever called as the symbol name for the parser function is constructed at the call site. Using grep on the source code will find nothing. The developer would assume it is dead code since there is no reference to it. Documenting the constructed case requires hand labor. Augmenting the document index with \calls macros adds information that the author knows and a developer needs to maintain the code at little cost of time and effort. Actually I think Clojure adds a third case because it uses Java libraries. This requires an explanation as well as explicit markup (hand labor) to provide the information a developer needs. Thus, 2 of the 3 cases involve hand labor to add valuable information a developer needs to maintain the code. There ought to be a way to write this down. Indeed, it could be written down in a form that the program analysis suite can use as advice. Tim Daly -- -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups Clojure group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out.
[GSOC 2013] Program analysis suite, based on Rich Hickey's Codeq
Hi all, I came up across the project idea for a Program analysis suite based on Codeq [1] a week ago and have been working on it since. The idea appealed to me instantly, since I had just started to look into program analysis possibilities, and analysing a (or many) repository's version history is a novel approach in itself. I have already gone through Datomic, codeq and codeq-playground [2], and I fully understand codeq's schema and the current clojure analyzer. Regarding my experience with clojure, I have been reading through Deuce's [3] source code since a month or two, and I have some experience with Ruby aside from the usual C and C++. I am currently going through all of the pages under [4] and there are a lot of ideas suggested there. I am not sure how many of those ideas are suitable for implementation in a program analysis suite, since many of them are about harvesting documentation and metadata from a repository. As far as the current clojure analyzer is concerned, I understand that all it does is to harvest the names and sources of all `def` and `ns` forms. While it is possible to analyze furthur and create a tree of codeqs, with leaves consisting of symbols, I am not sure what leverage it provides against getting a diff of a function between two commits. Thanks, looking forward to your suggestions. [1]: http://dev.clojure.org/display/community/Project+Ideas#ProjectIdeas-ProgramanalysissuitebasedonRichHickeysCodeq [2]: https://github.com/devn/codeq-playground/blob/master/src/com/thinkslate/codeq_playground/core.clj [3]: https://github.com/hraberg/deuce [4]: http://wiki.cfcl.com/bin/view/Projects/Codeq Navgeet Agrawal https://github.com/Navgeet -- -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups Clojure group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out.
Re: [GSOC 2013] Program analysis suite, based on Rich Hickey's Codeq
Navgeet- I'm delighted to read of your interest. Please see my responses, below. -r On Apr 12, 2013, at 09:49, Navgeet Agrawal wrote: Hi all, I came up across the project idea for a Program analysis suite based on Codeq [1] a week ago and have been working on it since. The idea appealed to me instantly, since I had just started to look into program analysis possibilities, and analysing a (or many) repository's version history is a novel approach in itself. Codeq extends Git's universe in two directions: - The notion of a codeq allows Git's version history to be interpreted in terms of semantically interesting units of code. - The ability to harvest information from multiple repos allows integration of information from local projects, libraries, etc. Combining these, it becomes possible to answer questions about the history of an entire project, down to the level of (say) functions. I am currently going through all of the pages under [4] and there are a lot of ideas suggested there. I am not sure how many of those ideas are suitable for implementation in a program analysis suite, since many of them are about harvesting documentation and metadata from a repository. I may have a broader view of program analysis than you do. Mine covers a range of inputs, including commit history, source code, operational data, and advice from humans. Basically, any source of information is worthy of consideration. However, many of these inputs are far beyond the scope of a GSoC 2013 effort. As far as the current clojure analyzer is concerned, I understand that all it does is to harvest the names and sources of all `def` and `ns` forms. While it is possible to analyze furthur and create a tree of codeqs, with leaves consisting of symbols, I am not sure what leverage it provides against getting a diff of a function between two commits. Diffing functions over time is useful, but it is only one of many tasks Codeq should be able to perform. The current Clojure analyzer does not harvest enough information to resolve questions about related sets of functions, use of global state, etc. An improved analyzer would provide leverage to Codeq as a whole. -- http://www.cfcl.com/rdmRich Morin http://www.cfcl.com/rdm/resume r...@cfcl.com http://www.cfcl.com/rdm/weblog +1 650-873-7841 Software system design, development, and documentation -- -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups Clojure group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out.
Re: [GSOC 2013] Program analysis suite, based on Rich Hickey's Codeq
Hi Navgeet, I'm glad you found codeq-playground helpful. It looks like you've found Rich Morin's wiki. Did you also see his codeq-cookbook project? When you said analysis I had something else in mind: check out https://github.com/clojure/jvm.tools.analyzer I wonder about how that could be used to add more queryable information during analysis. At the very least it might spawn new questions to answer. I'd also suggest checking out things like kibit. I have no idea where that will take you, but I'd be surprised if there aren't some interesting applications for core.logic. Clojail is the last one I'll mention. Could some kind of automated runtime evaluation based on analysis yield some interesting results? Just brainstorming here, but best wishes. Devn — Sent via Mobile On Fri, Apr 12, 2013 at 3:41 PM, Navgeet Agrawal navgeet.agra...@gmail.com wrote: Hi all, I came up across the project idea for a Program analysis suite based on Codeq [1] a week ago and have been working on it since. The idea appealed to me instantly, since I had just started to look into program analysis possibilities, and analysing a (or many) repository's version history is a novel approach in itself. I have already gone through Datomic, codeq and codeq-playground [2], and I fully understand codeq's schema and the current clojure analyzer. Regarding my experience with clojure, I have been reading through Deuce's [3] source code since a month or two, and I have some experience with Ruby aside from the usual C and C++. I am currently going through all of the pages under [4] and there are a lot of ideas suggested there. I am not sure how many of those ideas are suitable for implementation in a program analysis suite, since many of them are about harvesting documentation and metadata from a repository. As far as the current clojure analyzer is concerned, I understand that all it does is to harvest the names and sources of all `def` and `ns` forms. While it is possible to analyze furthur and create a tree of codeqs, with leaves consisting of symbols, I am not sure what leverage it provides against getting a diff of a function between two commits. Thanks, looking forward to your suggestions. [1]: http://dev.clojure.org/display/community/Project+Ideas#ProjectIdeas-ProgramanalysissuitebasedonRichHickeysCodeq [2]: https://github.com/devn/codeq-playground/blob/master/src/com/thinkslate/codeq_playground/core.clj [3]: https://github.com/hraberg/deuce [4]: http://wiki.cfcl.com/bin/view/Projects/Codeq Navgeet Agrawal https://github.com/Navgeet -- -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups Clojure group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out. -- -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups Clojure group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out.
