Re[2]: Refactoring with JDE?
Andrew Hyatt [EMAIL PROTECTED] seems to think that: [ ... ] I would be very interested if someone who really knew semantic well could tell us if it could do the kind of parsing necessary for these more advanced features. It seems to have more then the advertised API, since you can get the speedbar to know about types of local variables, while the advertised API does not go into within-method information. Hi, Semantic 1.4 local variable parsing depends on programmers having some mysterious need to declare all local variables before writing any code. As such, it can use the regular tagging parser within a method body scope and do a reasonable job. That particular function is advertised in the manual under context parsing, not under the general tagging information. As Paul mentioned, Semantic 2.0 includes David Ponce's full bison port, and a Java parser that can parse an entire Java file, including method code, faster than the Semantic 1.4 parser could tag the same file. The new incremental parser handles about 90% of common edits without a full reparse. My long term goals include indentation and font lock engines, underlining bad syntax, and all that good stuff. My long term goals could probably keep several people busy for several decades. When I wrote semantic 1.0 I was learning what a parser is as I wrote it, so the road to semantic 2.0 includes a massive renaming campaign to give all the nouns the correct names and reduce confusion. I wrote the thing and it confused me. Now it does not. Anyway, all that's done. The parser API is an elegant thing to behold. The new tag searching and completion engines bring joy to my heart. The parser construction process will bring new languages into the fold. All the wacky new minor modes will lure the young-uns to the glory of Emacs. Well, something like that. Paul gave me a little too much credit perhaps. Your other question was when?. Here is the inside scoop on Semantic 2.0. Scoop 1: Semantic, EIEIO, Speedbar, EDE, and Corge will all be shipped together as one package called CEDET 1.0 and have one master Makefile, but each will maintain their own version number and identity, and can be installed one chunk at a time. It cuts down on maintenance since they are all co-dependent. Version number dependencies will be enfoced with a new inversion package. Scoop 2: We've been trying to bring the Doc to be at least vaguely relevant to what's in CVS, and get the build harness working to desired parameters (a bit of EDE development there.) That's it before a CEDET 1.0 beta1 is built and published. Note 1: Most major API functions had their names changed. We devised a nice obsoletion mechanism that lets ECB and the JDE work fine with the new version. They just throw lots of warnings during byte-compilation time. (At least I think the JDE works. Isn't that so David?) Note 2: This beta version of semantic will have no indentation or font lock magic in it. The substructure is there, but little attention has been placed on it yet. Note 3: I owe David Ponce a huge debt of gratitude for the amazing things he has done to bring Semantic 2.0 to where it is today. I am also indebted to Richard Kim who has been instrumental to much of the new documentation, and for writing the python parser. Have fun Eric -- Eric Ludlam: [EMAIL PROTECTED], [EMAIL PROTECTED] Home: http://www.ludlam.netSiege: www.siege-engine.com Emacs: http://cedet.sourceforge.net GNU: www.gnu.org
Re: Refactoring with JDE?
Andrew Hyatt writes: [snip] Using a third party Java-based backend might be easiest, but I'm afraid that such a loose integration wouldn't give us many of the cool features we would want. My personal view, that is not shared by many I fear, is that we should do as much as humanly possible in elisp. The more we do in elisp, the faster and better integrated it will be. I share Andy's bias toward elisp. Regarding semantic, my long-range plan, as stated before, is to provide a parse data API that seamlessly (from the point of view of elisp clients) integrates the classpath (based on Java reflection and perhaps Andy's classfile parser) with the source parse database generated by semantic. The idea is the the API would look first in the semantic database for the requested parse info, then possible in Andy's xref database, and then in the classpath. This would allow completion on fields and methods for which only source code exists or for which only class files exist. My hope is that the Java parser generated by David Ponce's new yacc-style parser generator will be fast enough and complete enough to allow on-the-fly expression syntax error highlighting and other advanced features. Paul
RE: Refactoring with JDE?
Where can I find information on how to use jde-xref ? -Original Message- From: Andrew Hyatt [mailto:[EMAIL PROTECTED] Sent: Wednesday, September 03, 2003 10:03 PM To: Nascif Abousalh-Neto Cc: [EMAIL PROTECTED] Subject: Re: Refactoring with JDE? On Wednesday, September 3, 2003, at 07:19 PM, Nascif Abousalh-Neto wrote: Good point. What do you think should be the way to go with support for refactorings in JDE? I can see three major directions: 1) Expanding on the jde-xref functionality 2) Using the new semantic 2.0 - assuming it has all the necessary functionality 3) Use a Java-based backend or C-based, like Transmogrify or xrefactory. I think that using an approach similar to the JDE compile server, which keeps an instance of the java compiler running at all times, the Java approach may be feasible in terms of performance. Then again I don't know how Lisp-based grammars compare in terms of speed and memory usage with Java-based ones. I would think using semantic 2.0 is what we would want. From Paul's earlier email, I think he also is waiting on this. If semantic does not replicate the jde-xref functionality, we can change jde-xref to work on source files rather than class files by using semantic 2.0 to do the parsing. I'm certain it will be slower than what we have now, but it might be worth it. If we don't want to wait for semantic 2.0 we can implement some simple refactorings that use that functionality such as rename method with jde-xref. I had previously worked on such a system for doing refactorings, but ran up against a problem where I wasn't sure whether we should update the xref db based on the new refactoring or not. It's one of the strange issues that arise from us parsing class files instead of source file. Using a third party Java-based backend might be easiest, but I'm afraid that such a loose integration wouldn't give us many of the cool features we would want. My personal view, that is not shared by many I fear, is that we should do as much as humanly possible in elisp. The more we do in elisp, the faster and better integrated it will be.
RE: Refactoring with JDE?
Jayakrishnan Nair writes: Where can I find information on how to use jde-xref ? In the JDEE User's Guide under Cross-Referencing Classes. Paul -Original Message- From: Andrew Hyatt [mailto:[EMAIL PROTECTED] Sent: Wednesday, September 03, 2003 10:03 PM To: Nascif Abousalh-Neto Cc: [EMAIL PROTECTED] Subject: Re: Refactoring with JDE? On Wednesday, September 3, 2003, at 07:19 PM, Nascif Abousalh-Neto wrote: Good point. What do you think should be the way to go with support for refactorings in JDE? I can see three major directions: 1) Expanding on the jde-xref functionality 2) Using the new semantic 2.0 - assuming it has all the necessary functionality 3) Use a Java-based backend or C-based, like Transmogrify or xrefactory. I think that using an approach similar to the JDE compile server, which keeps an instance of the java compiler running at all times, the Java approach may be feasible in terms of performance. Then again I don't know how Lisp-based grammars compare in terms of speed and memory usage with Java-based ones. I would think using semantic 2.0 is what we would want. From Paul's earlier email, I think he also is waiting on this. If semantic does not replicate the jde-xref functionality, we can change jde-xref to work on source files rather than class files by using semantic 2.0 to do the parsing. I'm certain it will be slower than what we have now, but it might be worth it. If we don't want to wait for semantic 2.0 we can implement some simple refactorings that use that functionality such as rename method with jde-xref. I had previously worked on such a system for doing refactorings, but ran up against a problem where I wasn't sure whether we should update the xref db based on the new refactoring or not. It's one of the strange issues that arise from us parsing class files instead of source file. Using a third party Java-based backend might be easiest, but I'm afraid that such a loose integration wouldn't give us many of the cool features we would want. My personal view, that is not shared by many I fear, is that we should do as much as humanly possible in elisp. The more we do in elisp, the faster and better integrated it will be.
Re: Refactoring with JDE?
Andrew Hyatt writes: Nascif Abousalh-Neto [EMAIL PROTECTED] writes: I agree with you that it would be really nice to have this feature... it takes a good Java parser and a very efficient way to cache/access the syntatic trees associated with the source code. It seems that some of that info is already available in different bits and pieces (the semantic database, the new cross-reference functionality) but for refactoring you do need a lot of information, more than just function names and headers. Which means lots of processing time and memory. Regards, Nascif PS: As for the Ctrl-1 functionality, this is certainly much easier to I would be very interested if someone who really knew semantic well could tell us if it could do the kind of parsing necessary for these more advanced features. It seems to have more then the advertised API, since you can get the speedbar to know about types of local variables, while the advertised API does not go into within-method information. Hi Andy, Yes, the current version of semantic has the ability to parse local variables. The upcoming Semantic 2.0 will have a full-blown, yacc-style parser generator, developed by David Ponce, that should support any feature the JDEE needs. Indeed, it should have a revolutionary impact on other Emacs applications, allowing much faster and more accurate parsing that is possible with the regular-expression based parsers used, e.g., for syntax coloring and indentation. I am eagerly awaiting the release of Semantic 2.0. - Paul
Re: Refactoring with JDE?
Paul Kinnucan [EMAIL PROTECTED] writes: Hi Andy, Yes, the current version of semantic has the ability to parse local variables. The upcoming Semantic 2.0 will have a full-blown, yacc-style parser generator, developed by David Ponce, that should support any feature the JDEE needs. Indeed, it should have a revolutionary impact on other Emacs applications, allowing much faster and more accurate parsing that is possible with the regular-expression based parsers used, e.g., for syntax coloring and indentation. I am eagerly awaiting the release of Semantic 2.0. - Paul Thanks for the info. Is an early version of 2.0 ready to play with? If not, any idea when it would be ready?
RE: Refactoring with JDE?
Hi Andrew, -Original Message- From: Andrew Hyatt [mailto:[EMAIL PROTECTED] Sent: Wednesday, September 03, 2003 5:00 PM To: Abousalh-Neto, Nascif [NCRTP:3X20:EXCH] Cc: [EMAIL PROTECTED] Subject: Re: Refactoring with JDE? Nascif Abousalh-Neto [EMAIL PROTECTED] writes: Hi Andrew, Hmm, I have to disagree here. First I don't think you can beat a refactoring editor in terms of speed if you take into account propagating the changes to all affected modules, specially in a medium-large project (think rename method, or even rename variable for a protected variable used by many subclasses). Since emacs is all about speed, it makes sense to use a specialized tool rather than macros and other tricks if one is available. For things that need to know caller locations, true. But jde-xref already handles that info, it could be extended to some of those refactorings. But many of the common Good point. What do you think should be the way to go with support for refactorings in JDE? I can see three major directions: 1) Expanding on the jde-xref functionality 2) Using the new semantic 2.0 - assuming it has all the necessary functionality 3) Use a Java-based backend or C-based, like Transmogrify or xrefactory. I think that using an approach similar to the JDE compile server, which keeps an instance of the java compiler running at all times, the Java approach may be feasible in terms of performance. Then again I don't know how Lisp-based grammars compare in terms of speed and memory usage with Java-based ones. refactorings such as extract method, pull up method, etc, are easy to do by hand. For the extract method refactoring, consider a block of code that uses local variables. You have to create for each instance of a local variable an equivalent method parameter, and fill the method call with the associated arguments. And so on. A true refactoring tool is very much like a calculator; sure you can make basic calculations by hand and most of the time it is faster this way, but besides the trivial cases you always use the calculator not only for speed, but also so that you don't worry if any of the intermediate steps in your 5-digit multiplication was wrong and can trust the result - OK, maybe I am bad in math... :-) The problem with the more advanced IDE's is that they are SLOW. Eclipse is supposed to me much faster then JBuilder, and Eclipse is just a dog compared to emacs. Eclipse makes doing advanced things easy, and by doing this they slow down normal editing. Personally, I prefer a fast editor above all else. (Yes, I know you can turn off certain features in Eclipse to speed things up, but this eliminates many of the good features of Eclipse). Perhaps there is a way for us to have our cake and eat it too. But I for one haven't thought of how. Agreed. I also choose speed over bells and whistles hands down. But given Emacs power I think it may be possible to have both; it may be more a matter of finding the right balance between speed and functionality. Regards, /Nascif I have been thinking about it as well. I agree it would require a lot of JDE changes to make things more consistent, but is that necessarily a bad thing? I don't see why JDE would lose flexibility, and actually I think it would have two benefits: 1) It could make JDE more efficient by doing a very core thing - keeping the structure of the code in a usable format - in exactly one place. Granted it is a complex and expensive operatation, which is exactly why it should be done once and re-used by all other modules. 2) It would allow for more powerful extensions/plug-ins because the hard work of gathering semantic info from the code would already have been done; designers would only have to learn the API to get them; 3) Not only refactorings could be done but all kinds of operations can benefit from a better model of the code. For example, the try-catch replacement we have talked about in this list could wrap the code using the exact exceptions; searches could filter on exact characteristics (like search method call); and so on. It's not the flexibility of JDE I am concerned about, it's the flexibility of users. It somehow feels wrong if JDE would require a lock-in in the same way Eclipse (and probably other JDE's do). I agree with you to an extent about the usefulness of centralizing semantic info, but I'm not sure it should be done ahead of time. If it could be done as needed it might be better. Regards, /Nascif
Re: Refactoring with JDE?
On Wednesday, September 3, 2003, at 07:19 PM, Nascif Abousalh-Neto wrote: Good point. What do you think should be the way to go with support for refactorings in JDE? I can see three major directions: 1) Expanding on the jde-xref functionality 2) Using the new semantic 2.0 - assuming it has all the necessary functionality 3) Use a Java-based backend or C-based, like Transmogrify or xrefactory. I think that using an approach similar to the JDE compile server, which keeps an instance of the java compiler running at all times, the Java approach may be feasible in terms of performance. Then again I don't know how Lisp-based grammars compare in terms of speed and memory usage with Java-based ones. I would think using semantic 2.0 is what we would want. From Paul's earlier email, I think he also is waiting on this. If semantic does not replicate the jde-xref functionality, we can change jde-xref to work on source files rather than class files by using semantic 2.0 to do the parsing. I'm certain it will be slower than what we have now, but it might be worth it. If we don't want to wait for semantic 2.0 we can implement some simple refactorings that use that functionality such as rename method with jde-xref. I had previously worked on such a system for doing refactorings, but ran up against a problem where I wasn't sure whether we should update the xref db based on the new refactoring or not. It's one of the strange issues that arise from us parsing class files instead of source file. Using a third party Java-based backend might be easiest, but I'm afraid that such a loose integration wouldn't give us many of the cool features we would want. My personal view, that is not shared by many I fear, is that we should do as much as humanly possible in elisp. The more we do in elisp, the faster and better integrated it will be.
