Re: Perl6 and support for Refactoring IDE's

2005-06-03 Thread J Matisse Enzer


On May 26, 2005, at 10:03 AM, Piers Cawley wrote:


Stevan Little [EMAIL PROTECTED] writes:

... The way I
see it working is that the language itself has a bunch of minimal 
hooks that
get triggered by various phases of compilation etc. Your editor then 
becomes
something that instruments the compiler in such a way that loading a 
file for
editing compiles it, but the compilation process hangs populates the 
data

structures you need for editing.


Sorry to have dropped out of this thread for so long. I'm glad to see 
such smart people thinking about this too.


I like what Piers is suggesting, not because I am in any way qualified 
to say how practical it is, but because the idea he presents really 
would make it possible to have much better tools for Perl.


The idea that the language will allow an editor (an IDE) to really find 
out the knowable information about the source code is the crucial 
thing.


Frankly, there are huge improvements in tools possible for Perl5, just 
by accepting some limitations - we probably cannot duplicate the 
refactoring support available for Java in Perl5, but we can do one heck 
of a lot better than just extract subroutine (which is the *only* 
refactoring available now in a Perl IDE.)


We are still not even at the simple things should be easy stage of a 
refactoring IDE for Perl, and I'd be happy to at least get there before 
worrying about the hard things are possible stage :-)


Once I saw how cool the features in something like Eclipse are for 
another language it really has made me see how far behind Perl is in 
this area. I'm really scared that Perl will be an unacceptable choice 
for larger projects in less than a two years.


-Matisse

---
Matisse Enzer [EMAIL PROTECTED]
http://www.matisse.net/  - http://www.eigenstate.net/
415-225-6703 (work/cellphone)
415-401-8325 (home)




Re: Perl6 and support for Refactoring IDE's

2005-05-26 Thread Piers Cawley
Stevan Little [EMAIL PROTECTED] writes:

 On May 25, 2005, at 5:39 AM, Piers Cawley wrote:
 One of the 'mental apps' that's been pushing some of the things I've been
 asking for in Perl 6's introspection system is a combined
 refactoring/debugging/editing environment for the language.

 Maybe I have been reading too much about Smalltalk meta-classes lately, but in
 doing some draft ideas of the Perl6 meta-model for Pugs, I realized that given
 a really solid class/metaclass introspection system and access to the
 interpreter level meta-meta-classes, it would be possible to easily write a
 class browser much like Smalltalk.

Yes. The application in my head looks very a Smalltalk environment. The way I
see it working is that the language itself has a bunch of minimal hooks that
get triggered by various phases of compilation etc. Your editor then becomes
something that instruments the compiler in such a way that loading a file for
editing compiles it, but the compilation process hangs populates the data
structures you need for editing. So, when you go to edit, say:


class Collection {
   
  method inject_into ($self: $initial_value is copy, *block) {
$self.each
  :{ $initial_value = block($initial_value, $_) }
return $initial_value;
  }
}

You'd end up with a Class object which would have all sorts of interesting
things added to it, including collections of instance variables, class
variables, methods etc. Methods would have metadata about the environment in
which they were evaluated (so when you edit a method you could recompile it in
the correct environment), the file they're found in, their source code
(possibly attributed to allow for syntax highlighting), the resulting AST,
etc. Once you have such a rich set of metadata to hand, it becomes a simple
matter of programming to add all sorts of handy features, refactorings,
breakpoints, whatever...

The point being that you don't really need a massive amount of support from the
core language to do all this. At a pinch you can take advantage of the fact
that the grammar is replaceable and core classes are extensible (and if they're
not, you just stick them in boxes for the editor's purposes).

 And to extend that to also be a refactoring browser would require the
 meta-meta-class system to be able to generate and emit code, which, if we
 properly model the meta-meta-classes should also be fairly simple as well.

We've got eval for that. Assuming you can do eval with arbitrary bindings (and
if necessary you can drill down to parrot for it) you just do something like:

Class.can('fred').environment.eval($freds_new_sourcecode)

Of course, if you alter a macro you're going to have to reevaluate pretty much
all the source code, but so what?

 And of course all this is even simpler if the interpreter is written in Perl6
 (although the cyclical nature of that can be dizzying at times).

It really doesn't have to be. It'd be nice, but not necessary. You just have to
make sure you can extend the runtime using perl. And macros already do that.

 However adding debugging support to this is another matter all together, and 
 it
 is only partially useful to the non-OO programmer.

Not that hard to be honest. The same instrumented interpreter techniques can be
used to write the debugger.


Re: Perl6 and support for Refactoring IDE's

2005-05-25 Thread Piers Cawley
Luke Palmer [EMAIL PROTECTED] writes:

 On 5/6/05, J Matisse Enzer [EMAIL PROTECTED] wrote:
 I've become scared that if Perl is to continue to be viable for large,
 complex, multi-developer projects that the tools need to serious
 catch-up with what is available for Java, for example. Things like:
 
- Refactoring Support (see http://www.refactoring.com/)
- CVS and/or Subversion integration
- Support for integrating regression tests and auto-building
- Automated syntax and dependency checking
 
 I've been using Eclipse, with the EPIC plugin
 (http://e-p-i-c.sourceforge.net/) and so far I like it. It uses
 Devel::Refactor to support extract subroutine, but a lot more is
 needed to match what you can do with Java these days.
 
 What are others' thoughts on this?

 I think you're absolutely right.  Perl should have an IDE with
 Eclipse-like context-sensitivity and refactoring support.  However,
 it's hardly in Perl's philosophy or interest to bless one.

 One thing is for sure.  Perl 6 is providing enough introspection and
 parsing capabilities to make it possible to write a context-sensitive
 IDE, unlike Perl 5 (well, Perl 5 made it *possible*, I suppose, but
 Perl 6 will make it obvious).  Perl 6 is exposing its whole grammar at
 the language level, so you can say give me a syntax tree for this
 chunk of code and it will.  Even if there are modules that change the
 syntax with macros (though your editor might have trouble
 understanding what the macros mean).

One of the 'mental apps' that's been pushing some of the things I've been
asking for in Perl 6's introspection system is a combined
refactoring/debugging/editing environment for the language. One of the
annoyances of the 'only perl can parse Perl' thing is not so much the truth of
the statement, but that perl 5 doesn't allow you to ask about the parsed code
in ways that would be useful for an IDE. Perl 6 promises to change that -- it
should be possible to either write a fantastic Perl 6 IDE in perl itself, or to
write a codegrokker object that can be used by some pre existing IDE. 

 In other words, Perl 6 is open to the possibility of such an IDE, and
 is going to provide the machinery necessary to build a really good
 one, but I doubt it will become a development milestone.

What about the debugger?


Re: Perl6 and support for Refactoring IDE's

2005-05-25 Thread Deborah Pickett

Piers Cawley wrote:

One of the 'mental apps' that's been pushing some of the things I've been
asking for in Perl 6's introspection system is a combined
refactoring/debugging/editing environment for the language. One of the
annoyances of the 'only perl can parse Perl' thing is not so much the truth of
the statement, but that perl 5 doesn't allow you to ask about the parsed code
in ways that would be useful for an IDE. Perl 6 promises to change that -- it
should be possible to either write a fantastic Perl 6 IDE in perl itself, or to
write a codegrokker object that can be used by some pre existing IDE. 


While I agree with this wholeheartedly, I am afraid to say that I 
haven't seen a lot of evidence that introspection has been given much 
thought in the Perl 6 design process.  There was a flurry at one point 
about the execution of BEGIN blocks, but then silence again.


One thing is for sure: although B::Deparse is a wonder to behold, it 
isn't something that we should try to be repeating.  It doesn't give you 
control over how much of the code to parse, whether you trust the code 
your're parsing, or in the case of BEGIN blocks even whether you have 
the choice to execute them or not.  It doesn't give you the ability to 
load parts of another program into the running program's namespace, and 
it doesn't give you enough hooks and callbacks to be able to transform 
syntax trees.  At least, if it does do any of those things, then I 
haven't seen ways of doing them in the documentation for B and O.


I'm afraid that because of the dynamic parse/execute nature of Perl, it 
may be a theoretically intractable problem to parse Perl safely.  Don't 
run the BEGIN blocks and you change the way future source is parsed 
compared with true runtime.  Do run them and you risk someone inserting 
dangerous code into one.  It isn't good if turning on syntax 
highlighting on some found-in-the-wild code in your Perl-aware editor 
could wipe your hard disk.


Oh, and I don't have a lot of faith in the Safe module, either, which is 
a marvel of kludge engineering and broken, both at the same time. 
What's the plan for that module's moral equivalent in Perl 6?


I'd love it if someone could set my mind at ease here.  I have to 
process syntax trees of Perl code quite often in my line of work and I'd 
love something less finicky than a Safe/B::* combination.


Re: Perl6 and support for Refactoring IDE's

2005-05-25 Thread Stevan Little

On May 25, 2005, at 5:39 AM, Piers Cawley wrote:
One of the 'mental apps' that's been pushing some of the things I've 
been

asking for in Perl 6's introspection system is a combined
refactoring/debugging/editing environment for the language.


Maybe I have been reading too much about Smalltalk meta-classes lately, 
but in doing some draft ideas of the Perl6 meta-model for Pugs, I 
realized that given a really solid class/metaclass introspection system 
and access to the interpreter level meta-meta-classes, it would be 
possible to easily write a class browser much like Smalltalk.


And to extend that to also be a refactoring browser would require the 
meta-meta-class system to be able to generate and emit code, which, if 
we properly model the meta-meta-classes should also be fairly simple as 
well.


And of course all this is even simpler if the interpreter is written in 
Perl6 (although the cyclical nature of that can be dizzying at times).


However adding debugging support to this is another matter all 
together, and it is only partially useful to the non-OO programmer.


Anyway, just wanted to add my 2 cents. Back to $work for me :)

Stevan



Re: Perl6 and support for Refactoring IDE's

2005-05-25 Thread Luke Palmer
On 5/25/05, Deborah Pickett [EMAIL PROTECTED] wrote:
 I'm afraid that because of the dynamic parse/execute nature of Perl, it
 may be a theoretically intractable problem to parse Perl safely. 

Yep.  It's not really possible for the parser to distinguish between:

BEGIN {
%main::{'foo'} = sub { print Hello! };
}

And:

BEGIN { for 1... {} }

Or even:

BEGIN { system 'rm -rf /' }

So, like most things in Perl, we can trust the author of the code not
to do such things, or we can not trust them and lose something while
we're at it (the choice is given to the IDE user, not to us
designers).  BEGIN blocks should never do INITialization, opening
filehandles, etc., precisely for this reason.  This becomes even more
important in Perl 6 where bytecode will be more prevelant.

You may certainly parse Perl syntax trees without executing BEGIN
blocks, but you have to bite the bullet if the BEGIN changes the
syntax in a way that you can no longer parse it if you don't execute
it.  And if you can't accept that, I see only one option:  give up
BEGIN blocks.  And we're not going to do that  (but you're free to
place that restriction on whoever's code you're parsing).

Luke


Re: Perl6 and support for Refactoring IDE's

2005-05-06 Thread Luke Palmer
On 5/6/05, J Matisse Enzer [EMAIL PROTECTED] wrote:
 I've become scared that if Perl is to continue to be viable for large,
 complex, multi-developer projects that the tools need to serious
 catch-up with what is available for Java, for example. Things like:
 
- Refactoring Support (see http://www.refactoring.com/)
- CVS and/or Subversion integration
- Support for integrating regression tests and auto-building
- Automated syntax and dependency checking
 
 I've been using Eclipse, with the EPIC plugin
 (http://e-p-i-c.sourceforge.net/) and so far I like it. It uses
 Devel::Refactor to support extract subroutine, but a lot more is
 needed to match what you can do with Java these days.
 
 What are others' thoughts on this?

I think you're absolutely right.  Perl should have an IDE with
Eclipse-like context-sensitivity and refactoring support.  However,
it's hardly in Perl's philosophy or interest to bless one.

One thing is for sure.  Perl 6 is providing enough introspection and
parsing capabilities to make it possible to write a context-sensitive
IDE, unlike Perl 5 (well, Perl 5 made it *possible*, I suppose, but
Perl 6 will make it obvious).  Perl 6 is exposing its whole grammar at
the language level, so you can say give me a syntax tree for this
chunk of code and it will.  Even if there are modules that change the
syntax with macros (though your editor might have trouble
understanding what the macros mean).

In other words, Perl 6 is open to the possibility of such an IDE, and
is going to provide the machinery necessary to build a really good
one, but I doubt it will become a development milestone.

Luke


Re: Perl6 and support for Refactoring IDE's

2005-05-06 Thread Fagyal Csongor
Matisse,
Will Perl 6 help us have tools that are as good or better than the 
ones available for Java, C#, etc?

I've been using Perl since 1994 and for the past several years have 
used it to build a number of complex mod_perl applications. I love Perl.

The following  may be considered heresy, but I believe that these days 
the quality of the whole development environment is just as important, 
maybe *more* important than the language itself. I mean that the 
availability of *easily integrated* tools including IDE, automated 
testing tools, refactoring, deployment tools, etc. are a HUGE factor 
in determining what language gets used for a large project
Heresy or not, I think you are (mostly) right. Actually I would say your 
are right *unfortunately*.

Available tools are a huge factor, yes - not in a way as which language 
to select for a project, but which one to *deselect*. And currently Perl 
(I mean Perl5) is not doing very well compared to others.

I've become scared that if Perl is to continue to be viable for large, 
complex, multi-developer projects that the tools need to serious 
catch-up with what is available for Java, for example. Things like:

  - Refactoring Support (see http://www.refactoring.com/)
  - CVS and/or Subversion integration
  - Support for integrating regression tests and auto-building
  - Automated syntax and dependency checking
IMHO subversioning does not have too much to do with the language 
itself. Subversioning operates on files. An IDE might integrate some 
interface for this task, but that is a different question.
Syntax checking in general is very hard to do in Perl5/6 because of the 
great amount of line noise there (consider yourself in the middle of 
writing a regexp :)) The perl6 compiler must be able to do syntax 
checking, though :-), and the design of Perl6, as I understand, 
definetely gives you more possibilities compared to Perl5.

For the others, some more knowledgeable folks will probably provide some 
answers and ideas. However, I think that the language design and 
implementation does not have too much to do with IDE availability.

I've been using Eclipse, with the EPIC plugin 
(http://e-p-i-c.sourceforge.net/) and so far I like it. It uses 
Devel::Refactor to support extract subroutine, but a lot more is 
needed to match what you can do with Java these days.

What are others' thoughts on this?
Have you considered using the DONATE button on 
http://e-p-i-c.sourceforge.net/ ? :)))

- Fagzal


Re: Perl6 and support for Refactoring IDE's

2005-05-06 Thread J Matisse Enzer
Thanks for your comments - I was afraid I'd get flamed for suggesting 
something wasn't perfect about Perl :-)

On May 6, 2005, at 9:31 AM, Fagyal Csongor wrote:
IMHO subversioning does not have too much to do with the language 
itself. Subversioning operates on files. An IDE might integrate some 
interface for this task, but that is a different question.
You are right - it's not a language issue, sorry I left that in.
Have you considered using the DONATE button on 
http://e-p-i-c.sourceforge.net/ ? :)))
Yes :-) I've donated :-)
I contributed some changes to Devel::Refactor (added rename_subroutine 
and a test suite) and now I have loaded the Java source for the EPIC 
editor into Eclipse and started looking at how to add support for 
'rename_subroutine' - but my Java skills are less than minimal :-)

In another post,  On May 6, 2005, at 9:26 AM, Luke Palmer wrote:
I think you're absolutely right.  Perl should have an IDE with
Eclipse-like context-sensitivity and refactoring support.  However,
it's hardly in Perl's philosophy or interest to bless one.
and I agree that Perl should not bless any one IDE, and I am *very* 
happy to read that Perl6 will make it much easier for good tools to be 
created for Perl, as Luke Palmer said:

One thing is for sure.  Perl 6 is providing enough introspection and
parsing capabilities to make it possible to write a context-sensitive
IDE, unlike Perl 5 (well, Perl 5 made it *possible*, I suppose, but
Perl 6 will make it obvious).  Perl 6 is exposing its whole grammar at
the language level, so you can say give me a syntax tree for this
chunk of code and it will.  Even if there are modules that change the
syntax with macros (though your editor might have trouble
understanding what the macros mean).
So, perhaps that leave me (us) in the following situation:
  1) Do what I can do help Perl6 happen faster (not much I can do, the
 current types of programming required is beyond my skills)
 Maybe I can help evangelize Perl6.
  2) Try and improve the tools that do exist for Perl5 - I am doing this
 with Devel::Refactor and EPIC, but I don't know if I'll learn 
enough
 Java to make a difference in EPIC any time soon.
  3) Try and keep Perl alive in other ways so it's still around when 
Perl6 arrives.

Yesterday I was at an engineering Open House at Google, and I asked one 
of the engineers What can Perl do for Google?, and he basically said 
(I'm paraphrasing) Not much, we are a Java and Python shop. Getting a 
real refactoring IDE for Perl 5 is more compelling a story then a 
virtual machine (Parrot) that will run multiple dynamically-typed 
languages. (they already have Jython)

Now that was just one person's statement, but this fellow (John Brewer, 
www.jera.com) is not a Perl hater - he uses Perl, and he's bright, 
level headed and was not trying to be hostile. In fact he said these 
things to me and to Larry Wall who was listening very politely :-)

Maybe this isn't the right place to keep discussing this, so I'll take 
pointers to other places. I'm very worried about the continued 
viability of Perl for major projects and am trying connect with other 
people and see what can be done about it.

---
Matisse Enzer [EMAIL PROTECTED]
http://www.matisse.net/  - http://www.eigenstate.net/


Re: Perl6 and support for Refactoring IDE's

2005-05-06 Thread Larry Wall
On Fri, May 06, 2005 at 10:26:26AM -0600, Luke Palmer wrote:
: In other words, Perl 6 is open to the possibility of such an IDE, and
: is going to provide the machinery necessary to build a really good
: one, but I doubt it will become a development milestone.

I think that, just as Perl 1 built in all the system interfaces, and
Perl 5 allowed interfaces to every database under the sun, we should
just make sure that Perl 6 can interface easily to every existing
and future IDE, including any that might be written particular with
Parrot in mind.

Larry


Re: Perl6 and support for Refactoring IDE's

2005-05-06 Thread Fagyal Csongor
Matisse,
Just one note before we take this off-list:
Maybe this isn't the right place to keep discussing this, so I'll take 
pointers to other places. I'm very worried about the continued 
viability of Perl for major projects and am trying connect with other 
people and see what can be done about it.
I share some of your fears, and would be very much interested in talking 
about them with you (and others). Actually I gave a small lightening 
speech on this topic at last year's Perl gathering at Budapest, which I 
will hopefully make into an article some day...

please_spare_my_lifeIMHO Perl needs some more marketing 
:)/please_spare_my_life

- Fagzal