Re: performance, compilation, Spamassassin (was Re: Wishlist remarks)
Craig A. Berry wrote: At 12:19 AM -0400 6/7/04, John Malmberg wrote: Craig A. Berry wrote: To compile Perl directly, you need to interface to a back end code generator. True, but we need to define code here. For Perl, it generally means bytecode that is run on the Perl bytecode engine, which is roughly equivalent to what Java folks call a virtual machine. I say roughly because Perl is a dynamically typed language (unlike Java or C#). In Perl, 8.341 may just be a string right up until the point you do arithmetic on it, at which point it becomes a number. Several other compilers know how to deal with that. All variables are passed by descriptor to most routines. Others can be handled by the optimizer. An example in the C compiler is the handling of printf, sprintf, fprintf. If the optimizer can resolve the format string at compile time, it breaks up the call into more specific formatting routines. The Gnat ADA project uses the code generator in GCC for this, and if the other Perl compilers use this, then there may be a way to get this done on OpenVMS. Good point. If someone were to target the GCC code generator, and if this code generator were sufficiently smart enough to generate the right instructions on Alpha and Itanium (no small task), then there's no particular reason this wouldn't work as well on VMS as elsewhere. GCC has code generates for almost every processor, including VAX, Alpha, and IA64. As far as your other points, I am totally ignorant of Perl and what the compiler efforts are doing. Generally the way that a code generator works is that it provides an API to describe the instruction set to the front end of the compiler so that the front end can emit the proper code stream. Logically if an open source project like Perl is going to have a version that can be compiled to an executable, it would either have a custom code generator that will only support a few targets, or it would use the GCC code generator. Using the GCC code generator is not a trivial project. The GCC documentation appears to be several generations behind the current code base. -John [EMAIL PROTECTED] Personal Opinion Only
Re: performance, compilation, Spamassassin (was Re: Wishlist remarks)
At 12:19 AM -0400 6/7/04, John Malmberg wrote: Craig A. Berry wrote: http://www.perldoc.com/perl5.8.4/pod/perlcompile.html I think what this gets you in that subset of cases for which it even works is a Perl op-tree implemented in C. In other words, you get a C program that executes the particular stream of Perl byte codes that make up your program. To compile Perl directly, you need to interface to a back end code generator. True, but we need to define code here. For Perl, it generally means bytecode that is run on the Perl bytecode engine, which is roughly equivalent to what Java folks call a virtual machine. I say roughly because Perl is a dynamically typed language (unlike Java or C#). In Perl, 8.341 may just be a string right up until the point you do arithmetic on it, at which point it becomes a number. For OpenVMS, the back end translators used by HP are GEM and VCG, and as far as I know there is no public documentation for either. Nor is either capable of generating Perl bytecodes :-). The Gnat ADA project uses the code generator in GCC for this, and if the other Perl compilers use this, then there may be a way to get this done on OpenVMS. Good point. If someone were to target the GCC code generator, and if this code generator were sufficiently smart enough to generate the right instructions on Alpha and Itanium (no small task), then there's no particular reason this wouldn't work as well on VMS as elsewhere. What I think you'd want to do would be to make a cross assembler, as it were, one that converts the pseudo-instructions known as Perl bytecodes into native machine instructions. And if you were going to invest the effort required to do that, you'd really want to use the Perl 6 bytecodes (a clean sheet of paper reimplementation known as Parrot) rather than Perl 5. For more on Parrot, which will form the guts of Perl 6, and most likely some future version of Perl 5, see http://www.parrotcode.org/. But none of this is what was under discussion in the desire to compile Spamassassin. The executable one would distribute would not be an OpenVMS VAX, Alpha, or I64 image; instead it would be a platform-independent stream of bytecodes read off disk and run on the Perl virtual machine. This would save you the parsing, syntax checking, bytecode optimization, and probably a few other steps I'm forgetting at the moment. It would not eliminate the virtual machine. It would not have any particular speed advantages over a server process that compiles to bytecode once at start-up and processes requests continuously until shut down. It could have some deployment advantages if bundled with a Perl interpreter -- see Nicco's post in this thread. -- Craig A. Berry mailto:[EMAIL PROTECTED] ... getting out of a sonnet is much more difficult than getting in. Brad Leithauser
Re: performance, compilation, Spamassassin (was Re: Wishlist remarks)
Craig A. Berry wrote: At 11:40 AM +0200 6/2/04, Willem Grooters wrote: http://www.perldoc.com/perl5.8.4/pod/perlcompile.html I think what this gets you in that subset of cases for which it even works is a Perl op-tree implemented in C. In other words, you get a C program that executes the particular stream of Perl byte codes that make up your program. To compile Perl directly, you need to interface to a back end code generator. For OpenVMS, the back end translators used by HP are GEM and VCG, and as far as I know there is no public documentation for either. The Gnat ADA project uses the code generator in GCC for this, and if the other Perl compilers use this, then there may be a way to get this done on OpenVMS. From the posting of the Gnat developer, apparently building the current GCC back end now requires Perl, and in particular some changes to the VMS Perl implementation to remove the command line length limitations and to invoke BASH as the shell. So there appears to be something that multiple people could collaborate on. -John [EMAIL PROTECTED] Personal Opinion Only
performance, compilation, Spamassassin (was Re: Wishlist remarks)
At 11:40 AM +0200 6/2/04, Willem Grooters wrote: Sorry, something went wrong and a reply was sent to the original sender in stead of the list. The Perl lists give you reply to sender by default; you need to explicitly choose reply to all to reply to the list. So this is our communication. For completeness, I kept all of the original answer. For efficiency, I've snipped quite a bit :-). What you ask for is a real PERL compiler that creates VMS-compliant objects, isn't it? Yes, or even simply a builder that could compile and link an entire set pf perl scripts into an image that one could invoke with RUN. Perl will never be a compiled language the way FORTRAN and COBOL are, though of course it compiles its scripts to bytecode before executing them. On the Perl to C translator, see: http://www.perldoc.com/perl5.8.4/pod/perlcompile.html I think what this gets you in that subset of cases for which it even works is a Perl op-tree implemented in C. In other words, you get a C program that executes the particular stream of Perl byte codes that make up your program. I guess what I'm saying here is that typically a Perl script pulls in many other perl modules and if compiling them separately into individual modules was too hard then for my purposes at this point just having an image alone would be nice. Loading a large number of modules can be a performance problem. Over the years we've occasionally discussed storing Perl modules in text libraries and using the librarian to fetch them; it should be much more efficient than rooting around in the file system. You'd still have all the image activations for the shareable images. My particular need arises from my recent work in Spamassassin. Think about the way it's used. It's invoked once for every single mail message coming into the system and each time it's run that means Perl is re-building (er, um compiling and linking or whatever the Perl builder does internally) the program before running it. That's a ton of wasted CPU time and unnecessary latency in mail proccessing and won't scale well. Indeed, but that's not the way most people use it on Unix. It can be embedded in a wide variety of mailers or run as a separate daemon. You'd have a Perl interpreter started and running, and you'd have Spamassassin compiled and running, sitting there waiting for mail messages to process. These things may not work out of the box on VMS, but that's a porting challenge. Karol, I seem to remember you had trouble some time back getting an embedded Perl to work, but I think that's really the right way to do this and worth revisiting. Still, I do have some concerns but didn't reply yet (on- or off-list) on Craig's message. Most important is there is the real danger(!) that Perl programs will introduce even more Unix-like requirements like STREAM_LF files and inter-process communication (the PIPEs). I agree that if there is some form of communication between different systems, you'll have to use some 'generic' format, but system-specific methods must be possible when communication on one platform (MBX on VMS, for instance). It should be VERY clear - to ANY developer on this project - that VMS is NOT another Unix. It can do some things the Unix way, but it has quite some capabilities Unix can only deram about. USE THEM. Calm down, Willem. What the project needs is more developers able and willing to keep up the tradition of letting Perl be Perl, and use the C run-time, homegrown hacks, VMS system services, DCL, and anything else within reach to make that happen. What the project does NOT need is to have this list become yet another forum for OS wars. -- Craig A. Berry mailto:[EMAIL PROTECTED] ... getting out of a sonnet is much more difficult than getting in. Brad Leithauser
RE: performance, compilation, Spamassassin (was Re: Wishlist remarks)
LS, For windows Unix there are packaging tools that get a perl interpreter + source (zipped) + needed modules (also zipped) together in one executable. What it gets you is execute a perl script on a system that has no perl installed. (perl2exe). To use spamassassin effectively there also exist packages like amavis/amavisd, that keep a certain amount preforked (ready waiting servers using Net::Server ) around to dispatch a mail to for analysis (spam virus). Spamassassin can run as a server, using spamc as a client executable to feed it its favorite food. Both these systems have evolved because of the overhead that spamassassin has. Kind Regards, Nico Baggus - ATTENTION: The information in this electronic mail message is private and confidential, and only intended for the addressee. Should you receive this message by mistake, you are hereby notified that any disclosure, reproduction, distribution or use of this message is strictly prohibited. Please inform the sender by reply transmission and delete the message without copying or opening it. Messages and attachments are scanned for all viruses known. If this message contains password-protected attachments, the files have NOT been scanned for viruses by the ING mail domain. Always scan attachments before opening them. -
Re: performance, compilation, Spamassassin (was Re: Wishlist remarks)
For efficiency, I've snipped quite a bit :-). What you ask for is a real PERL compiler that creates VMS-compliant objects, isn't it? Yes, or even simply a builder that could compile and link an entire set pf perl scripts into an image that one could invoke with RUN. Perl will never be a compiled language the way FORTRAN and COBOL are, though of course it compiles its scripts to bytecode before executing them. On the Perl to C translator, see: http://www.perldoc.com/perl5.8.4/pod/perlcompile.html I think what this gets you in that subset of cases for which it even works is a Perl op-tree implemented in C. In other words, you get a C program that executes the particular stream of Perl byte codes that make up your program. Combined with Nico Baggus' remark: that's ok - for me anyway. See what can be achieved on VMS for that. Most important is there is the real danger(!) that Perl programs will introduce even more Unix-like requirements like STREAM_LF files and inter-process communication (the PIPEs). I agree that if there is some form of communication between different systems, you'll have to use some 'generic' format, but system-specific methods must be possible when communication on one platform (MBX on VMS, for instance). It should be VERY clear - to ANY developer on this project - that VMS is NOT another Unix. It can do some things the Unix way, but it has quite some capabilities Unix can only deram about. USE THEM. Calm down, Willem. What the project needs is more developers able and willing to keep up the tradition of letting Perl be Perl, and use the C run-time, homegrown hacks, VMS system services, DCL, and anything else within reach to make that happen. What the project does NOT need is to have this list become yet another forum for OS wars. Don't worry, I don't start a war. I just wanted to point out that this is exactly what is needed on the VMS side. I'm quite willing to team in, were it not that my knowledge of Unix (or Windows) is limited to the 'outside' of these system. But I'll try and work my way into Perl to let it work - whereever required and possible - the VMS way as well. (What level of platform independence is either required, advisable or just handy? And in what way can discrepancies be handled? These are, IMHO,. quite basic questions, if you want Perl to be really usable on virtually ANY platform. But that might well be a very different discussion) Willem Grooters OpenVMS developer System manager e: [EMAIL PROTECTED] w: www.grootersnet.nl
