Re: Using Inline Python to import Python vars
-Original Message-
From: sisyph...@optusnet.com.au
Sent: Tuesday, April 09, 2013 12:44 PM
To: Chris Nighswonger ; inline@perl.org
Subject: Re: Using Inline Python to import Python vars
-Original Message-
From: Chris Nighswonger
@{$DATA{$pkg}} = split /(?m)(__\S+?__\n)/, $data;
Try replacing that line (in Inline.pm) with:
#warn \n\n\$data:\n$data\n\n;
@{$DATA{$pkg}} = split /(?m)(\n[ \t]{0,}__\S+?__\n)/, $data;
for(@{$DATA{$pkg}}) {$_ = (split /\s/, $_)[-1] . \n if $_ =~
/^\s{0,}__\S+?__\n$/} # remove any whitespace that precedes marker
#warn \n\n, scalar(@{$DATA{$pkg}}), \n;l
#for(@{$DATA{$pkg}}) {warn $_\n}
It's only 2 lines of code - the (commented out) warnings might be useful if
things go awry. They won't appear in the final version of Inline.pm.
That works ok for my failing Inline::C script, and without causing any of
the Inline tests in the Inline-0.52 test suite to fail.
I'm hopeful it will do the trick for your Inline::Python script, too.
It's possibly a little convoluted at the moment ... maybe someone can
improve on that.
Cheers,
Rob
Re: Using Inline Python to import Python vars
Hi Rob,
On Tue, Apr 9, 2013 at 4:15 AM, sisyph...@optusnet.com.au wrote:
-Original Message- From: sisyph...@optusnet.com.au
Sent: Tuesday, April 09, 2013 12:44 PM
To: Chris Nighswonger ; inline@perl.org
Subject: Re: Using Inline Python to import Python vars
-Original Message- From: Chris Nighswonger
@{$DATA{$pkg}} = split /(?m)(__\S+?__\n)/, $data;
Try replacing that line (in Inline.pm) with:
#warn \n\n\$data:\n$data\n#**###\n;
@{$DATA{$pkg}} = split /(?m)(\n[ \t]{0,}__\S+?__\n)/, $data;
for(@{$DATA{$pkg}}) {$_ = (split /\s/, $_)[-1] . \n if $_ =~
/^\s{0,}__\S+?__\n$/} # remove any whitespace that precedes marker
#warn \n\n, scalar(@{$DATA{$pkg}}), \n;l
#for(@{$DATA{$pkg}}) {warn $_\n}
It's only 2 lines of code - the (commented out) warnings might be useful
if things go awry. They won't appear in the final version of Inline.pm.
That works ok for my failing Inline::C script, and without causing any of
the Inline tests in the Inline-0.52 test suite to fail.
I'm hopeful it will do the trick for your Inline::Python script, too.
It's possibly a little convoluted at the moment ... maybe someone can
improve on that.
I'll be out of the office most of the day today, but will try this out
later this afternoon/evening and post back.
Kind Regards,
Chris
Re: Using Inline Python to import Python vars
On Tue, Apr 9, 2013 at 4:15 AM, sisyph...@optusnet.com.au wrote:
-Original Message- From: sisyph...@optusnet.com.au
Sent: Tuesday, April 09, 2013 12:44 PM
To: Chris Nighswonger ; inline@perl.org
Subject: Re: Using Inline Python to import Python vars
-Original Message- From: Chris Nighswonger
@{$DATA{$pkg}} = split /(?m)(__\S+?__\n)/, $data;
Try replacing that line (in Inline.pm) with:
#warn \n\n\$data:\n$data\n#**###\n;
@{$DATA{$pkg}} = split /(?m)(\n[ \t]{0,}__\S+?__\n)/, $data;
for(@{$DATA{$pkg}}) {$_ = (split /\s/, $_)[-1] . \n if $_ =~
/^\s{0,}__\S+?__\n$/} # remove any whitespace that precedes marker
#warn \n\n, scalar(@{$DATA{$pkg}}), \n;l
#for(@{$DATA{$pkg}}) {warn $_\n}
It's only 2 lines of code - the (commented out) warnings might be useful
if things go awry. They won't appear in the final version of Inline.pm.
That works ok for my failing Inline::C script, and without causing any of
the Inline tests in the Inline-0.52 test suite to fail.
I'm hopeful it will do the trick for your Inline::Python script, too.
This fixes the problem! Fantastic!
It's possibly a little convoluted at the moment ... maybe someone can
improve on that.
Is there any case where the marker would not be on a line by itself? If not
perhaps we could just do such as
@{$DATA{$pkg}} = split /(?m)^(__\S+?__\n)/, $data;
and eliminate the need to remove whitespace?
Kind Regards,
Chris
Re: Using Inline Python to import Python vars
From: Chris Nighswonger
This fixes the problem! Fantastic!
It's possibly a little convoluted at the moment ... maybe someone can
improve on that.
Is there any case where the marker would not be on a line by itself?
If it's not on a line by itself, it will be deemed to *not* be a marker.
If it matches /__\S+?__\n/ and is on a line by itself, it *will* be regarded
as a marker.
There is therefore still an opening (in Inline::C, at least) to break
Inline.
Even after the alteration to sub read_DATA has been made, this Inline::C
code is still broken:
void foo() {
printf (%d\n,
__MINGW32__
+ 7);
}
Similarly, if it's valid python to rewrite
return __version__
as
return
__version__
then Inline::Python would still be broken by the latter.
I reckon it's probably a real can of worms to try and fix that - so I'll
just leave it as is (until someone complains, anyway).
I think we would have to start checking for matches against known markers
(__C__, __Python__, etc.) as you originally suggested. Even then there would
be nothing to stop me from breaking Inline::C by defining a symbol named
__Python__ (or __CPP__, etc.) and using that symbol in my Inline::C code.
Hopefully, everyone writes their code *across* the page, not *down* the page
!!
If not perhaps we could just do such as
@{$DATA{$pkg}} = split /(?m)^(__\S+?__\n)/, $data;
and eliminate the need to remove whitespace?
In the past, Inline has removed the whitespace that precedes the marker -
and it performs a check to verify that the whitespace has been removed.
So ... we need to remove the whitespace to satisfy that check.
(Alternatively, I could rewrite the check, but I think it's safer to stick
to the original formatting rules.)
In fact, in the past, Inline has removed everything (not just whitespace)
that precedes the marker on the line. So, in the past, you should have been
able to write:
__DATA__
some arbitrary stuff __Python__
..
Inline would then remove the some arbitrary stuff that precedes the
__Python__ marker.
This will no longer be the case. If the marker is preceded by anything other
than whitespace, it will be regarded as not being a marker.
So if anyone has been preceding the marker with anything other than
whitespace, they'll find their code broken.
Surely, no-one has been doing that ?
Anyway, I'll chew things over for a day or two - wait and see if there's any
other thoughts on what ought to be done.
Then I'll upload a 0.52_01 release that contains this amendment to
Inline.pm. (I'll let the list know when that happens.)
Thanks for pursuing this, Chris, Stefan.
Cheers,
Rob
Re: Using Inline Python to import Python vars
On Tue, Apr 9, 2013 at 6:48 PM, sisyph...@optusnet.com.au wrote: Thanks for pursuing this, Chris, Stefan. Many thanks to Stefan for helping me along and to Rob for fixing things up. Kind Regards, Chris
Re: Using Inline Python to import Python vars
On Mon, Apr 1, 2013 at 12:57 PM, Chris Nighswonger
cnighswon...@foundations.edu wrote:
However, this code borks:
use Inline Python = 'DATA',
DIRECTORY = '/usr/share/webmin/pykota/.Inline/',
NAME= 'PyKota::Test';
print pykota_version();
1;
__DATA__
__Python__
def pykota_version():
from pykota.version import __version__
return __version__
The error is:
File string, line 3
from pykota.version import
^
SyntaxError: invalid syntax
Error -- py_eval raised an exception at
/usr/local/lib/perl/5.14.2/Inline/Python.pm line 177.
INIT failed--call queue aborted.
So I threw together some C to see if the problem was with my Python syntax
in the context of Inline::Python. Below is what I did and it works as
expected.
Any suggestions as to where I'm going wrong here?
Kind Regards,
Chris
#include Python.h
main( ) {
PyObject *pstr, *pmod, *pdict;
Py_Initialize( );
pmod = PyImport_ImportModule(__main__);
pdict = PyModule_GetDict(pmod);
pstr = PyRun_String(
def pykota_version():\n
from pykota.version import __version__\n
return __version__\n
print pykota_version()\n
, Py_file_input, pdict, pdict);
if (!pstr) {
PyErr_Print();
exit (1);
}
Py_DECREF(pmod);
Py_DECREF(pstr);
}
Re: Using Inline Python to import Python vars
Hi Chris, On Monday 08 April 2013 14:45:40 Chris Nighswonger wrote: However, this code borks: ... __DATA__ __Python__ def pykota_version(): from pykota.version import __version__ return __version__ The error is: File string, line 3 from pykota.version import ^ SyntaxError: invalid syntax Using perl Makefile.PL -debug when building Inline::Python gives a good hint. With this flag it will print loads of information including Python source code it compiles to stderr. This gives us: py_eval: code: def pykota_version(): from pykota.version import py_eval: main_module=0x7fd023985be8 py_eval: globals=0x229aac0 py_eval: type=1 py_eval: context=257 File string, line 3 from pykota.version import ^ SyntaxError: invalid syntax I've never used DATA file handles in Perl much less with Inline::Python, so I'm not exactly sure. But the debug output looks like the Python source code ends at __version__. So I guess, the code reading the DATA file processes everything from __Python__ to the next marker and treats every word enclosed in double underscores as such marker. Indeed, every other way of giving Python source code to Inline::Python seems to work ok. Regards, Stefan
Re: Using Inline Python to import Python vars
On Mon, Apr 8, 2013 at 4:25 PM, Chris Nighswonger
cnighswon...@foundations.edu wrote:
On Mon, Apr 8, 2013 at 3:14 PM, Stefan Seifert n...@detonation.orgwrote:
I've never used DATA file handles in Perl much less with Inline::Python,
so
I'm not exactly sure. But the debug output looks like the Python source
code
ends at __version__. So I guess, the code reading the DATA file processes
everything from __Python__ to the next marker and treats every word
enclosed
in double underscores as such marker.
So it does look as though the DATA splitting algorithm is a bit loose.
In Inline.pm abt line 354:
@{$DATA{$pkg}} = split /(?m)(__\S+?__\n)/, $data;
shift @{$DATA{$pkg}} unless ($ {$DATA{$pkg}}[0] || '') =~ /__\S+?__\n/;
}
It appears we split on anything prefixed and suffixed with double
underscores.
Assuming that I'm not way off here:
I'm guessing that this is because we really don't know whether this is C,
Java, Python, or FooBar?
Can we have the language extension pass in what it is and actually search
for that marker rather than for __ANYTHING__?
Could we actually check for some specific end-of-code-block maker? Say
'__EOI__' for end-of-inline or some such?
Kind Regards,
Chris
Re: Using Inline Python to import Python vars
-Original Message-
From: Chris Nighswonger
Sent: Tuesday, April 09, 2013 6:39 AM
To: inline@perl.org
Subject: Re: Using Inline Python to import Python vars
So it does look as though the DATA splitting algorithm is a bit loose.
In Inline.pm abt line 354:
@{$DATA{$pkg}} = split /(?m)(__\S+?__\n)/, $data;
shift @{$DATA{$pkg}} unless ($ {$DATA{$pkg}}[0] || '') =~
/__\S+?__\n/;
}
It appears we split on anything prefixed and suffixed with double
underscores.
A newline (immediately after the closing __) is also needed before any
splitting takes place:
##
use warnings;
$data = __DATA__ __C__ __other__ __Python__\n__version__;
@stuff = split /(?m)(__\S+?__\n)/, $data;
print $_\n for @stuff;
##
Outputs:
__DATA__ __C__ __other__
__Python__
__version__
I can get the same type of failure with this (non-portable) Inline::C
script:
##
use strict;
use warnings;
use Inline C = 'DATA';
foo();
1;
__DATA__
__C__
void foo() {
printf(%d %s\n,__MINGW32__
,Hello);
}
##
However, there's no problem at all if I rewrite foo() as:
void foo() {
printf(%d %s\n,__MINGW32__,
Hello);
}
Thanks for digging this up, Chris !!
I don't personally use the __DATA__ approach.
I'll do some digging myself right now, and see if I can come up with a
simple improvement to the regex.
(All the better if someone else beats me to it.)
As a quick guess, perhaps we split only if there's nothing other than
whitespace between the previous newline and the opening double underscore.
In the meantime, if (in Python) it's permissible to add whitespace at the
end of a line, then adding a space after __version__ should do the trick
methinks.
Cheers,
Rob
