Re: Page Aligned allocation

[David Mertens]

Hello Eyal,

Are you looking for a pure Perl way of getting aligned memory, or an XS
approach?

One really hackish, pure Perl way to do what you want is to allocate more
memory than you need---more by a page size---then give an address within
$buffer that aligns to your memory page size, which is probably 64 bytes.
This is far from ideal, but it would work.

How then would you read or modify the contents from Perl? You'd have to use
substr. (I looked for a way to utilize Perl's internal sv_chop C method,
but that only appears to work on very short strings, as documented here:
https://perldoc.perl.org/perlguts.html#Offsets).

use strict;
use warnings;
use Devel::Peek;

my $BUFFER_SIZE = 128;
my $PAGE_SIZE = 64;

# Initial memory allocation
my $buffer = "\0" x ($BUFFER_SIZE + $PAGE_SIZE - 1);
print "=== Initial PV has COW flag set:\n";
Dump($buffer);

# Must modify to remove COW flag
$buffer .= '';
print "=== After appending empty string, we have\n";
Dump($buffer);

# Find offset to use, which is also needed for substr operations
my $c_address = unpack('Q', pack('P', $buffer));
my $bytes_into_page = $c_address % $PAGE_SIZE;
my $offset = $PAGE_SIZE - $bytes_into_page;
printf "=== address is 0x%x, which is %d into a page; use offset of %d\n",
$c_address, $bytes_into_page, $offset;

# Modify the contents using substr
substr($buffer, $offset, 5) = 'a' x 5;
print "=== After substr, dump gives === \n";
Dump($buffer);

-- David

On Thu, Oct 18, 2018 at 11:39 AM Eyal BenDavid  wrote:

> Hi all,
>
> A driver exposes its API by ioctl interface.
>
> The argument for the ioctl call is a memory buffer that its address
> must aligned to OS page size.
>
> For example the allocation in C would call valloc (or posix_memalign)
>
> Simple Perl allocation of the buffer like this:
>
>  $buffer = "\0" x  BUFFER_SIZE ;
>
> is not enough since most probably the starting address of the scalar
> wont be aligned to OS page size.
>
> Is there a simple way to achieve this?
>
> Note: I convert the buffer to the C address like this:
>
>  my $c_address = unpack('Q', pack('P', $buffer));
>
> Thanks!
> Eyal
>


-- 
 "Debugging is twice as hard as writing the code in the first place.
  Therefore, if you write the code as cleverly as possible, you are,
  by definition, not smart enough to debug it." -- Brian Kernighan

Re: calling function written for perl consumption from c

[David Mertens]

Hello Bill,

In my experience, it is easier to to split the function into two pieces, a
pure C function and a thin XS wrapper. This makes it easy for other C code
to call the pure C version, while providing a Perl-accessible hook.

If you need to get that C function in XS code for an unrelated module, you
could try many approaches. One is to stash the pure C function's pointer in
an obfuscated package variable. Or you could try using ExtUtils::Depends.
Let me know if you need a bit more help as this is a little trickier.

If you want to call the xsub itself from another xsub, you'll need to
prepare the argument stack. I am pretty sure that this is just like calling
any other Perl function. For that, see details in perlcall:
http://perldoc.perl.org/perlcall.html

Finally, I wrote C::Blocks, which provides an entirely different approach
to this problem.

Hope that helps!
David

On Mon, Jun 12, 2017 at 9:05 AM, Bill Schoolfield  wrote:

> Hi,
>
> Completely new to perlxs. So far I've managed to write a few c functions
> that work fine when referenced in perl. Now I need to call one of those
> functions from another c function. I assume that is possible and I assume
> the perl stack has to be used to make the call instead of the normal c
> conventions. Is this right?
>
> Can someone point me in the right direction on how to do this?
>
> Bill
>



-- 
 "Debugging is twice as hard as writing the code in the first place.
  Therefore, if you write the code as cleverly as possible, you are,
  by definition, not smart enough to debug it." -- Brian Kernighan

Topics for C::Blocks Advent?

[David Mertens]

Hello everyone,

Many of you are probably aware that I've been attempting to write an Advent
Calendar for C::Blocks this year. While I've fallen woefully behind, I'm
having a lot of fun and I still plan on writing 24 entries overall. I have
a slate of topics to get me to 24 topics. I will be happy to alter these
planned topics if anybody had questions about C::Blocks that might serve as
good topics for an Advent entry. Please let me know!

In case you are not sure about what I'm talking about, I introduce the
library on Day 1,
http://blogs.perl.org/users/david_mertens/2016/12/cblocks-advent-day-1.html.
A reasonably useful synopsis of the whole calendar can be found in my
December archives: http://blogs.perl.org/users/david_mertens/2016/12/.

Thanks!
David

-- 
 "Debugging is twice as hard as writing the code in the first place.
  Therefore, if you write the code as cleverly as possible, you are,
  by definition, not smart enough to debug it." -- Brian Kernighan

Re: Help setting up appveyor for Windows continuous integration testing

[David Mertens]

Hello Rob,

Thanks for all of this work! I'll go through all of this at various points
throughout the week, so I may send a couple of follow-up emails later.

Thanks!
David

On Sun, Apr 24, 2016 at 8:05 PM, <sisyph...@optusnet.com.au> wrote:

> From: David Mertens
> Sent: Monday, April 25, 2016 7:16 AM
> To: perl-xs@perl.org
> Subject: Help setting up appveyor for Windows continuous integration
> testing
> Hello everyone,
>
> The specific tcc error (given at
>> https://ci.appveyor.com/project/run4flat/c-blocks/build/1.0.17#L47) is:
>>
>>  C:/STRAWB~1/perl/lib/CORE/perl.h:2708: error: ';' expected (got
>> "perl_os_thread")
>>
>
> What does line 2708  of perl.h contain ?
> I get a very similar type of error when building C-Blocks-0.03 on Windows
> (perl-5.16, mingw.org build of gcc-4.7.0):
>
> Building C-Blocks
> Creating lib/C/Blocks/PerlAPI.xs
> In file included from perl_h_QSJqVkp1.c:15:
> In file included from C:/Mingw/perl516/lib/CORE/perl.h:610:
> C:/MinGW/include/sys/types.h:27: error: ';' expected (got "__time32_t")
>
> where line 27 is:
>
> typedef __int32 __time32_t;
>
> My first thought was that perl might be doing something incompatible with
> the __int32 symbol, as there's no problem with a C program that #includes
> sys/types.h.
> But #including sys/types.h into an Inline::C script is also fine, so I
> think it's unlikely that perl is the culprit
>
> Moving on  I don't get that error when I switch to perl-522, which
> uses the gcc-4.9.2 compiler provided by mingw-w64 project (different
> vendor).
> Indeed, this compiler doesn't typedef __int32 at all.
>
> However, with this combo, the C-Blocks-0.03 build fails with:
>
> Building C-Blocks
> Creating lib/C/Blocks/PerlAPI.xs
> In file included from perl_h_Xd9p6ge7.c:15:
> In file included from C:/MinGW/perl522_64int/lib/CORE/perl.h:699:
> In file included from
> C:/_32/gcc-straw-492/i686-w64-mingw32/include/sys/types.h:13:
> In file included from
> C:/_32/gcc-straw-492/i686-w64-mingw32/include/crtdefs.h:10:
> In file included from
> C:/_32/gcc-straw-492/i686-w64-mingw32/include/_mingw.h:275:
> C:/_32/gcc-straw-492/i686-w64-mingw32/include/vadefs.h:35: error: #error
> VARARGS not implemented for this compiler
> Unable to serialize the header file
>
> The error implies that the symbol  __WIDL__ has not been defined.
> It looks to me (not entirely certain) that __WIDL__ is normally *not*
> defined for me.
>
> Does anybody know what's going on?
>>
>
> Not me.
> Note that these errors are being obtained just running "cpan -i
> C::Blocks". Appveyor is not involved (unless, of course, it's being called
> in as part of the cpan build).
>
> One other odd thing:
> I had to set the CPATH environment variable to the location of winsock2.h
> - otherwise that header could not be located.
> In both C and Inline::C scripts that header is in the default search path
> - and gets #included fine.
>
> Here's the error:
>
> Building C-Blocks
> Creating lib/C/Blocks/PerlAPI.xs
> In file included from perl_h_Yryks05Z.c:15:
> In file included from C:/MinGW/perl522_64int/lib/CORE/perl.h:3060:
> In file included from C:/MinGW/perl522_64int/lib/CORE/win32thread.h:4:
> C:/MinGW/perl522_64int/lib/CORE/win32.h:131: warning: WIN32_LEAN_AND_MEAN
> redefined
> In file included from perl_h_Yryks05Z.c:15:
> In file included from C:/MinGW/perl522_64int/lib/CORE/perl.h:3060:
> In file included from C:/MinGW/perl522_64int/lib/CORE/win32thread.h:4:
> In file included from C:/MinGW/perl522_64int/lib/CORE/win32.h:419:
> C:/MinGW/perl522_64int/lib/CORE/sys/socket.h:21: error: include file
> 'winsock2.h' not found
> Unable to serialize the header file
> lib/C/Blocks/PerlAPI.xs.PL failed at
> C:/MinGW/perl522_64int/site/lib/Module/Build/Base.pm line 2930.
>
> I just noticed that setting CPATH also removes the "WIN32_LEAN_AND_MEAN"
> redefinition warning.
> Also, win32.h #includes windows.h - and I *think* (could check if it's
> important) that should be enough to get winsock2.h #included.
> Something is not quite right.
>
> Cheers,
> Rob
>



-- 
 "Debugging is twice as hard as writing the code in the first place.
  Therefore, if you write the code as cleverly as possible, you are,
  by definition, not smart enough to debug it." -- Brian Kernighan

Announcing C::Blocks, a different way to interface Perl and C code

[David Mertens]

Hey everyone,

tl;dr: C::Blocks is a new TinyCC-based module, presently only available on
github. (1) It jit-compiles blocks of C code, building and inserting OPs
into the Perl OP tree, making invocation of C code essentially free. (2) It
will allow different blocks of C code to share function and struct
declarations, thus removing the need to always recompile perl.h, an
otherwise major cost of jit-compiling C code that can interface with Perl
and Perl data structures.

I am currently seeking help and encouragement to squash the segfaults that
currently prevent the completion of the second feature. :-)



I like Perl, but I like C, too. I would like to be able to write and call C
code from Perl in about as painless a way as possible. Inline::C is nice,
as are XS::TCC and C::TinyCompiler, but we can do better. C::Blocks is my
attempt to do better.

*Pain point 1*. C code should be a first class citizen. With XS::TCC and
C::TinyCompiler, you pass your code to the compiler via a string. With
Inline::C, you either place your code at the bottom of your script in a
__DATA__ section, or you enclose it in a string. Steffen's module is
probably the most transparent in this sense. Still, working with an
interface that requires me to compile a string to get my product feels the
same as compiling a regex from a string. This is Perl! We can do better!

C::Blocks does better by using a keyword parser hook. Blocks of code that
you want executed are called like so:

print Before cblock\n;
cblock {
printf(In cblock\n);
}
print After cblock\n;

If stdio.h is included, you get the output

Before cblock
In cblock
After cblock

Because it uses a parser hook, the C code really is inline with your Perl
code.

*Pain point 2*. Calling C code should be obvious and cheap. All three
modules discussed so far provide a mechanism for calling C functions. This
means that for a simple, small operation, I must wrap my idea into a
function one place and invoke it in another. Furthermore, if I want to
repeatedly call a block of C code in a loop, I must define that block of
code somewhere outside of the loop, potentially very far from the call
site. C::TinyCompiler suffers further because it uses a complicated and
rather slow calling mechanism.

C::Blocks solves this by extracting and jit-compiling the C code at Perl
parse time, generating an OP and inserting it into the Perl OP tree. This
means that you can insert your C code exactly where you want it and not
worry about repeated re-compiles. If you were to wrap the example given
above in a for loop, you would see how this works.

*Pain point 3*. Sharing C code should be as easy as sharing Perl code.
C::TinyCompiler provides a fairly complex mechanism to allow modules to add
declarations and symbols to a compiler context. Any string that uses that
will need to recompile those declarations, however, tempting me to
prematurely optimize by placing all of my C code in one giant string
instead of interspersed among my Perl code. Neither XS::TCC nor Inline::C
provide much (if any) automated machinery to share code.

C::Blocks provides a mechanism to share function declarations, struct
definitions, and other identifiers with other cblocks in the current
lexical scope, as well as to share them on a per-package basis. It is even
more versatile than normal Perl function scoping, allowing you to correctly
correlate functionality with lexical scope. (It is also somewhat buggy, as
discussed next.)

*Pain point 4*. Changing C code should not cost anything. Inline::C can
take seconds to recompile a changed set of C code. In contrast, there is no
cost associated with changing code when using XS::TCC and C::TinyCompiler
because they jit-compile their code. That comes at the cost, however, of
always compiling everything each time you invoke your Perl script. If your
C code needs the Perl C API, you will have to re-parse perl.h every time
you *compile* a code block, which can happen many times with each execution
of your script. Inline::C's caching mechanism provides a big win in that
respect, unless you change your code. It would be nice if we could somehow
cache the result of parsing ``#include perl.h''.

C::Blocks uses a fork of tcc that I've been working on for many months
aimed at allowing one compiler context to share its symbol table with other
compiler contexts. This is related to the previous point. The sharing
mechanism discussed in the previous point applies to preprocessor includes,
so once I have compiled a block that uses the Perl headers, I can share all
of those declarations with later compilation units, without recompiling. In
future work, I plan to store these symbol tables to disk so that they don't
even need to be re-parsed each time you run your script.



C::Blocks currently addresses, completely, pain points 1 and 2 above. It
has taken many months to hack on tcc to reach this point. I have now
encountered some segfault-causing issues when 

Re: Perl XS Beginners

[David Mertens]

I agree, PerlMonks has had some good content on XS. My problem is that I
don't know how to automatically track PerlMonks for messages. In fact, as
far as I can tell, there is no way to do so. It can, however, be done on
Stack Overflow, but that's not really a discussion forum.

That said, while exploring PerlMonks, I came up with this search that is
pretty useful at locating new threads about Perl XS stuff:

http://www.perlmonks.org/?node_id=3989;HIT=xs;HET=Re%3A%20Re^%20XSLT

There's actually a lot more discussion on XS than I had realized. :-)

If PerlMonks just had a notification system, I would be able to stay on top
of important queries, like XS, or PDL.

David


On Fri, Dec 13, 2013 at 6:20 PM, bulk 88 bul...@hotmail.com wrote:



  Date: Fri, 13 Dec 2013 18:47:17 +0100
  From: p...@pjcj.net
  To: wolfs...@gmail.com
  CC: perl-xs@perl.org
  Subject: Re: Perl XS Beginners
 
 
  

  Counting your message, I see six messages on this list in the last six
  months, and 20 in the last year. The list description, on
  http://lists.perl.org/list/perl-xs.html is A list for people interested
  in XS.
 
  FWIW, I would be very happy to see this list used for that purpose. And
  thanks for taking the initiative to do this.

 In my observations I've seen PerlMonks used more often for XS help than
 this list.




-- 
 Debugging is twice as hard as writing the code in the first place.
  Therefore, if you write the code as cleverly as possible, you are,
  by definition, not smart enough to debug it. -- Brian Kernighan

Re: Apparent linkage conflicts with dVAR; dXSARGS; when compiled using nvcc

[David Mertens]

Sorry for not getting back to this as quickly as I had hoped. My machine
with nVidia hardware is at home and I haven't had much time in the las few
evenings to play with it. This may be the case through the weekend; we
shall see.

bulk88, I was not entirely sure what PERL_NO_GET_CONTEXT did. After some
digging, I came across this article that seemed to clarify things quite a
bit, at least for me:
http://perl.active-venture.com/pod/perlguts-concurreny.html. I also did
some reading and discovered that there are ways to have thread-local data
in C, which was news to me. (I never learned the intricacies of C.)
Suddenly the behavior of dTHX; and the whole api with the _nocontext suffix
make so much more sense.

Thanks!
David


On Mon, Feb 18, 2013 at 6:34 AM, bulk88 bul...@hotmail.com wrote:

 David Mertens wrote:

 Sorry for the rapid-fire. I'm up late and the ideas are only trickling
 out.

 It appears that the code doesn't think __cplusplus is defined. That is
 odd. nvcc claims (or, it used to) that it always runs the code through a
 C++ compiler at the end of the day, so why it wouldn't have __cpluplus
 defined is beyond me.

 David

 Compiler flag on cmd line? file name auto detection (.c vs .cc or .cpp)?
 Read the compilers manual or google it. Also your XS module needs to use
 PERL_NO_GET_CONTEXT for efficiency.

 your file is called Minimal.c, per http://sbel.wisc.edu/Courses/**
 ME964/2008/Documents/**nvccCompilerInfo.pdfhttp://sbel.wisc.edu/Courses/ME964/2008/Documents/nvccCompilerInfo.pdfyour
  file will be compiled in regular C mode. Try a (untested)

 #ifndef __cplusplus #error c++ is not defined
 #endif

 for debugging.




-- 
 Debugging is twice as hard as writing the code in the first place.
  Therefore, if you write the code as cleverly as possible, you are,
  by definition, not smart enough to debug it. -- Brian Kernighan

Re: Apparent linkage conflicts with dVAR; dXSARGS; when compiled using nvcc

[David Mertens]

bulk88 -

nvcc knows about C, C++, and CUDA-C. I do not recall if it differentiates
between C and C++, though. I know that I correctly specify that Minimal.c
should be compiled as CUDA-C, but I'm not sure if I told it to do that in
the preprocessor command that produced the code I included in the above
gist. I will have to check when I get home tonight, which will be in 8
hours or so.

As for the #ifndef error idea for debugging, that's perfect. I'll give it a
run and see what it produces.

David

On Mon, Feb 18, 2013 at 6:34 AM, bulk88 bul...@hotmail.com wrote:

 David Mertens wrote:

 Sorry for the rapid-fire. I'm up late and the ideas are only trickling
 out.

 It appears that the code doesn't think __cplusplus is defined. That is
 odd. nvcc claims (or, it used to) that it always runs the code through a
 C++ compiler at the end of the day, so why it wouldn't have __cpluplus
 defined is beyond me.

 David

 Compiler flag on cmd line? file name auto detection (.c vs .cc or .cpp)?
 Read the compilers manual or google it. Also your XS module needs to use
 PERL_NO_GET_CONTEXT for efficiency.

 your file is called Minimal.c, per http://sbel.wisc.edu/Courses/**
 ME964/2008/Documents/**nvccCompilerInfo.pdfhttp://sbel.wisc.edu/Courses/ME964/2008/Documents/nvccCompilerInfo.pdfyour
  file will be compiled in regular C mode. Try a (untested)

 #ifndef __cplusplus #error c++ is not defined
 #endif

 for debugging.




-- 
 Debugging is twice as hard as writing the code in the first place.
  Therefore, if you write the code as cleverly as possible, you are,
  by definition, not smart enough to debug it. -- Brian Kernighan

Apparent linkage conflicts with dVAR; dXSARGS; when compiled using nvcc

[David Mertens]

Hello everybody -

I recently resuscitated CUDA::Minimal, a module that provides basic memory
allocation and memory transfer functionality for CUDA (i.e. video card)
parallel programming. Note, you must write your own CUDA kernels
separately; I recommend writing them in XS wrapper code, using
ExtUtils::nvcc and can answer questions if you're interested.

When using ExtUtils::nvcc, your code gets compiled through a C++ compiler.
I seem to have struck a problem with how XS handles its dVAR and dXSARGS
declarations in the Boot section and in other sections. The applicable
error can be found here:
https://github.com/run4flat/perl-CUDA-Minimal/issues/7. The actual C code
that leads to this error (generated from the XS code on my machine) is
given in this gist: https://gist.github.com/run4flat/4974702.

The problem is a linkage conflict in the declaration for Perl__notused on
lines 163 https://gist.github.com/run4flat/4974702#file-minimal-c-L163and
329 https://gist.github.com/run4flat/4974702#file-minimal-c-L374, which I
presume is embedded in dVAR or dXSARGS. However, I'm left scratching my
head as to why the linkage would differ between these two. Based on my
understanding of how extern C
workshttp://stackoverflow.com/questions/10458545/what-is-extern-linkage-and-with-c-language-linkage,
there is no difference that I can see. That is, there is no difference in
the linkage specification for the two, because the extern C on line 369
does not have curly braces and therefore only effects the declaration of
the function definition. It should not impact the declarations of any
variables.

Anybody have any idea what's causing this linkage specification problem,
and how I might fix it?

Thanks!
David

-- 
 Debugging is twice as hard as writing the code in the first place.
  Therefore, if you write the code as cleverly as possible, you are,
  by definition, not smart enough to debug it. -- Brian Kernighan

Fwd: Apparent linkage conflicts with dVAR; dXSARGS; when compiled using nvcc

[David Mertens]

Sorry, meant to reply-all.

David

-- Forwarded message --
From: David Mertens dcmertens.p...@gmail.com
Date: Mon, Feb 18, 2013 at 12:51 AM
Subject: Re: Apparent linkage conflicts with dVAR; dXSARGS; when compiled
using nvcc
To: bulk88 bul...@hotmail.com


bulk88 -

I've updated the original gist with a new file that includes the direct
C(++) code as preprocessed by nvcc. The two look identical to me. Should I
try to bump-up the verbosity of the compiler?

David


On Mon, Feb 18, 2013 at 12:25 AM, bulk88 bul...@hotmail.com wrote:

 **
 David Mertens wrote:

   Hello everybody -

  I recently resuscitated CUDA::Minimal, a module that provides basic
 memory allocation and memory transfer functionality for CUDA (i.e. video
 card) parallel programming. Note, you must write your own CUDA kernels
 separately; I recommend writing them in XS wrapper code, using
 ExtUtils::nvcc and can answer questions if you're interested.

  When using ExtUtils::nvcc, your code gets compiled through a C++
 compiler. I seem to have struck a problem with how XS handles its dVAR and
 dXSARGS declarations in the Boot section and in other sections. The
 applicable error can be found here:
 https://github.com/run4flat/perl-CUDA-Minimal/issues/7. The actual C code
 that leads to this error (generated from the XS code on my machine) is
 given in this gist: https://gist.github.com/run4flat/4974702.

  The problem is a linkage conflict in the declaration for Perl__notused on
 lines 163 https://gist.github.com/run4flat/4974702#file-minimal-c-L163and
 329 https://gist.github.com/run4flat/4974702#file-minimal-c-L374, which
 I presume is embedded in dVAR or dXSARGS. However, I'm left scratching my
 head as to why the linkage would differ between these two. Based on my
 understanding of how extern C 
 workshttp://stackoverflow.com/questions/10458545/what-is-extern-linkage-and-with-c-language-linkage,
 there is no difference that I can see. That is, there is no difference in
 the linkage specification for the two, because the extern C on line 369
 does not have curly braces and therefore only effects the declaration of
 the function definition. It should not impact the declarations of any
 variables.

  Anybody have any idea what's causing this linkage specification problem,
 and how I might fix it?

 Thanks!
 David

 I think dVAR becomes dNOOP which is
 http://perl5.git.perl.org/perl.git/blob/f6fe275c937ceb508cdcfd033ce162e1fa01989e:/perl.h#l354.
 Note a C++ and a C version of that macro. Can you post the preprocessed
 version of the .c file or have you looked at the preprocessed version for
 clues as to what happened?




-- 
 Debugging is twice as hard as writing the code in the first place.
  Therefore, if you write the code as cleverly as possible, you are,
  by definition, not smart enough to debug it. -- Brian Kernighan



-- 
 Debugging is twice as hard as writing the code in the first place.
  Therefore, if you write the code as cleverly as possible, you are,
  by definition, not smart enough to debug it. -- Brian Kernighan

Re: Typemap AV reference to char **

[David Mertens]

Bill -

Sorry I haven't replied to this earlier. I mentally book-marked it,
but never returned. I see two potential paths for discussion.

First, suppose that you want to convert the array-of-strings each time
this function gets called because char * names[] can change with each
function call. In that case the nicest handling of this would be
typemaps. I presume that you expect your caller to supply an anonymous
list of scalars, in which case you should be able to do the following:

1) Get the length of the AV.
2) Use Newx to have Perl allocate a char ** for you with the length of
the AV 
[http://search.cpan.org/~rjbs/perl-5.16.0/pod/perlguts.pod#Memory_Allocation]
3) Mark that just-allocated char ** to be freed at the end of the
current scope with SAVEFREEPV so that your own XS code doesn't have to
SAVEFREE it 
[http://search.cpan.org/~rjbs/perl-5.16.0/pod/perlguts.pod#Localizing_changes],
and you prevent memory leaks
4) Iterate through the SVs in the AV; assign each char * in your array
to the return value of SvPVbyte_nolen on the given SV
[http://perldoc.perl.org/perlapi.html#SV-Manipulation-Functions]

On the other hand, if you know that char * names[] does not change
because it's an immutable part of a Singleton or it's a Perl-side
constant, you could cache the array-of-strings in a C-level global.
If you do the former, you can populate that cache
1) with C code in your BOOT section
2) with a special internal C function that is called once from Perl
(tracking the state of the cache with Perl code)
3) with your function's C code that creates the cache when the
function is invoked if it doesn't already exist

You can free the cache
1) at object destruction, if your object is a true Singleton
2) with a special internal C function that is called from an END block
in your module
3) never, as it's not a huge memory leak

If you do a C-level global, you can use savepv for the string copy so
that you're not pointing to potentially changed PV elements of a
mutable SV.

Just my 2 cents for now.
David

On Tue, Jul 17, 2012 at 9:36 AM, Bill Moseley mose...@hank.org wrote:
 Quick follow up:

 On Tue, Jul 17, 2012 at 7:05 AM, Bill Moseley mose...@hank.org wrote:


 family( int count, char* names[], int ages[], char* family_name );


 [1] I guess I'd better see if family() holds on to the addresses of
 names[] and ages[] after returning.   Hum, if I don't know that maybe I need
 to malloc my own object so I can free those on DESTROY.   Is that right?



 Thinking about this a bit more I realized that this specific constructor is
 only called ONCE per process.  So, if I malloc names[] and ages[] I'm not
 sure it's a problem if they don't get freed.   Does that simplify things?
 i.e. don't need to malloc a struct to hold these until DESTROY.

 Second, this is my first use of XS and C++, and in the simple tests I've
 been doing XS is setting up a THIS for me.  If I need to malloc a struct
 that holds names[], ages[], and the address returned by the C++ constructo
 (my_cpp_object), if my new method returns this struct then is that what
 THIS ends up as?   That is, instead of THIS-method() I would then do
 THIS-my_cpp_object-method().

 Am I making this harder that it is?

 --
 Bill Moseley
 mose...@hank.org



-- 
 Debugging is twice as hard as writing the code in the first place.
  Therefore, if you write the code as cleverly as possible, you are,
  by definition, not smart enough to debug it. -- Brian Kernighan

Re: Typemap AV reference to char **

[David Mertens]

On Fri, Jul 27, 2012 at 11:32 AM, David Oswald daosw...@gmail.com wrote:
 I've sort of moved away from recommending the automatic string
 conversion typemap that is available to both Inline::C and
 Inline::CPP, because it turns 100% of the responsibility for
 understanding everything there is to know about Unicode over to the
 C/CPP/XS programmer.  I could be wrong in this notion, but to me the
 lure of simplicity in passing a Perl string to a c-string is deceptive
 and while presenting a slight savings in code up front, will create a
 need for a lot more hand-rolled code to support Unicode.

 In my opinion (and I'd be happy to discover I'm wrong about this),
 it's easier to pass the string-containing SV, *as an SV*, and then use
 Perl's own string-handling guts and API calls to manipulate it.  I
 realize that this somewhat defeats the performance improvement
 purpose of dropping into XS, but strings are truly one of those areas
 where Perl's native performance is as it is because it's doing so much
 (most of which has become necessary in a Unicode world).

Yeah. What one should do here depends on how much control you have
over the code. You can have Perl downgrade the string to a true byte
string if you use SvPVbyte_nolen, as I suggest, but then you loose any
unicode characters that your user may have passed in. If you don't
control your C++ library and it expects true C-style strings, then you
haven't much of an option.

David

-- 
 Debugging is twice as hard as writing the code in the first place.
  Therefore, if you write the code as cleverly as possible, you are,
  by definition, not smart enough to debug it. -- Brian Kernighan

RE: Creating an XSUB on the fly

[David Mertens]

In this case, Perl stack args. The easy solution would be to include
EXTERN.h, perl.h, etc (as with a normal XS file), but that leads to a lot
of work for the preprocessor. This work targets true jit compiling, so the
less work I throw at the preprocessor, the better. My plan here is to comb
over those header files and pull out the most useful bits into fine-grained
pieces so function writers only pull in what they need. One of those pieces
would be Perl stack manipulation.

An alternative would be to simply return a function pointer that function
writers could wrap with a Foreign Function Interface caller. A few months
ago I was pushing Reini and others on their Ctypes work, and Joel Berger on
Alien::Base so I could distribute libffi as a Ctypes dependency. Actually,
I think this would be a simpler solution to the problem rather than digging
around the Perl XS headers. For me, work on both fronts has stopped as my
attention has shifted to other projects.

David
On Jul 21, 2012 2:01 PM, bulk 88 bul...@hotmail.com wrote:



  Date: Sat, 21 Jul 2012 06:28:17 -0500
  Subject: Re: Creating an XSUB on the fly
  From: dcmertens.p...@gmail.com
  To: bul...@hotmail.com
  CC: perl-xs@perl.org
 
  bulk88 -
 
  That's exactly what I needed! Thank you! Now I have to figure out how
  to add args processing to this jit-compiled C function, but that's a
  different problem. :-)
 
  David

 C stack args or Perl stack args?

Re: Stringwise comparisons (Inline::C)

[David Mertens]

Now now, let's not be too hasty. Using an undocumented function is
generally not advised, and documenting how to use an undocumented function
is generally frowned upon. As such I see two possibilities here:

   1. Use pp_scmp and document it with lots of caveats indicating that the
   function is not part of Perl's public API.
   2. Ask p5p if pp_scmp (and any other handy UTF8 string functions) can be
   added to the public API, as well as how you might go about doing that. The
   difference between public and private API for Perl, as far as I know, is
   just the decision to document the thing. So this probably means simply
   adding the docs for the function. I could help out with this.

Would you like me to email p5p and get their ideas about it?

David

On Mon, May 7, 2012 at 10:41 PM, Sisyphus sisyph...@optusnet.com.au wrote:


 - Original Message - From: David Oswald

  But for this:

 bsearch_str $needle, @haystack

 I wouldn't be calling out to a subroutine implemented in Perl, but
 rather, to a Perl internal built-in that does comparisons.

 The perl-xs@perl.org mailing list has been completely silent on my
 request for suggestions.  Max Maischein (Corion) has been helpful,
 with this suggestion over at PerlMonks:

   Corion says: I think you'll need to call the appropriate OP,
   which seems (wildly guessing) to be pp_scmp in pp.c.
   Or appropriate the code from there.


 Yep, if you can't re-structure the code to call a perl sub then, afaik,
 you'll have to do as Corion has suggested.


  Once I figure it out it actually may make a good addition to the
 Inline::C cookbook.


 Yes, I think so.

 Cheers,
 Rob




-- 
 Debugging is twice as hard as writing the code in the first place.
  Therefore, if you write the code as cleverly as possible, you are,
  by definition, not smart enough to debug it. -- Brian Kernighan

Re: Stringwise comparisons (Inline::C)

[David Mertens]

To those not CC'd:

I posted a question on p5p and Nicholas Clark pointed out that sv_cmp looks
like it's *exactly* what you need, both as a general comperator, and as a
means to implement eq, lt, gt, etc. Does this look right?

David

On Tue, May 8, 2012 at 6:26 AM, David Mertens dcmertens.p...@gmail.comwrote:

 Now now, let's not be too hasty. Using an undocumented function is
 generally not advised, and documenting how to use an undocumented function
 is generally frowned upon. As such I see two possibilities here:

1. Use pp_scmp and document it with lots of caveats indicating that
the function is not part of Perl's public API.
2. Ask p5p if pp_scmp (and any other handy UTF8 string functions) can
be added to the public API, as well as how you might go about doing that.
The difference between public and private API for Perl, as far as I know,
is just the decision to document the thing. So this probably means simply
adding the docs for the function. I could help out with this.

 Would you like me to email p5p and get their ideas about it?

 David


 On Mon, May 7, 2012 at 10:41 PM, Sisyphus sisyph...@optusnet.com.auwrote:


 - Original Message - From: David Oswald

  But for this:

 bsearch_str $needle, @haystack

 I wouldn't be calling out to a subroutine implemented in Perl, but
 rather, to a Perl internal built-in that does comparisons.

 The perl-xs@perl.org mailing list has been completely silent on my
 request for suggestions.  Max Maischein (Corion) has been helpful,
 with this suggestion over at PerlMonks:

   Corion says: I think you'll need to call the appropriate OP,
   which seems (wildly guessing) to be pp_scmp in pp.c.
   Or appropriate the code from there.


 Yep, if you can't re-structure the code to call a perl sub then, afaik,
 you'll have to do as Corion has suggested.


  Once I figure it out it actually may make a good addition to the
 Inline::C cookbook.


 Yes, I think so.

 Cheers,
 Rob




 --
  Debugging is twice as hard as writing the code in the first place.
   Therefore, if you write the code as cleverly as possible, you are,
   by definition, not smart enough to debug it. -- Brian Kernighan




-- 
 Debugging is twice as hard as writing the code in the first place.
  Therefore, if you write the code as cleverly as possible, you are,
  by definition, not smart enough to debug it. -- Brian Kernighan

Creating an XSUB on the fly

[David Mertens]

Hello everybody -

I'm looking into hooking the Tiny C Compiler into Perl. Among other things,
it can JIT-compile a string of C code and let you get a function pointer
(or a pointer to any other structure of the compiled code that is globally
scoped) that you can subsequently call from your original C program. I
would like to try to use this to build Perl CVs/XSUBs, but I see no
interface for creating an XSUB that is not installed into a package.

The easy work around is to have a designated package into which all such
xsubs are installed and simply use newxs (see
http://perldoc.perl.org/perlembed.html). The name given to the Perl-side
function would be composed from a hashing algorithm of some sort to avoid
collisions. Still, I would really like to create simple function references
without having to create the entry in the package.

Thoughts? Wisdom?
David

P.S. TCC compiles for i686 and ARM hardware. It can create executables for
Windows and Linux (with some work on porting it to Mac OSX). I realize this
is a limited set of platforms, but I suspect a lot of users would still be
able to benefit from it and I would like to get this working. Heck, if this
is successful, it may draw enough developers into the TCC project to get it
working on other hardware.

-- 
 Debugging is twice as hard as writing the code in the first place.
  Therefore, if you write the code as cleverly as possible, you are,
  by definition, not smart enough to debug it. -- Brian Kernighan

Re: ExtUtils::ParseXS can generate invalid C++

[David Mertens]

Hey David -

Looking over the perl-xs mailing list history, I see somebody inquiring
about C++ and XS, with a suggestion to look at this: p3rl.org/ExtUtils::XSpp

I'm not suggesting that you switch over to using EU::XSpp, but I thought
the additional point of reference might be helpful. Perhaps adding a link
in the See Also section would be good, too.

David

On Fri, Feb 24, 2012 at 3:51 AM, Sisyphus sisyph...@optusnet.com.au wrote:


 - Original Message - From: David Oswald daosw...@gmail.com
 To: perl-xs@perl.org
 Cc: inl...@perl.org
 Sent: Friday, February 24, 2012 7:01 PM
 Subject: ExtUtils::ParseXS can generate invalid C++



  [ This message is being sent to perl-xs@perl.org seeking suggestions,
 with a CC to inl...@perl.org as an FYI. ]

 We have a situation with Inline::CPP wherein the module is failing to
 pass smoke testing when the compiler is g++ version 4.4.3 (though I
 think the issue probably goes back to 4.2.1).  Here is an example of
 two failure reports:

 http://www.cpantesters.org/**cpan/report/35702916-59fc-**
 11e1-8417-42773f40e7b5http://www.cpantesters.org/cpan/report/35702916-59fc-11e1-8417-42773f40e7b5


 Can't reproduce the problem.
 I've tried gcc versions 4.7.0, 4.5.2, 4.4.7, and 4.4.3 on various builds
 of perl (all on MS Windows).

 In each case I was building 0.34_002, with Inline-0.50_01 and
 Parse-RecDescent-1.67006 - same as the above tester.

 I reckon I've seen similar before, and just assumed it must be the result
 of some locale setting.

 Cheers,
 Rob




-- 
 Debugging is twice as hard as writing the code in the first place.
  Therefore, if you write the code as cleverly as possible, you are,
  by definition, not smart enough to debug it. -- Brian Kernighan