Re: Storable Stream - Module namespace help
On Mon, Aug 9, 2010 at 12:41 PM, Marco Neves perl-module-auth...@knowhunter.cjb.net wrote: On Monday 09 August 2010 20:59:50 David Nicol wrote: On Sun, Aug 8, 2010 at 4:58 AM, Marco Neves [ModAuthors] perl-module-auth...@knowhunter.cjb.net wrote: Hello, In need for a way to transfer a large amount of small datastructures I created a module that stores and retrieves a stream of storables in a file. I'm confused -- a stream can't be stored in a file, because it is continually generated. Also, what is different about this freezer/thawer that makes it superior to, for instance, YAML or compressed Data::Dumper dumps? That is, for what general class of use cases would one prefer the Neves data marshalling system to the available others? Also, if you're reusing other things in it -- you say storables are you doing something like prefixing length and colon to the output of Storable and jamming a bunch of those together? -- the underlying technology and then stream might make sense, especially if it can take an open handle instead of a file name in its constructor. Such a thing could be nice for creating an abstracted pipe between distributed processes for instance. Hi, Yes, the module does store a lot of storables on a single file, and yes, it accepts an open handler as constructor. I written it to transfer large amounts of small data structures between servers. I'm thinking to use the name Archive::StorableStream for the module. The git repository is in: http://git.magick-source.net/perl5/storablestream Before you tell it, I know, none of the docs are still done. FWIW, have you considered how to deal with what happens if your chunks don't match the size prefix? If I read your protocol correctly, you are sending the length of the following blob as ASCII digits terminated by a \0. $blob = Storable::nfreeze( ... ) $packed = length( $blob ) . \0$blob and to read: $length = 0 + $packed; Storable::thaw( substr $packed, length $length ) I dunno your objection to YAML but it has the nice property that it provides context for helping to recover from mis-transmitted documents in the stream by virtue of having a nested-ness marker (the indentation) and both starting and ending document tokens (--- and ...). Further, I would highly discourage you from naming anything using Storable with Archive because absolutely no one should ever consider actually *storing* a Storable blob. If you keep one around then you'll be tempted to read it in a client version that didn't match what you created it with. That way lies madness. Josh
Re: Storable Stream - Module namespace help
On Sun, Aug 8, 2010 at 2:40 AM, theMage [Marco Neves] perl-c...@knowhunter.cjb.net wrote: Hello, In need for a way to transfer a large amount of small datastructures I created a module that stores and retrieves a stream of storables in a file. I created it in a private Namespace using in the company I worked for but, with they permission, and because I think this may be useful for someone else, I want to publish it on cpan. I'm finishing documentation and tests, before uploading it to CPAN, but I don't know exactly which namespace to use. For me, the obvious namespace would be Storable::Stream (and eventually Storable::Stream::GZip), but as Storable is part of core, I think we are not expected to publish modules in that namespace. So, which name should I use for this small module? The namespace ought to be just fine. The package Storable is owned by core but Storable::Stream isn't. FWIW though, I am provoked by SO MUCH PAIN transferring Storable objects between processes and servers that I feel like I must warn you against ever actually using Storable in IPC. I've had so many problems when one of either client or server for Storable-trading IPC wanted to change. Have you tried upgrading or downgrading either side of the transaction to a new, potentially incompatible version of Storable yet? I'd feel much better suggesting you consider streaming with YAML or JSON or something hand-rolled but anything but Storable. Consider that maybe your next client will want to be implemented in Ruby but now won't be able to read the data. BTW, where's your repo? github? I'd like to take a gander. Josh
Re: Which Build Module?
On Tue, Jul 6, 2010 at 12:50 PM, David Golden xda...@gmail.com wrote: On Tue, Jul 6, 2010 at 3:40 PM, Joshua ben Jore twi...@gmail.com wrote: FWIW, even the most up-to-date CPAN.pm comes broken out of the box for configure_requires because we say we prefer the YAML module to read the configure_requires out of $whatever.yml and don't fail-over to use the thing that's actually packaged with perl. That's incorrect. CPAN.pm prefers Parse::CPAN::Meta and will attempt to parse with it. If it's not available (or the parse fails), it will fall-back to the CPAN-defined preferred YAML module. Parse::CPAN::Meta is read-only and CPAN's preferred YAML module is so that it can write (and read back) data as it processes things. What? That's certainly not what happens for me. I just ensured I had no ~/.cpan or ~/perl5, installed perlbrew, then installed new perl-5.12.1. When I ran the newly created cpan client, my default `o conf yaml_module' is YAML and not Parse::CPAN::Meta. I just tried requesting an upgrade to CPAN and my client is already up to date (1.94_56). Josh
Re: Which Build Module?
On Wed, Jul 7, 2010 at 9:01 AM, David Golden xda...@gmail.com wrote: On Wed, Jul 7, 2010 at 10:01 AM, Joshua ben Jore twi...@gmail.com wrote: What? That's certainly not what happens for me. I just ensured I had no ~/.cpan or ~/perl5, installed perlbrew, then installed new perl-5.12.1. When I ran the newly created cpan client, my default `o conf yaml_module' is YAML and not Parse::CPAN::Meta. I just tried requesting an upgrade to CPAN and my client is already up to date (1.94_56). There are two situations in which CPAN.pm reads YAML files. (1) A distribution contains META.yml or produces a MYMETA.yml after running Build.PL/Makefile.PL -- in this case, Parse::CPAN::Meta is preferred and CPAN.pm will fall back to the configured 'yaml_module' setting if Parse::CPAN::Meta is not installed or if Parse::CPAN::Meta throws an error during parsing. (2) CPAN.pm is reading distroprefs or its own cached data -- in this case CPAN.pm uses the configured 'yaml_module' (which is also used to create the cached data files) o_O Dear perl-qa, I didn't think this would work because it never has, but boy was I wrong. I built perl-5.12.1 and was able to use a configure_requires using module directly out of the box. Seriously, I don't know what's been wrong every other time prior to just now. cd ~josh rm -rf perl5 .cpan curl -LO http://xrl.us/perlbrew chmod +x perlbrew ./perlbrew install perl5/perlbrew/bin/perlbrew init source perl5/perlbrew/etc/bashrc perlbrew install -f -D=DEBUGGING=both -D=use64bitint -D=usethreads perl-5.12.1 perlbrew switch perl-5.12.1 cpan Data::Dump::Streamer Success. Er, wtf? Josh
Re: Which Build Module?
On Tue, Jul 6, 2010 at 10:39 AM, Eric Wilhelm enoba...@gmail.com wrote: # from David Cantrell # on Monday 05 July 2010 06:31: consider Module::Build, but be aware that for a great many users it will introduce an extra dependency, as it was not in core until 5.10.0. First, the great many is shrinking quickly, at least if Debian's oldstable is any indicator of what is old (etch includes 5.10 and will likely become oldstable this year.) At that point, I would consider 5.8.8 as obsolete and anything older as ridiculously obsolete. Second, 'configure_requires' takes care of the dependency automatically (given a non-obsolete CPAN client.) FWIW, even the most up-to-date CPAN.pm comes broken out of the box for configure_requires because we say we prefer the YAML module to read the configure_requires out of $whatever.yml and don't fail-over to use the thing that's actually packaged with perl. I'd like to have this fixed both for 5.12.2 and in CPAN.pm but I've been busy and haven't spent time corralling those cats. Perhaps cpanminus is better? I haven't tried. Josh
EU::MM LIBS winnowed too-early at .PL-time?
Hi,
In Judy I have a PREREQ_PM using Alien::Judy which by it's action
installs libJudy.so and Judy.h. I also have a LIBS and INC which refer
to the thing being installed. It appears that my Makefile.PL's LIBS
argument is being tossed because the specified directories and their
contents don't exist but *will* by the time it will be actually used.
PREREQ_PM = {
'Alien::Judy' = 0
},
INC = -I$Config{sitearch}/Alien/Judy,
LIBS = -L$Config{sitearch}/Alien/Judy -lJudy
Can I get EU::MM to stop filtering my LIBS for me?
Josh
Re: Suggestions needed for a Module Name similar to DBIx::OO::Tree
On Thu, Dec 31, 2009 at 5:59 PM, ian docherty i...@iandocherty.com wrote:
Hi
I have written a Moose Role which can be used together with DBIx::Class to
implement 'nested sets' type of hierarchy trees (another Perl Module that
implements this is DBIx::OO::Tree)
I think these are typically named MooseX::Role::${your_module}.
Josh
Re: Structured Changelog?
On Sat, Jun 27, 2009 at 8:42 AM, Joshua ben Joretwi...@gmail.com wrote: Last year I started seeing change logs in perl modules that looked more YAMLish. Is there any spec out there on this? I'm currently just copying input from `git log --pretty=oneline' into my Changes file and as of this moment, including the git repo URL. Is there any external standard or pattern I can conform to? Thanks!
Structured Changelog?
Last year I started seeing change logs in perl modules that looked more YAMLish. Is there any spec out there on this? I'm currently just copying input from `git log --pretty=oneline' into my Changes file and as of this moment, including the git repo URL. Is there any external standard or pattern I can conform to? Revision history for Perl extension App::Perldoc::Search at http://github.com/jbenjore/App-Perldoc-Search/commits/master. 0.02 Sat Jun 27 08:33:00 2009 304a4464b38b71f10f34c8866da7e3bab1369d3e Clean up dependency list 0.01 Fri Jun 26 01:10:54 2009 9e60c318ebd5543bb342725f7f4834b2502e65f5 Add META.yml c69869d09c07115b04120ce0d294011940d76321 0.01 Josh
Re: lexical warnings question
On Thu, Jun 18, 2009 at 2:11 PM, Jonathan Rockwayj...@jrock.us wrote:
* On Tue, Jun 16 2009, Bill Ward wrote:
I'm more interested (at $JOB) in global warnings, actually. Of course
one can enable those with $^W or perl -w and I do, but developers
ignore the warnings all too often. Many of our core modules were
written without warnings enabled, and people are slow to fix those
warnings. So, we want to make warnings fatal in the development
environment to force developers to fix those niggling uninitialized
value warnings that are all over the place in our log files.
Honestly, I think this is worth bringing up on p5p. I think it should
be possible to convince perl to give user code control over the hints
hash as each scope is compiled.
I tried the obvious technique of:
package forcewarn;
use strict;
use B::Hooks::OP::Check::StashChange;
our $id = B::Hooks::OP::Check::StashChange::register(sub {
my ($new, $old) = @_;
warn compiling $new;
warnings-import( FATAL = 'all' );
});
1;
And then use forcewarn before any of my code compiled, but the
warnings-import only seems to affect the calling scope, not the scope
currently being compiled.
Without reading B::Hooks::OP::blah to know how it works, I notice
you're calling import from the forcewarn package, not the package you
want to affect. Consider:
eval
package $target_package;
warnings-import( FATAL = 'all' );
instead. Also, you should require warnings if you're going to call a
method on it. I know you probably already have it loaded but it's good
practice.
Josh
Re: lexical warnings question
Uh... yeah, maybe.
Reading warnings.pm's import is revealing. Appears the current
package doesn't matter at all - it just writes to ${^WARNING_BITS}
which is global because it is one of those ${^...} variables.
I dunno. I guess if I knew this B::Hooks thing I could answer
reasonably. It's likely just a story of making sure you're writing to
the variable at the right time. Might not be sufficient hook to get it
then or this Hooks thing is the wrong time.
Josh
On 6/27/09, Hans Dieter Pearcey hdp.perl@weftsoar.net wrote:
On Sat, Jun 27, 2009 at 08:46:12AM -0700, Joshua ben Jore wrote:
Without reading B::Hooks::OP::blah to know how it works, I notice
you're calling import from the forcewarn package, not the package you
want to affect. Consider:
eval
package $target_package;
warnings-import( FATAL = 'all' );
instead. Also, you should require warnings if you're going to call a
method on it. I know you probably already have it loaded but it's good
practice.
Since the effect of warnings-import is lexical, this won't do anything,
will
it?
hdp.
How to get search.cpan to show script documentation?
In App-Perldoc-Search, I have a script and two helper classes. Search.cpan.org shows only the class's documentation and ignores that I have a script. The script is the most important entry point. How do I hint that it should render the POD in the script? Build.PL Changes MANIFEST MANIFEST.SKIP README lib/App/Perldoc/Search.pm lib/App/Perldoc/Search/_Parser.pm scripts/perldoc-search t/App-Perldoc-Search.t I let h2xs make my base dist for me, then edited to taste. Josh
Re: managing your perl
On Sun, Apr 12, 2009 at 11:20 PM, Eric Wilhelm enoba...@gmail.com wrote: # from Joshua ben Jore # on Sunday 12 April 2009 20:06: http://www.perlfoundation.org/perl5/index.cgi?perl_best_admin_practices It may be a best practice to maintain your own perl but having just done this at work, it's a massive time sink. Our new platform at work is an Ubuntu mod_perl system with 208 CPAN modules. We produced a .deb for perl+modules and another for mod_perl Only *one* .deb for perl and all of the modules? Yes. One deb. We've whittered about making more individual .debs but it's so annoying to create them that I think we'll likely not change. We don't go through this effort for Ruby+gems because currently we think we can apt-get install it and gem install the rest without conflicting since we deem it unlikely anything on the current system actually cares super-hard about it. but it took us several weeks to do it and we had to learn a bunch about how to author for Debian. It was painful and I don't recommend it for most people. You don't have to do it as Debian packages. Simply installing from source and then building a full set of modules has never taken me more than a few hours. /Now/ I can rebuild the set inside of half an hour which is /actually/ about 28 minutes too long. More highly annoying things are CPAN.pm being unable to install from a set of local tarballs. I tried for a bit to manufacture some small bits of program to create a local CPAN repo and had some success but not enough that my sysadmin incorporated it. If I figure out a recipe that works nicely, I'll share it. Josh
Re: managing your perl
On Mon, Apr 13, 2009 at 10:22 AM, Jonathan Rockway j...@jrock.us wrote: * On Mon, Apr 13 2009, Joshua ben Jore wrote: /Now/ I can rebuild the set inside of half an hour which is /actually/ about 28 minutes too long. More highly annoying things are CPAN.pm being unable to install from a set of local tarballs. I tried for a bit to manufacture some small bits of program to create a local CPAN repo and had some success but not enough that my sysadmin incorporated it. Uh, CPAN::Mini and CPAN::Mini::Inject? I wish I could recall why CPAN::Mini::Inject didn't seem appropriate. I'll be redoing this thing anyway. I looked into Andreas' distroprefs but never found enough time to suss it. I have a blind spot toward CPANPLUS and didn't even consider it. Separately, I want to also solve this problem for ruby 1.8.6 + some gems. :-/ So, later, I guess I'll just report Huzzah! and that it all works nicely when I remember to include all the good ideas already written. Josh
Re: managing your perl
On Thu, Apr 9, 2009 at 2:20 PM, Eric Wilhelm enoba...@gmail.com wrote: # from Jonathan Rockway # on Thursday 09 April 2009 13:30: * On Thu, Apr 09 2009, Bill Ward wrote: How about you write a how to manage Perl on your system doc and get it into the core as a new perlxyz perldoc file then. That is a very good idea. Of course, the people that will update to a version of perl that includes this doc probably won't need it ;) If everyone can get past the idea that something non-core is somehow unusable, a fine document already exists http://www.perlfoundation.org/perl5/index.cgi?perl_best_admin_practices It may be a best practice to maintain your own perl but having just done this at work, it's a massive time sink. Our new platform at work is an Ubuntu mod_perl system with 208 CPAN modules. We produced a .deb for perl+modules and another for mod_perl but it took us several weeks to do it and we had to learn a bunch about how to author for Debian. It was painful and I don't recommend it for most people. Some of the stuff that made it most annoying were CPAN modules that had prompts or interacted with things outside the installation fake root. Josh
Re: Perl Critic and (honest) hash references
On Wed, Feb 18, 2009 at 8:05 AM, Roger Hall raha...@ualr.edu wrote:
All,
...
Do I have to redesign the module to get past this syntatical doppelganger?
If you redesigned, replacing your hash with an array would be harder
to typo, faster, smaller, not as nice to dump with Dumper, and harder
for 3rd parties to extend.
From:
$obj-{foobar} = ...
To:
use constant FOOBAR = 0;
$obj-[FOOBAR] = ...
Harder to typo:
$obj[FOBAR] # syntax error
Faster:
$obj-[0] is faster than $obj-{foobar}
Smaller:
Peek with Devel::Peek to get an idea
Not as dumpable:
You get [ ... ] instead of { ... }
Unless you implement some kind of -hashify method.
It's a mixed bag, on the whole I prefer array objects to hash objects
for the built-in typo detection. I need it.
Josh
Re: Perl Critic and (honest) hash references
On Mon, Mar 2, 2009 at 12:22 PM, Nicholas Clark n...@ccl4.org wrote: On Mon, Mar 02, 2009 at 10:23:38AM -0800, Bill Ward wrote: Personally I always use hashes for objects. Hashes are pretty fast in Perl, especially when there aren't many keys, so I don't think the benefits of using arrays are worth it. The risk of typos is pretty small, and the Hash lookup should be O(1), independent of number of keys. Of course, a hash with more keys uses more memory, but so does an array with more elements. I once found some very fast code varying in something I'm guessing was O(n) on the length of the keys. I've occasionally wished I could get static lookups to compile with the hashed I32 already stashed. Josh
Re: Readonly::XS failure under 5.10
On Sat, Feb 7, 2009 at 6:33 AM, Chris Dolan ch...@chrisdolan.net wrote: Eric, There's an utterly trivial error in Readonly::XS that prevents it from compiling under 5.10. There have been three RT reports about this problem since Oct 2007. The fix is incredibly simple: just put parentheses around the argument to Cart::croak. Eric++. Fixed it a few hours ago.
Re: Proposal: Test::Refcount
On Tue, Jul 15, 2008 at 6:08 AM, Paul LeoNerd Evans [EMAIL PROTECTED] wrote: I've relented now, and written a Devel::Refcount; see http://search.cpan.org/~pevans/Devel-Refcount-0.01/lib/Devel/Refcount.pm Your function is already core. *Devel::RefCount::refcount = \Devel::Peek::SvREFCNT; Josh
Re: Licenses of CPAN modules
On Wed, Jun 4, 2008 at 1:50 AM, Gabor Szabo [EMAIL PROTECTED] wrote: - CPAN distributions should have a LICEN[CS]E file with the exact text of the license in it Dual licensed modules are not accommodated for by a single file. The GPL and Artistic license seem to have normative filenames like Artistic, Artistic.txt, Copying and Copying.txt, etc. I would not want to lose that. To be specific WRT to license, I'm about to post something to CPAN with the files Artistic-v1.txt and gpl-v3.0.txt where I was specifically asked by a release lawyer to use filenames like that. That may not have been a hard requirement but I didn't go back to negotiate for wiggle room or to find out the reason. Josh
Re: How to challenge a cpan-testers test result?
On Mon, May 26, 2008 at 4:29 PM, David Golden [EMAIL PROTECTED] wrote:
On Mon, May 26, 2008 at 6:10 PM, Eric Wilhelm
[EMAIL PROTECTED] wrote:
Forget about fairness... Can we please just have some way to adjust the
signal/noise ratio?
Would you like to be copied on all PASS reports? That would improve
your perception of the ratio. :-)
It's no different than requiring Extutils::MakeMaker to be installed, so
it's not really a bootstrapping problem as much as one of outdated
assumptions, but in either case it seems to be the norm to test with an
outdated (they call it stock toolchain), so the signal/noise ratio is
roughly zero.
That's only a failing of CPANPLUS/CPAN::YACSmoke.
CPAN::Reporter::Smoker requires Module::Build because I think a smoke
test should assume a relatively modern toolchain. In my opinion,
smoke testing bare metal installs of years-old Perl doesn't add a
lot of value.
I figure a bare metal 5.8.{latest,latest-1} ought to be standard
PASS/FAIL targets. Requiring smokers to have upgraded toolchains means
you're not testing ordinary perl. I'm only guessing but I think it's
more common to have a stock toolchain than one that's been fully
patched.
Josh
Re: YA CSV parser
On Nov 25, 2007 10:59 AM, Jim Schneider [EMAIL PROTECTED] wrote:
I wrote a streaming CSV parser yesterday because I couldn't find a CSV
parsing module that does what I want (despite the plethora of available
choices). The parsing rules are pretty simple:
1) At the start of a field, if you find a quote string, eat the quote
string and go to the state that handles quoted strings. If you find a
separator, add the current field (which is blank) to the line, and start
over at this step. If you find an end of line string, add the current
field to the current target line, push the target line onto the list of
parsed lines, and start over. For anything else, go to the state that
handles unquoted strings.
2) In the state that handles quoted strings, search the string
sequentially for the first instance of the quote string. Add the string
up to that point (not including the quote) to the string. If what
immediately follows is another quote string, start this step over.
Otherwise, go to the unquoted string state. In the event no quote
string is found, append the remainder of the string being processed to
the current field. Parsing of the next chunk of data will resume in
this state.
3) In the state that handles unquoted strings, search the string
sequentially for either the first instance of the separator string, or
the first instance of the end of line string. If neither is found,
append what's in the string being processed to the current field, and
note that the parser will resume in this state. Otherwise, append the
part of the string up to, but not including, the separator or end of
line that was found to the current field, then, append the current field
to the current target line. If the found string was the end of line
string, append the current target line onto the list of parsed lines,
too. Then, return to the initial state.
This set of rules happens to produce results that match how Microsoft's
Excel handles CSV files. As you may have determined from my use of
separator string, quote string, and end of line string, each of
these entities is a string (',', '', and \r\n, by default). They
also happen to be parameters, so you can parse other simple text
formats, too. For example, to parse a standard /etc/passwd file, you
could use :, \0, and \n as the separator, quote, and end of line
strings.
If anyone knows of a module on CPAN that does all this, please let me
know. Otherwise, I'll upload my module sometime in the next week or two.
Didn't you just reinvent Text::CSV_XS? The only tweak required is
saying binary to enable the use of newlines inside quoted fields.
-new({
binary = 1,
# defaults
eol = qq(\r\n),
sep_char = q(,),
quote_char = q(),
escape_char = q(),
})
Josh
Re: Incompatible change in blead perl for Safe.pm?
On 8/16/07, Burak Gürsoy [EMAIL PROTECTED] wrote: Hi, I have a module named Text::Template::Simple ( http://search.cpan.org/dist/Text-Template-Simple/ ) and one of it's tests (t/05-safe) is dying under 5.9.5: http://www.nntp.perl.org/group/perl.cpan.testers/2007/08/msg576957.html http://www.nntp.perl.org/group/perl.cpan.testers/2007/07/msg546071.html http://www.nntp.perl.org/group/perl.cpan.testers/2007/05/msg499523.html my default permit list is like this: my @permit = qw(:default require); everything is fine under $] 5.9.x but it dies with this error under blead: 'caller' trapped by operation mask I couldn't locate an entry for this change in the 5.9.5 change log. Can someone give me info on this subject? Or shall I just add caller (and anything it requires) to my permit list? caller() is a less-safe kind of operation because it now returns a hash ref of the current lexical pragmas. I don't recall why this new behavior warranted its removal from the default list of safe opcodes. Josh
Re: Incompatible change in blead perl for Safe.pm?
On 8/16/07, Dominique Quatravaux [EMAIL PROTECTED] wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Joshua ben Jore wrote: caller() is a less-safe kind of operation because it now returns a hash ref of the current lexical pragmas. I don't recall why this new behavior warranted its removal from the default list of safe opcodes. Maybe because if it returns *refs*, the evil guy could then alter what they point to? It isn't clear that modifying the reference does anything. The reference is constructed in the moment that it is asked for. It can contain only strings. I wouldn't swear that it is impossible to have a change be reflected in the data stored in the optree but I suspect it is unlikely. Josh
Re: peer review for first CPAN module? (JavaScript minification)
On 5/25/07, Dr.Ruud [EMAIL PROTECTED] wrote: Joshua ben Jore schreef: Adriano Ferreira: Peter Michaux: I'm writing a new version of JavaScript::Minification on CPAN. This is my first CPAN module and first Perl project! If someone is willing to take a look to see if I've done something terribly wrong packaging the code I would greatly appreciate any feedback. Currently the new module is in a subversion repository. If you have subversion you should be able to check it out and test it with svn co http://dev.michaux.ca/svn/random/JavaScript-Minifier cd JavaScript-Minifier perl MakeFile.PL make test Some comments on packaging: * it may be a good idea to add a LICENSE parameter to Makefile.PL (supported by ExtUtils::MakeMaker = 6.31) * you might like to add POD and POD coverage tests (for CPANTS' sake) Some comments on code: * it does not look like Perl code, but C code translated literally to Perl. That's because this is a translation of the C program, jsmin. http://javascript.crockford.com/jsmin.html The code contains the strange '\000'. I see basically 3 steps: 1. respect strings 2. remove comments 3. compress everything else You may want want to convert from UTF16 to perl native strings, then convert back. JavaScript is defined as being stored in UTF16 but that's kind of unusual elsewhere. Isn't it? Josh
Re: Module Proposal: Parse::Reversible
On 4/20/07, Paul LeoNerd Evans [EMAIL PROTECTED] wrote:
The requirement for this module came about intially because I was
thinking about how to handle virtual URLs in websites; for example:
/photos/album12/photo17.jpg
This will fetch the 17th photo from the 12th album, by whatever method
internally is used. Internally, we need to know these values. Trying to
make as generic a system as possible, I came up with the idea that
somewhere in site config, would live a regexp-like pattern to explain
how to parse that. This pattern needs to be reversible - the logic that
generates pages has to be able to construct URLs that give paths to the
files in question.
The format I came upon would look like this:
'/photos/album${ALBUM:\d+}/photo${PHOTO:\d+}.jpg'
This is just named capturing, isn't it? In perl 5.10:
qr!/photos/album(?ALBUM\d+)/photo(?PHOTO\d+).jpg!;
$url = /photos/album$+{ALBUM}/photo$+{PHOTO}.jpg;
With my hacky CPAN module for earlier perls:
use Regexp::NamedCaptures;
my %photo;
qr!/photos/album(?\$photo{ALBUM}\d+)/photo(?\$photo{PHOTO}\d+.jpg!;
$url = /photos/album$photo{ALBUM}/photo$photo{PHOTO}.jpg;
Josh
Re: echo die Makefile.PL
On 4/16/07, Eric Wilhelm [EMAIL PROTECTED] wrote: If META.yml says you need Module::Build and I don't provide a Makefile.PL, wouldn't that be a good time to install Module::Build? Should I just put a die in Makefile.PL to avoid this silly we're obviously missing the required build system but we'll try to pretend anyway stuff? http://www.nntp.perl.org/group/perl.cpan.testers/2007/04/msg461340.html So now you'll put systems in the Ah, I have a Makefile.PL. I'll use it! Arrgh! It died. This distribution fails, obviously! mode which is very similar to where you are now. I've wondered about this - can you get a Makefile.PL to prereq Module::Build so CPAN.pm installs it, then transfer control over to M::B? Josh
Re: echo die Makefile.PL
On 4/17/07, Eric Wilhelm [EMAIL PROTECTED] wrote: # from Paul LeoNerd Evans # on Tuesday 17 April 2007 02:14 am: I ran into similar issues; I've discussed them earlier on this list. Yep. # from David Golden on Sunday 11 March 2007 05:43 am: This bug is the result of CPANPLUS trying to be clever and falling back to generating a Makefile.PL from a Build.PL. My thinking was that exposing the bug where it lives is probably a better long-term approach. The suggestion people made was to have M::B produce a 'traditional'-style makefile as well. So at least M::B-deprived systems might have a good go. My problem with the create_makefile_pl = 'traditional' is that it doesn't install Module::Build, so we're not making any progress. If you have a custom build subclass, you might as well put a die in Makefile.PL. The passthrough seems much more sane because it actually installs Module::Build, yet nobody seems to encourage its use. Why? The passthrough, IIRC, usually broke stuff. It might be better now that M::B has had another year of bugs shaking out. The problem with having a Makefile.PL that dies is now you've presented an interface that can't be queried against. Consider this like duck typing - your distribution -can( 'Makefile' ) but when you call the method it dies. That's reasonable if your distribution can actually use a Makefile. When it can't, its just making the job of toolchains harder. At least with an absent Makefile.PL, its possible to look at a distribution and sanely conclude that there's no Makefile.PL. Or do you wish for a protocol for fake Makefile.PL files to communicate to toolchains that Ooops! No, don't use me!? This comes because you want nicer data from CPAN smokers, right? I already get dumb failures where the smoker registered the fact that my modules don't pass tests when the prerequisites aren't installed. Amazing. That's better fixed with a note to the person running the smoker than a fix to the CPAN distribution. Josh
Re: echo die Makefile.PL
On 4/17/07, Eric Wilhelm [EMAIL PROTECTED] wrote: # from Sébastien Aperghis-Tramoni # on Tuesday 17 April 2007 05:12 pm: If you really want Module::Build to be installed, why not simply create a traditional Makefile.PL and add Module::Build as a prerequisite? That way your module can be installed with ExtUtils::MakeMaker yet Module::Build also gets installed for the next time. That assumes the module *can* be built/installed with EU::MM (without extra shenanigans.) If you have a custom build class, this is often not the case. The solution really should not involve 'traditional'. Furthermore, I already do declare a dependency on Module::Build in the META.yml. It seems that this is ignored though. I understand Adam's argument about platform-sensitive dependencies, but shouldn't we have some way to say you absolutely must install the build tool first? Wouldn't a static yaml file be the place for that? I've read that META.yml's declaration is considered to be a suggestion but not the authoritative resource for tools. My guess is that your automated tools really should run (Build|Makefile).PL but fail over to META.yml if that's not possible. Perhaps some machines can't run perl but still want to examine this information. They're second class citizens since they're reading generated data that's potentially out of sync. Too bad, I guess. Since you have no Makefile.PL (this is unusual but sane under limited circumstances), I'd guess I'd just consider any smoker that generates that file for you to be broken. Josh
Re: Locally installed modules
On 4/9/07, Andreas J. Koenig [EMAIL PROTECTED] wrote: On Mon, 9 Apr 2007 15:13:56 -0700, Joshua ben Jore [EMAIL PROTECTED] said: On 4/9/07, Bill Moseley [EMAIL PROTECTED] wrote: I'm currently replicating an automated install of Catalyst against a fresh 5.8.8. Interestingly, I saw one interesting failure. Catalyst has oodles of dependencies. At some point during the installation of a sub-sub-sub-* dependency, CPAN.pm deleted its downloaded copy of something it was still in the process of installing. I guess CPAN's build caching mechanism doesn't take into account things in the cache that it still cares about. Ooops! I could imagine you didn't use CPAN.pm 1.90, right? For sure. I was testing that a fresh 5.8.8 could install it without error. I use this to help suss out undeclared prerequisites. I have the same environment for all of 5.6.2/5.8.1-5.8.8/5.9.5 under non-/threaded, non-/debugging. While I did it manually today, I've got a harness over it all so I can essentially just request that *all* those perls attempt to go install a CPAN module of my choice (or a local distribution) and I get TAP back about which perls passed and failed. # for a single perl and what happened last night cd /opt rm -rf perl-5.8.8/* tar xjf perl-5.8.8.tar.bz2 # a fresh 5.8.8 but with updated CPAN::Config /opt/perl-5.8.8/bin/cpan Catalyst /dev/null Josh
Re: Locally installed modules
On 4/9/07, Bill Moseley [EMAIL PROTECTED] wrote: I'm currently replicating an automated install of Catalyst against a fresh 5.8.8. Interestingly, I saw one interesting failure. Catalyst has oodles of dependencies. At some point during the installation of a sub-sub-sub-* dependency, CPAN.pm deleted its downloaded copy of something it was still in the process of installing. I guess CPAN's build caching mechanism doesn't take into account things in the cache that it still cares about. Ooops! Catalyst installed without any other error. Template ignores my settings when it comes to that /usr/local/tt2 stuff. It doesn't succeed when there's no STDIN to ask me questions about the install. Dang. If you read the Makefile.PL, you'll see that you need to specify some PL args like TT_PREFIX which is *different* than any other option. Oh well. Here's my ultra simple setup. PERL5LIB is set to ~/.perl/lib. Period. I don't do that arch/version stuff because I don't see a reason to. CPAN/Config.pm (or ~/.cpan/CPAN/Myconfig.pm if you prefer) has its makepl_arg fully specified. *every* lib directory is specified and pegged to ~/.perl/lib. I like being nice to myself so all the scripts and binaries are installed to ~/bin. 'makepl_arg' = q[INSTALLARCHLIB=/home/josh/.perl/lib INSTALLBIN=/home/josh/bin INSTALLMAN1DIR=/home/josh/.perl/man/man1 INSTALLMAN3DIR=/home/josh/.perl/man/man3 INSTALLPRIVLIB=/home/josh/.perl/lib INSTALLSCRIPT=/home/josh/bin INSTALLSITEARCH=/home/josh/.perl/lib INSTALLSITEBIN=/home/josh/bin INSTALLSITELIB=/home/josh/.perl/lib INSTALLSITEMAN1DIR=/home/josh/.perl/man/man1 INSTALLSITEMAN3DIR=/home/josh/.perl/man/man3 INSTALLVENDORARCH=/home/josh/.perl/lib INSTALLVENDORBIN=/home/josh/bin INSTALLVENDORLIB=/home/josh/.perl/lib INSTALLVENDORMAN1DIR=/home/josh/.perl/man/man1 INSTALLVENDORMAN3DIR=/home/josh/.perl/man/man3], I could have also set mbuild_arg here too but I chose to use a ~/.modulebuidrc which does the same thing: all the libraries are installed to the same location and all the scripts are installed to my ~/bin. --installpath lib=/home/josh/.perl/lib --installpath arch=/home/josh/.perl/lib --installpath bin=/home/josh/bin --installpath bindoc=/home/josh/.perl/man/man1 --installpath libdoc=/home/josh/.perl/man/man3 It works. It's verbose but it also means I generally don't have to worry about my installs much. Josh
Re: Package::Unicorn
On 2/18/07, Eric Wilhelm [EMAIL PROTECTED] wrote:
You can do parts of this with Best. It doesn't presume to name your
anonymous code so you're stuck with populating %INC but then I thought
the point was that you didn't know what name you were going to use
yet. Once you have a name, you can just substitute that in and Best
will go use that instead of whatever you have inline. It just finds
the first thing that succeeds, really.
use Best [
# 'The::Permanent::Name',
sub { ... },
];
Josh
Delete hate speech module
I'd just read of Time::Cube, a disjointed rant full of hate speech. This is the kind of content that is most deserving of deletion from CPAN. Would the responsible parties please go nuke this, please? Josh
Re: Benefits of Test::Exception
On 12/31/06, Paul LeoNerd Evans [EMAIL PROTECTED] wrote: \ No it doesn't... This is one of those things about perl - code that looks like a function call is never checked to see if the function exists until runtime: #!/usr/bin/perl use warnings; use strict; print Here I have started running now\n; foobarsplot(); ^-- won't complain until runtime. That's what gave me the motivation to write B::LintSubs, by the way: http://search.cpan.org/~pevans/B-LintSubs-0.02/ Isn't that just a copy of a 5.8-ish perl -MO=B,undefined-subs? The 5.9.something+ versions added support for threaded perls and you might want to copy that too. There's also support for arbitrary plugins so you can write your tests code as a plain plugin and it gets called during the single runloop and you don't have to have a runloop in your tests anymore. Josh
Re: Benefits of Test::Exception
On 12/30/06, Paul LeoNerd Evans [EMAIL PROTECTED] wrote:
I recently stumbled upon Test::Exception, and wondered if it might make
my test scripts any better.. So far I'm struggling to see any benefit,
for quite a lot of cost.
Without using this module, my tests look like:
eval { code() };
ok( $@, 'An exception is raised' );
This is a buggy test. $@ can be cleared by destructors firing during
the end of the eval. You must test that the return value of eval is
true to be unambiguously correct.
my $ok = eval { code(); 1 };
my $e = $@;
ok( ! $ok, 'An exception was raised' );
like( $e, qr/.../, 'Exception was ...' );
like( $@, qr/some string match/, 'Exception type' );
You should expect $@ to be cleared during anything that might
conceivably call perl code. This includes using any operators other
than assignment. That includes comparison and boolean tests.
ok( [EMAIL PROTECTED]isa( Thing ), 'Exception type' );
Same problem. Copy $@ away before examining it so it won't be
overwritten under your feet.
Have I missed something here? Does Test::Exception provide me with some
greater functionallity I haven't yet observed? Or should I just not
bother using it?
It provides you better verbs so your tests are talking about a task
and not required to perform the task. Your tests are still buggy
regardless of whether you use Test::Exception.
Josh
Re: Updating a module that's included in another distro
On 11/25/06, Andy Armstrong [EMAIL PROTECTED] wrote: On 25 Nov 2006, at 23:19, Jonathan Rockway wrote: I probably didn't make myself clear; that's exactly what I've done. It seems to be confused because it's saying that Geo::Cache has a higher version - which it doesn't - it's at 0.06. Presumably it's the fact that Geo::Gpx is currently distributed alongside Geo::Cache and I'm trying to set it free on it's own that's causing the problem :) You can't do that. Once a package is versioned, it remains versioned forever. You'll need to have incrementing version numbers for Geo::Gpx or somehow get PAUSE to forget that the previous versioned Geo::Gpx exists. I'm not aware of any way to do that. Josh
Re: Preventing PAUSE from indexing a module?
On 10/10/06, Andreas J. Koenig [EMAIL PROTECTED] wrote: On Mon, 9 Oct 2006 18:27:15 -0500, Joshua ben Jore [EMAIL PROTECTED] said: I assume I am doing something wrong because I'm not hearing from other people complaining that PAUSE is ignoring their no_index settings. It's amazing, the indexer might indeed work correctly:) Yay! Thanks all. Josh
Preventing PAUSE from indexing a module?
The Carp::Clan module uses the DB package temporarily and the PAUSE indexer seems to think I'm attempting to publish something in the DB namespace. I added a no_index entry to my META.yml but that doesn't appear to be stopping PAUSE. Is this documented somewhere? I looked in ExtUtils::MakeMaker for notes on Makefile.PL magic (none) and the CPAN faq (nothing apropos) and got nothing useful. The PAUSE faq mentions no_index but doesn't document how to use it. The META.yml spec doesn't mention no_index. The PAUSE source code appears to be treating no_index like a hash looking things up by package but my setting didn't affect it. Here's my http://search.cpan.org/src/JJORE/Carp-Clan-5.8/META.yml: # http://module-build.sourceforge.net/META-spec.html name: Carp-Clan version: 5.8 license: perl installdirs: site distribution_type: module no_index: DB: 1 I assume I am doing something wrong because I'm not hearing from other people complaining that PAUSE is ignoring their no_index settings. WTH am I doing wrong? Josh
Re: Indexing for AI-Prolog-0.735_01?
On 8/5/06, Ovid [EMAIL PROTECTED] wrote: - Original Message From: Ken Williams [EMAIL PROTECTED] BTW, you might want to add a see also in the AI::Prolog docs for the SWI-Prolog wrapper? Probably a good idea. It's recommended in AI::Prolog::Article, but that's about it. Already applied to my source. Good suggestion. Thanks. Josh
Fwd: Indexing for AI-Prolog-0.735_01?
PAUSE says to send problems to modules@perl.org but perl.org's web page says that's a closed list?! So now this is two problems. One is why PAUSE isn't indexing AI-Prolog and the other is that the published contact doesn't accept messages from authors. Or doesn't *seem* to anyway. Josh -- Forwarded message -- From: Joshua ben Jore [EMAIL PROTECTED] Date: Aug 3, 2006 8:11 PM Subject: Indexing for AI-Prolog-0.735_01? To: modules@perl.org I uploaded AI-Prolog-0.735_01.tar.gz the other day but it hasn't been indexed yet. The following is what PAUSE says about my access to this namespace. I think something when awry because I've since uploaded something else an hour ago that has been indexed. AI::Prolog JJORE co-maint AI::Prolog::BuiltinsJJORE co-maint AI::Prolog::ChoicePoint JJORE co-maint AI::Prolog::Engine JJORE co-maint AI::Prolog::Engine::Primitives JJORE co-maint AI::Prolog::IntroductionJJORE co-maint AI::Prolog::KnowledgeBase JJORE co-maint AI::Prolog::Parser JJORE co-maint AI::Prolog::Parser::PreProcessorJJORE co-maint AI::Prolog::Parser::PreProcessor::Math JJORE co-maint AI::Prolog::TermJJORE co-maint AI::Prolog::Term::Cut JJORE co-maint AI::Prolog::Term::NumberJJORE co-maint AI::Prolog::TermListJJORE co-maint AI::Prolog::TermList::ClauseJJORE co-maint AI::Prolog::TermList::Primitive JJORE co-maint AI::Prolog::TermList::Step JJORE co-maint Josh
Re: Fwd: Indexing for AI-Prolog-0.735_01?
On 8/4/06, Smylers [EMAIL PROTECTED] wrote: Joshua ben Jore writes: PAUSE says to send problems to modules@perl.org but perl.org's web page says that's a closed list?! Closed in the sense that only the Pause admin folk can subscribe to it. Think of it as less of a mailing list and more of a generic address for contacting all the Pause volunteers at once. The message I'd sent to modules@perl.org didn't appear in the archive and the text on http://lists.cpan.org/ seemed to mean that it wouldn't ever appear. It'd be nice to update http://lists.cpan.org/ to indicate that it's moderated so the next person doesn't just assume their message has been routed to /dev/null. Josh
