Re: [Dbix-class] ★ VOTE NOW: DBIC Governance and Namespace Control ★

[Sam Kington]

> List members should reply to this email with an email body indicating clearly 
> "Proposal A" or "Proposal B".  Other responses, such as "+1" or "me, too" 
> replies to others' votes will be disregarded.

Proposal A.

Sam
-- 
Website: http://www.illuminated.co.uk/


___
List: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/dbix-class
IRC: irc.perl.org#dbix-class
SVN: http://dev.catalyst.perl.org/repos/bast/DBIx-Class/
Searchable Archive: http://www.grokbase.com/group/dbix-class@lists.scsys.co.uk

Re: [Dbix-class] Decision time: which fork inherits the existing DBIx::Class namespace

[Sam Kington]

Hi,

Delurking; I’ve been following the discussion but haven’t said anything 
previously because I haven’t historically been part of the DBIx::Class 
community, nor have I been using the distribution for that long.

Also, I’ve struggled to understand what Peter Rabbitson’s problem with Perl, 
CPAN authors and the likes has been, despite reading a number of his blog and 
mailing list posts. This first bit is another example of such:

On 3 Dec 2016, at 05:50, Peter Rabbitson  wrote:
> I do, however, firmly believe that Matt's perspective[1] of:
> 
>> I don't really think open source is *about* the people developing as
>> such. What I think is that open source is *made of* the people
>> developing it
> 
> is more or less an embodiment of what is wrong with the Open Source
> industry today.

…and then he just leaves it at that.

Well, at some points you just have to say “it’s not me, it’s you”. I have no 
idea what that means, but Peter’s clearly not interested in explaining. No 
doubt this all makes sense in his head, and to the small handful of people who 
have been arguing with Peter for years on this subject, but it leaves me none 
the wiser.

But OK, not everyone is a trained writer, and by all accounts Peter’s technical 
skills are superb. So I was intrigued to hear this next bit:

> The name of the fork is not interesting - the important thing is that it
> will continue providing the DBIx::Class namespace ( via a novel method,
> with safe-by-default conflict-resolution at install time, very much
> *un*like the Alt convention ). This means that a user willing to
> "switch" will have to adjust nothing more than their list of dependencies.

…and again nothing. How will this cunning plan work? What does “safe-by-default 
conflict-resolution at install time” even mean? Will it work for people 
installing DBIx::Class or support modules via Linux distributions’ package 
managers, not just via CPAN.pm, cpanm or similar? Given that Peter also wants 
the PAUSE rights to the DBIx::Class namespace (which makes me wonder why he’s 
even talking about the name of the fork if he wants to still call it 
DBIx::Class), is there any reason why Peter’s DBIx::Class would even have to 
care about this stuff at all?

The time for something like this was months ago when the split was first 
mooted. Now it feels, wilfully or not, like vapourware.

Sam
-- 
Website: http://www.illuminated.co.uk/


___
List: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/dbix-class
IRC: irc.perl.org#dbix-class
SVN: http://dev.catalyst.perl.org/repos/bast/DBIx-Class/
Searchable Archive: http://www.grokbase.com/group/dbix-class@lists.scsys.co.uk

Re: [Dbix-class] Inserting binary data without corrupting

[Sam Kington]

On 27 Feb 2015, at 07:54, Peter Rabbitson rabbit+d...@rabbit.us wrote:
 On 02/26/2015 11:38 PM, Sam Kington wrote:
 Is this the best way of handling this, or is there a simpler or purer way 
 I've overlooked?
 
 Interesting. This is one way of handling it, yes (btw `unpack 'H*', $binary 
 data` would be much faster). If this definitely fixes things on a combination 
 of a DBD and RDBMS version (you never said what you are using) - I can look 
 into making this implicitly handled by the driver itself.

MySQL 5.1.73-1, DBIx::Class 0.082810, DBI 1.622, DBD::mysql 4.022.

Sam
-- 
Website: http://www.illuminated.co.uk/


___
List: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/dbix-class
IRC: irc.perl.org#dbix-class
SVN: http://dev.catalyst.perl.org/repos/bast/DBIx-Class/
Searchable Archive: http://www.grokbase.com/group/dbix-class@lists.scsys.co.uk

[Dbix-class] Inserting binary data without corrupting (was: Re: Monkey-patching around a MySQL/UTF8 bug)

[Sam Kington]

On 26 Feb 2015, at 18:14, David Cantrell da...@cantrell.org.uk wrote:
 On Thu, Feb 26, 2015 at 04:55:10PM +0100, Peter Rabbitson wrote:
[...]
 Also (as noted in that RT) - this will badly 
 break BLOB operations.
 
 Good point. That prompted me to look over our code. We had two BLOB
 fields, one of which can be replaced with a TEXT. The other really does
 contain binary data, but it seems that it's getting corrupted anyway
 even without my monkey-patch. Yuck.

The problem appeared to be encoding issues between DBI and mysql. Delving into 
DBIx::Class::Storage::DBI:_dbh_execute I found that we were correctly passing a 
byte-encoded string from DBIx::Class, but it ended up corrupted in the database:

   DB11 x $sql   
   
 0  'INSERT INTO customer_static_file ( bytes, customer_id, date_created, 
 date_updated, filename, user_id) VALUES ( ?, ?, ?, ?, ?, ? )'
   DB12 x $bind  
   
 0  ARRAY(0x122bf2b8)
0  ARRAY(0x122bef70)
   0  HASH(0x122780a8)
  'dbic_colname' = 'bytes'
  'sqlt_datatype' = 'LONGBLOB'
   1  ?PNG\cM\cJ\cZ\cJ??Stuff

DBI appears to know that this is a binary column, but that's not helping.

The solution I arrived at after some googling was as follows:

 __PACKAGE__-load_components('FilterColumn');
 __PACKAGE__-filter_column(
 'bytes',
 {
 filter_from_storage = sub { pop },
 filter_to_storage = sub {
 my ($resultset, $binary_data) = @_;
 my $encoded_string = sprintf(
 x'%s',
 join('',
 map { sprintf('%02x', ord($_)) } split('', $binary_data))
 );
 return \$encoded_string;
 },
 }
 );

which uses MySQL's preferred way of encoding binary data on the command line.

Is this the best way of handling this, or is there a simpler or purer way I've 
overlooked?

Sam
-- 
Website: http://www.illuminated.co.uk/


___
List: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/dbix-class
IRC: irc.perl.org#dbix-class
SVN: http://dev.catalyst.perl.org/repos/bast/DBIx-Class/
Searchable Archive: http://www.grokbase.com/group/dbix-class@lists.scsys.co.uk

[Dbix-class] Monitoring how many rows are returned per query

[Sam Kington]

Hi,

I'm new to the mailing list, and only quickly skimmed the archives, so 
apologies if this question has been asked before.

At $WORK, for our Chimera project, we bought fully into the way of prefetching, 
and that worked fine with our tests that were working on limited data sets. But 
now we've found that those ever-so-clever queries that prefetched multiple 
independent tables at a time are now generating hundreds of megabytes of data 
and thousands of lines of SQL which we don't actually want or need. So we'd 
like to put a stop to that. And one of the ways is monitoring the number of 
rows that queries produce in tests, staging or, failing that, production.

Unfortunately, there doesn't appear to be a clean way of doing this. So far my 
best attempt is a rather nasty monkeypatch:

my $sql_row_count;
no warnings 'redefine';
$old_dbh_execute = \DBIx::Class::Storage::DBI::_dbh_execute;
*DBIx::Class::Storage::DBI::_dbh_execute = sub {
my $self = shift;
my ($rv, $sth, @bind) = $old_dbh_execute-($self, @_);

Test::Chimera-inspect_statement_handle($sth);

return (wantarray ? ($rv, $sth, @bind) : $rv);
};

package Test::Chimera;

my $sql_row_count;
sub inspect_statement_handle {
my ($sth) = @_;

my $sql_statement = $sth-{Statement};
if (Database::Chimera-schema-storage-debug
 $sql_statement !~ /^ (INSERT \s INTO | UPDATE ) \s log \s /x)
{
$sql_row_count += $sth-rows;
if ($ENV{VERBOSE_SQL_ROW_COUNT}) {
printf STDERR
# %s\nwith bind [%s]\n,
$sql_statement, join('|', map { $_-[1] } @bind);
print STDERR # Got back  . $sth-rows .  rows\n;
}
}
}

The problem is that, at least in the version current on the CPAN, _dbh_execute 
does the following:

 $self-_query_start($sql, $bind);
my $sth = $self-_bind_sth_param(...);
my  $rv = $sth-execute;
$self-throw_exception(...) if !$rv;
$self-_query_end($sql, $bind);
return (wantarray ? ($rv, $sth, @$bind) : $rv);

Any debugging objects get the query and bind arguments - the what of the query 
- but not the how - how many rows were returned, for instance, or anything else 
you can get from the DBI statement handle.

Maybe it could be worth adding e.g. $self-_query_details($sth) which would 
call the appropriate debugger object?

Or is there a better way of capturing this information for debugging? I chose 
_dbh_execute as the method to monkey-patch as it seemed the most obvious, but I 
don't know the guts of DBIx::Class at all well so I could certainly have missed 
something.

Thanks in advance.

Sam
-- 
Website: http://www.illuminated.co.uk/


___
List: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/dbix-class
IRC: irc.perl.org#dbix-class
SVN: http://dev.catalyst.perl.org/repos/bast/DBIx-Class/
Searchable Archive: http://www.grokbase.com/group/dbix-class@lists.scsys.co.uk