Hi Darin -
Please pull from my adapter_extensions branch
(http://github.com/mainej/adapter_extensions/tree) or wait until Anthony pulls
my changes. Unfortunately, I'm not sure that will fix
test_bulk_import_with_empties. I can't be sure because I don't have MySQL
installed, and don't have time to do that right now.
The change I made to adapter_extensions allows Postgres to import empty strings
as nulls. That's what the test is about. I put the test in for both adapters,
hoping that MySQL would do the right thing out of the box. If it's not
working, I suggest either making your proposed change, or looking at
adapter_extensions to see if you can get MySQL to handle empty strings in a
more useful way. I'll look at this stuff more this weekend to make sure my
system is up-to-date and tests are passing for Postgres.
Hopefully with two groups developing for Postgres and MySQL, we'll get AW & AW
ETL back into a stable state.
Best,
Jacob
----- Darin Wilson <[EMAIL PROTECTED]> wrote:
> I just pulled the latest from your github repo and ran the ETL test suite
> locally against MySQL. I got 10 errors that were about SCDs, so I'm assuming
> those were the same ones you mentioned in your last message(?).
>
> There was one other error:
>
> 1) Failure:
> test_bulk_import_with_empties(ProcessorTest) [./test/processor_test.rb:19]:
> <ActiveRecord::StatementInvalid> exception expected but none was thrown.
>
> It turns out that MySQL is much less strict about what goes into the bulk
> loader - if you try to load a row with an empty value into a column that
> disallows null, MySQL just skips it without an error. Nice! :)
>
> I'm not sure how you'd like to handle that. I tried poking around a bit to
> see if there's a way to force MySQL into being more strict, but I couldn't
> get anything to work reliably. The simplest fix would be to do something
> like this in the test:
>
> unless ETL::Engine.connection(:data_warehouse).class ==
> ActiveRecord::ConnectionAdapters::MysqlAdapter
> assert_raise(ActiveRecord::StatementInvalid) {
> do_bulk_import('bulk_import_with_empties.txt') }
> end
>
> which sort of invalidates the test, but it doesn't seem like there's a way
> to make this test actually work with MySQL. Your call. ;)
>
> Darin
>
>
> > From: Anthony Eden <[EMAIL PROTECTED]>
> > Date: Thu, 16 Oct 2008 15:23:04 -0400
> > To: Darin Wilson <[EMAIL PROTECTED]>
> > Cc: <activewarehouse-discuss@rubyforge.org>
> > Subject: Re: [Activewarehouse-discuss] would the real codebase please stand
> > up? :)
> >
> > I think I have most of the fixes to ETL in for the 2.1 changes. I've
> > started changing over some of the tests to use shoulda 1.x, so if you
> > want to run the built in tests then you'll need the shoulda gem. There
> > are currently two failing tests related to the SCD implementation.
> > Jacob, was that an area you worked on? If so could you take a look at
> > the failing tests? Also, I currently have tested with Postgres locally
> > but not MySQL, so if someone wants to fight through the test suite on
> > MySQL, be my guest. :-)
> >
> > Thanks.
> >
> > -Anthony
> >
> > On Thu, Oct 16, 2008 at 12:11 PM, Darin Wilson
> > <[EMAIL PROTECTED]> wrote:
> >> Cool - that is what I was going to start working on. :)
> >>
> >> Please let us know when your 2.1 fixes to ETL are checked in - I'd like to
> >> try it out with our ETL (and I have a bunch of test cases for it).
> >>
> >> Thanks!
> >> Darin
> >>
> >>> From: Anthony Eden <[EMAIL PROTECTED]>
> >>> Date: Thu, 16 Oct 2008 06:59:41 -0400
> >>> To: Marty Haught <[EMAIL PROTECTED]>
> >>> Cc: <activewarehouse-discuss@rubyforge.org>
> >>> Subject: Re: [Activewarehouse-discuss] would the real codebase please
> >>> stand
> >>> up? :)
> >>>
> >>> I've started the upgrade to 2.1. I've already modified ActiveWarehouse
> >>> ETL to handle the new migration process and I have it running right
> >>> now under 2.1. Once I get past my first ETL load on the project I'm
> >>> working on I'll move onto the AW plugin.
> >>>
> >>> -Anthony
> >>> _______________________________________________
> >>> Activewarehouse-discuss mailing list
> >>> Activewarehouse-discuss@rubyforge.org
> >>> http://rubyforge.org/mailman/listinfo/activewarehouse-discuss
> >>
> >>
> >> _______________________________________________
> >> Activewarehouse-discuss mailing list
> >> Activewarehouse-discuss@rubyforge.org
> >> http://rubyforge.org/mailman/listinfo/activewarehouse-discuss
> >>
>
>
> _______________________________________________
> Activewarehouse-discuss mailing list
> Activewarehouse-discuss@rubyforge.org
> http://rubyforge.org/mailman/listinfo/activewarehouse-discuss
_______________________________________________
Activewarehouse-discuss mailing list
Activewarehouse-discuss@rubyforge.org
http://rubyforge.org/mailman/listinfo/activewarehouse-discuss
