Where's the best place to report bugs?
When using the database_destination and truncate == true, only the last
32 records are actually manifested.
From database_destination.rb, it looks like the table is repeated
truncated during processing (every time the buffer is flushed)...
# Flush the currently buffered data
def flush
conn = ETL::Engine.connection(target)
conn.transaction do
conn.truncate(table_name) if truncate
Setting truncate to false and emptying table manually allowed me to
fully load the data.
One other thing: I, too am experiencing issues with the bulk load
approach. The odd thing is that it works sometimes. For example,
worked fine for four or five smaller dimension tables, but will not work
for larger dimension tables. When it does fail, I see the LOAD DATA sql
in the etl.log file and pasting this SQL into the MySQL console is
failing, as well, so not likely Active Record issue.
Regards,
Michael
Jesper Rønn-Jensen wrote:
> Hi Paul
> Thanks for putting effort into this strange weird behaviour.
>
> The change you suggest, however, makes no difference for me. I still
> get the same error. I'm including stack trace below signature.
>
> /Jesper
>
>
> On Feb 20, 2008 8:51 AM, Paul Cortens wrote:
>
> I think I have made some progress. I added one line to
> mysql_adapter.rb in the adapter extensions plugin:
> + @connection = Mysql.new([EMAIL PROTECTED])
>
>
>
>
> --
> ___________________________________________________________
>
> Jesper Rønn-Jensen / Capgemini Denmark
> Tel. +45 3977 8220 / Mob. +45 2373 6220 / Fax +45 7011 2201
> www.dk.capgemini.com <http://www.dk.capgemini.com> / Blog
> http://justaddwater.dk/
> ___________________________________________________________
>
> Executing screens
> Screens passed
>
> Executing post processes
> c:/ruby/lib/ruby/gems/1.8/gems/activerecord-2.0.2/lib/active_record/connection_adapters/abstract_adapter.rb:150:in
>
> `log': Mysql::Error: #42000The used command is not allowed with this
> MySQL version: LOAD DATA LOCAL INFILE 'C:/Documents and
> Settings/Administrator/My
> Documents/rails/ourpeople/import/output/employees.out.txt' INTO TABLE
> employees FIELDS TERMINATED BY ',' ENCLOSED BY '"' IGNORE 1 LINES
> (id,cgid,longid,firstname,lastname,displayname,middleinitial,initials,phone,mobile,fax,subcellphone,email,englishtitle,localtitle,secretary_id,mentor_id,substitute_id,manager_id,comment,licenseplate,employment_cgemployee,employment_cgexternal,employment_cgtemporary,employeework_employeeid,employeework_workpercentage,employeework_actualworkpercentage,company_id,grade_id,location_id,organisation_id,profession_id,resourcetype_id,skillsunit_id,staffingunit_id)
>
> (ActiveRecord::StatementInvalid)
> from
> c:/ruby/lib/ruby/gems/1.8/gems/activerecord-2.0.2/lib/active_record/connection_adapters/mysql_adapter.rb:281:in
>
> `execute'
> from
> c:/ruby/lib/ruby/gems/1.8/gems/adapter_extensions-0.4.0/lib/adapter_extensions/connection_adapters/mysql_adapter.rb:49:in
>
> `do_bulk_load'
> from
> c:/ruby/lib/ruby/gems/1.8/gems/adapter_extensions-0.4.0/lib/adapter_extensions/connection_adapters/abstract_adapter.rb:18:in
>
> `bulk_load'
> from
> c:/ruby/lib/ruby/gems/1.8/gems/activewarehouse-etl-0.9.0/bin/../lib/etl/processor/bulk_import_processor.rb:73:in
>
> `process'
> from
> c:/ruby/lib/ruby/gems/1.8/gems/activerecord-2.0.2/lib/active_record/connection_adapters/abstract/database_statements.rb:66:in
>
> `transaction'
> from
> c:/ruby/lib/ruby/gems/1.8/gems/activewarehouse-etl-0.9.0/bin/../lib/etl/processor/bulk_import_processor.rb:62:in
>
> `process'
> from
> c:/ruby/lib/ruby/gems/1.8/gems/activewarehouse-etl-0.9.0/bin/../lib/etl/engine.rb:485:in
>
> `post_process'
> from
> c:/ruby/lib/ruby/gems/1.8/gems/activewarehouse-etl-0.9.0/bin/../lib/etl/engine.rb:484:in
>
> `each'
> ... 13 levels...
> from
> c:/ruby/lib/ruby/gems/1.8/gems/activesupport-2.0.2/lib/active_support/dependencies.rb:496:in
>
> `require'
> from
> c:/ruby/lib/ruby/gems/1.8/gems/activewarehouse-etl-0.9.0/bin/etl:28
> from c:/ruby/bin/etl:19:in `load'
> from c:/ruby/bin/etl:19
>
> ------------------------------------------------------------------------
>
> _______________________________________________
> Activewarehouse-discuss mailing list
> Activewarehouse-discuss@rubyforge.org
> http://rubyforge.org/mailman/listinfo/activewarehouse-discuss
>
_______________________________________________
Activewarehouse-discuss mailing list
Activewarehouse-discuss@rubyforge.org
http://rubyforge.org/mailman/listinfo/activewarehouse-discuss
