I was recently able to write a script to take a mysql dumpfile and
load it into an h2 in-memory database. I have found h2 to be
considerably faster than hsql in the liquibase unit tests.
We are building up a "test" dataset in mysql with a good variety of
normal and edge case data that we can usefor our development servers.
We have our dao tests running against this data, but I was wanting it
to run much faster.
We would do the creation and population of the h2 database once at
test startup and have each dao test run in a transaction so they are
independent.
I was surprised to find, however, that the h2 tests too basically the
same amount of time to run as the mysql tests. Apparently the time
creating and loading h2 balanced out any query speed improvements.
Your milage will vary depending on size of data and database and
usage, but you shouldn't nessisarily assume that an in-memory database
will make your tests run any faster.
Nathan
On Mar 12, 2009, at 7:56 AM, "Paul Keeble" <cs...@yahoo.co.uk> wrote:
>
> In the past for database testing I've used DBUnit to clear out the
> database data and only insert what you wanted, effectively
> generating a database fixture. Its got little to do with Liquibase
> but its a nice neat way to deal with the data that functional tests
> need - meaning you only need to build the database once rather than
> recreating over and over a clean one.
>
> Paul
>
>
> ----- Original Message ----
> From: David C. Hicks <dhi...@i-hicks.org>
> To: liquibase-user@lists.sourceforge.net
> Sent: Thursday, 12 March, 2009 2:41:19
> Subject: Re: [Liquibase-user] How to debug upgrade called from API?
>
> That's not a bad idea, Paul. Thanks for the tip.
>
> My solution ended up being somewhat hacked, but it works. In addition
> to our usual "development" databases, we will add a "test" database
> that
> can be trashed by these functional unit tests. So, from our
> functional
> test base class we drop and recreate the database then run the
> Liquibase
> update on it. Each "TestCase" now gets a clean database to play with.
>
> I would have preferred to have an in-memory DB for speed, but since
> speed isn't a possibility, I'm working on segregating the execution of
> these functional unit tests so that they don't have to run every
> time a
> developer does a build.
>
> All's well that ends well, I suppose. I agree that it's shameful the
> DBMS vendors don't feel any need to conform to accepted standards, but
> that's the world we live and develop in.
>
> Dave
>
> Paul Keeble wrote:
>> I would love to say one day the portability issues between
>> databases would be fixed, but well I only see yet more divergence
>> in the future. None of the database designers care much about the
>> SQL standards, which means that without DBMS specific setups only
>> the simplest of databases can work without any targeting.
>>
>> One thing you might want to try is using the dbms/context
>> properties on ChangeSet to make specific versions for testing (but
>> they won't get applied in production, acting as replacements for
>> the non supported things in HSQL). Although its not perfect because
>> the databases will be different, if you can find a way to represent
>> your types/indexes in HSQL then these settings will keep the
>> specific changes away from production but still get the same
>> changelog used for both, with the known differences explicitly shown.
>>
>> I know its not perfect because now your test database differs to
>> the real one and that could cause its own problems but its better
>> than not having a test database setup automatically at all.
>>
>> Paul
>>
>>
>> ----- Original Message ----
>> From: David C. Hicks <dhi...@i-hicks.org>
>> To: liquibase-user@lists.sourceforge.net
>> Sent: Wednesday, 11 March, 2009 19:53:23
>> Subject: Re: [Liquibase-user] How to debug upgrade called from API?
>>
>> I figured out the problem.
>>
>> HypersonicSQL does not keep indexes as subordinates to their
>> corresponding tables. In our changelog.xml, we have indexes
>> associated
>> with multiple tables for which the index has the same name. (ie.
>> K_code
>> is an index name on tables A, B, and C). This works fine in most
>> databases, because the indexes are subordinate to their tables. The
>> index names, in our case, were generated by the Liquibase schema
>> generator, based on an existing MySQL database. I can't even think
>> of a
>> way to "fix" this without altering the existing changesets, which
>> would
>> break in our existing production database. So, I'll have to go a
>> different route for our testing.
>>
>> I discovered after changing some of the index names (just for kicks),
>> that we have data types that Hypersonic doesn't get along with, as
>> well
>> - like LONGTEXT. So, that, too, made this whole thing a bit of a
>> moot
>> point.
>>
>> Dave
>>
>> Peter Rebholz wrote:
>>
>>> I'm not very familiar with HSQL or MySQL but is it possible that
>>> it's
>>> complaining because the primary key (product_type) is indexed by
>>> default? Oracle behaves this way.
>>>
>>>
>>> On Mar 11, 2009, at 12:43 PM, David C. Hicks wrote:
>>>
>>>
>>>
>>>> A little more info/revision:
>>>>
>>>> I see now that the entire changelog.xml is *not* being
>>>> performed. It
>>>> does, indeed, bail out on the first failure. I just don't
>>>> understand
>>>> why, I guess. Here is the specific change set that is failing:
>>>>
>>>> <changeSet author="dhicks (generated)" id="1224863298725-24">
>>>> <createIndex indexName="K_code" tableName="product_type"
>>>> unique="false">
>>>> <column name="code" />
>>>> </createIndex>
>>>> </changeSet>
>>>>
>>>> It complains that this index already exists, but I don't see
>>>> where it
>>>> would have been created previously. In fact, outside of some
>>>> "insert"
>>>> statements to create data, here is the only other mention of that
>>>> table:
>>>>
>>>> <changeSet author="dhicks (generated)" id="1224863298725-11">
>>>> <createTable tableName="product_type">
>>>> <column autoIncrement="true" name="id" type="BIGINT">
>>>> <constraints nullable="false"
>>>> primaryKey="true" />
>>>> </column>
>>>> <column name="version" type="BIGINT">
>>>> <constraints nullable="false" />
>>>> </column>
>>>> <column defaultValue="" name="code"
>>>> type="varchar(255)">
>>>> <constraints nullable="false" />
>>>> </column>
>>>> <column defaultValue="" name="name"
>>>> type="varchar(255)">
>>>> <constraints nullable="false" />
>>>> </column>
>>>> </createTable>
>>>> </changeSet>
>>>>
>>>> You can see, by the ID, that this changeset would come before the
>>>> previous one.
>>>>
>>>>
>>>> David C. Hicks wrote:
>>>>
>>>>
>>>>> Hi guys,
>>>>>
>>>>> I've been trying to get Liquibase to run our changelog.xml using
>>>>> the
>>>>> API. I found a piece of code that Nathan had made available to
>>>>> someone at a previous time. It seems to work, mostly, but I get a
>>>>> SQLException at the end of it all. Here's a code excerpt:
>>>>>
>>>>> private void doLiquibaseUpdate() throws Exception {
>>>>> FileOpener fileOpener = new FileSystemFileOpener();
>>>>> DatabaseFactory databaseFactory =
>>>>> DatabaseFactory.getInstance();
>>>>> Database database = null;
>>>>> database =
>>>>>
>>>>> databaseFactory.
>>>>> findCorrectDatabaseImplementation(getConnection());
>>>>> Liquibase liquibase = new
>>>>> Liquibase(databaseChangeLogFile,
>>>>> fileOpener, database);
>>>>> liquibase.update("");
>>>>> }
>>>>>
>>>>> As I mentioned, everything seems to work correctly, up to the
>>>>> end. I
>>>>> get log messages indicating that each change set gets applied. At
>>>>> the
>>>>> end of it all, though, a SQLException is thrown from
>>>>> liquibase.update("") indicating that an attempt to create an index
>>>>> that already exists has occurred. I've searched through our
>>>>> changeLog.xml and I don't see any duplication of that particular
>>>>> index.
>>>>>
>>>>> My goal is to use this in a unit test base class to set up an
>>>>> in-memory HSQLDB database during the onSetUp() phase of an
>>>>> AbstractTransactionalDataSourceSpringContextTests test case. The
>>>>> idea
>>>>> being that it's a nice way to set up a database for testing DAO
>>>>> methods. [I'm not interested in a religious war about how to test
>>>>> DAO
>>>>> methods.]
>>>>>
>>>>> Normally, I apply our changeLog.xml using either
>>>>> maven-liquibase-plugin, or the command line tools. So, the
>>>>> differences are: a) using the API, b) using HSQLDB instead of
>>>>> MySQL.
>>>>> I just don't know of any good way to "debug" what's happening on
>>>>> the
>>>>> inside of that call to try to track this down.
>>>>>
>>>>> Any thoughts?
>>>>> Thanks,
>>>>> Dave
>>>>>
>>>>> ---
>>>>> ---
>>>>> ------------------------------------------------------------------
>>>>>
>>>>> ---
>>>>> ---
>>>>> ---
>>>>> ---
>>>>> ------------------------------------------------------------------
>>>>> Apps built with the Adobe(R) Flex(R) framework and Flex
>>>>> Builder(TM)
>>>>> are
>>>>> powering Web 2.0 with engaging, cross-platform capabilities.
>>>>> Quickly and
>>>>> easily build your RIAs with Flex Builder, the Eclipse(TM)based
>>>>> development
>>>>> software that enables intelligent coding and step-through
>>>>> debugging.
>>>>> Download the free 60 day trial. http://p.sf.net/sfu/www-adobe-com
>>>>> ---
>>>>> ---
>>>>> ------------------------------------------------------------------
>>>>>
>>>>> _______________________________________________
>>>>> Liquibase-user mailing list
>>>>> Liquibase-user@lists.sourceforge.net
>>>>> https://lists.sourceforge.net/lists/listinfo/liquibase-user
>>>>>
>>>>>
>>>>>
>>>> ---
>>>> ---
>>>> ---
>>>> ---
>>>> ------------------------------------------------------------------
>>>> Apps built with the Adobe(R) Flex(R) framework and Flex Builder(TM)
>>>> are
>>>> powering Web 2.0 with engaging, cross-platform capabilities.
>>>> Quickly
>>>> and
>>>> easily build your RIAs with Flex Builder, the Eclipse(TM)based
>>>> development
>>>> software that enables intelligent coding and step-through
>>>> debugging.
>>>> Download the free 60 day trial. http://p.sf.net/sfu/www-adobe-com
>>>> _______________________________________________
>>>> Liquibase-user mailing list
>>>> Liquibase-user@lists.sourceforge.net
>>>> https://lists.sourceforge.net/lists/listinfo/liquibase-user
>>>>
>>>>
>>> ---
>>> ---
>>> ---
>>> ---
>>> ------------------------------------------------------------------
>>> Apps built with the Adobe(R) Flex(R) framework and Flex
>>> Builder(TM) are
>>> powering Web 2.0 with engaging, cross-platform capabilities.
>>> Quickly and
>>> easily build your RIAs with Flex Builder, the Eclipse(TM)based
>>> development
>>> software that enables intelligent coding and step-through debugging.
>>> Download the free 60 day trial. http://p.sf.net/sfu/www-adobe-com
>>> _______________________________________________
>>> Liquibase-user mailing list
>>> Liquibase-user@lists.sourceforge.net
>>> https://lists.sourceforge.net/lists/listinfo/liquibase-user
>>>
>>>
>>
>> ---
>> ---
>> ---
>> ---------------------------------------------------------------------
>> Apps built with the Adobe(R) Flex(R) framework and Flex Builder(TM)
>> are
>> powering Web 2.0 with engaging, cross-platform capabilities.
>> Quickly and
>> easily build your RIAs with Flex Builder, the Eclipse(TM)based
>> development
>> software that enables intelligent coding and step-through debugging.
>> Download the free 60 day trial. http://p.sf.net/sfu/www-adobe-com
>> _______________________________________________
>> Liquibase-user mailing list
>> Liquibase-user@lists.sourceforge.net
>> https://lists.sourceforge.net/lists/listinfo/liquibase-user
>>
>>
>>
>>
>>
>> ---
>> ---
>> ---
>> ---------------------------------------------------------------------
>> Apps built with the Adobe(R) Flex(R) framework and Flex Builder(TM)
>> are
>> powering Web 2.0 with engaging, cross-platform capabilities.
>> Quickly and
>> easily build your RIAs with Flex Builder, the Eclipse(TM)based
>> development
>> software that enables intelligent coding and step-through debugging.
>> Download the free 60 day trial. http://p.sf.net/sfu/www-adobe-com
>> _______________________________________________
>> Liquibase-user mailing list
>> Liquibase-user@lists.sourceforge.net
>> https://lists.sourceforge.net/lists/listinfo/liquibase-user
>>
>
> ---
> ---
> ---
> ---------------------------------------------------------------------
> Apps built with the Adobe(R) Flex(R) framework and Flex Builder(TM)
> are
> powering Web 2.0 with engaging, cross-platform capabilities. Quickly
> and
> easily build your RIAs with Flex Builder, the Eclipse(TM)based
> development
> software that enables intelligent coding and step-through debugging.
> Download the free 60 day trial. http://p.sf.net/sfu/www-adobe-com
> _______________________________________________
> Liquibase-user mailing list
> Liquibase-user@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/liquibase-user
>
>
>
>
>
> ---
> ---
> ---
> ---------------------------------------------------------------------
> Apps built with the Adobe(R) Flex(R) framework and Flex Builder(TM)
> are
> powering Web 2.0 with engaging, cross-platform capabilities. Quickly
> and
> easily build your RIAs with Flex Builder, the Eclipse(TM)based
> development
> software that enables intelligent coding and step-through debugging.
> Download the free 60 day trial. http://p.sf.net/sfu/www-adobe-com
> _______________________________________________
> Liquibase-user mailing list
> Liquibase-user@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/liquibase-user
------------------------------------------------------------------------------
Apps built with the Adobe(R) Flex(R) framework and Flex Builder(TM) are
powering Web 2.0 with engaging, cross-platform capabilities. Quickly and
easily build your RIAs with Flex Builder, the Eclipse(TM)based development
software that enables intelligent coding and step-through debugging.
Download the free 60 day trial. http://p.sf.net/sfu/www-adobe-com
_______________________________________________
Liquibase-user mailing list
Liquibase-user@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/liquibase-user
