In the past the APIM product has relied on an external component such as a
migration client for upgrading from a given product version to a higher
version.The end user was required to configure the latest product that they
are upgrading to against their current data(databases, synapse files,
registry) and run the migration client manually to upgrade the product.
This can be a cumbersome and error prone process to accomplish for end
users, making product version upgrades time consuming.
To overcome the above problem on the fly upgrades were proposed where the
product code detects if relevant data being accessed needs to be migrated
to the latest version and migrates the data when the code is executed when
the respective feature is used. Upgrading product data is much easier from
3.0.0 onwards because all data related to the product is stored in a
central database.This means that the end user does not need to get involved
in upgrading, it happens without them even being aware as they use the
latest version of the product by pointing it against their current
database.This makes for a more pleasant user experience when upgrading,
putting the burden of the upgrade to be added by the developer into the
functional code itself.
The following outlines a design that can be supported from 3.0.0 onwards to
outline a uniform way of handling product upgrades. This is inspired by the
methodology used by FlywayDB to enable DB migrations[1] but also taking
into account the requirement of being able to run on the fly at
runtime(Note: DB schema changes between releases will need to be handled
via DB vendor specific scripts prepared by the team to be run by the
customer against their DB).
*1.* A new table will be added to the schema called PRODUCT_VERSION_AUDIT
to track the product version upgrades that take place on a given dataset
PRODUCT_VERSION_AUDIT
VERSION VARCHAR(5)
CREATED_TIME TIMESTAMP(6)
If a user begins using APIM version 3.0.0 and then upgrades to version
3.1.0 the table will contain the following values,
VERSION CREATED_TIME
3.0.0 2018-11-11 13:23:44
3.1.0 2019-10-14 9:26:22
This gives a historical view of the product versions a customer has been
using. A new row will be inserted into the table when a given product
version is started for the first time.
*2*. Each table in the database will have a new column called
PRODUCT_VERSION(VARCHAR(5)) added. When a row is inserted for the first
time it will populate this column with the current product version being
used.
For example the AM_API table could have the following entries for a
customer using APIM 3.0.0,
UUID PROVIDER NAME VERSION CONTEXT PRODUCT_VERSION
123e4567-e89b-12d3-a456-426655440000 admin abc 1.0.0 /abc 3.0.0
00112233-4455-6677-8899-aabbccddeeff admin xyz 1.0.0 /xyz 3.0.0
Lets assume when upgrading to 3.1.0 the leading '/' character in the
context needs to be removed. On the fly migration code will run when a
given row is accessed by the DAO layer to remove the '/'. Once the
migration of the row is completed the PRODUCT_VERSION column will be
updated with the value 3.1.0 to signify that the migration for this row has
been completed. The PRODUCT_VERSION column can be validated to check if
migration code needs to be executed. So assuming the API abc is accessed
first the table will look as follows after migration,
UUID PROVIDER NAME VERSION CONTEXT PRODUCT_VERSION
123e4567-e89b-12d3-a456-426655440000 admin abc 1.0.0 abc 3.1.0
00112233-4455-6677-8899-aabbccddeeff admin xyz 1.0.0 /xyz 3.0.0
As a pre-requisite the product team will need to create respective DB
scripts for the schema changes that will take place with a given release.
This will only include schema modifications. Customer will need to run
these manually against their DB but actual data migration will take place
automatically under the hood.
*3*. New Java interfaces will be added for each DB entity that will
responsible for migrating the respective entity. For example for APIs we
can have,
public interface APIMigrator {
API migrate(PreparedStatement statement) throws MigrationException;}
This will accept the PreparedStatement created for data retrieval and
returns the migrated API object. The implemenataion of the above could look
as follows,
public class APIMigratorImpl implements APIMigrator {
public API migrate(PreparedStatement statement) throws MigrationException {
API api = null;
try (ResultSet rs = statement.executeQuery()) {
while (rs.next()) {
String dataProductVersion = rs.getString("PRODUCT_VERSION");
// Assume that currentProductVersion == "3.2.0"
while (!currentProductVersion.equals(dataProductVersion)) }
if ("3.0.0".equals(dataProductVersion)) {
// Logic to migrate data to next available version 3.1.0
// And update the PRODUCT_VERSION column of the row to
3.1.0
dataProductVersion = "3.1.0";
}
if ("3.1.0".equals(dataProductVersion)) {
// Logic to migrate data to next available version 3.2.0
// And update the PRODUCT_VERSION column of the row to
3.2.0
dataProductVersion = "3.2.0";
}
}
}
}
return api;
}
}
The above interface implementation will be called from within DOA layer
before returning an object instance.
For the 3.0.0 release we just need to implement steps *1* and *2* above.
Step *3* can be done for all subsequent releases.
[1] https://flywaydb.org
--
Regards,
Uvindra
Mobile: 777733962
_______________________________________________
Architecture mailing list
[email protected]
https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
