Hi Jan,
I'm running Myosotis in a test environment using it for batch upload
about 4GB of data into a sequoia mysql cluster with two controllers
and two backends per controller. I've had massive performance and
stability problems before just using the sequoia console with the sql
client feature. Now, using Myosotis it runs like a charm.
Just to give an impression what I'm doing:
- Receiving preformatted csv-files form an I-series (AS400-DB2)
containing table data (product/prices/catalogdata/factoryprices/...)
via sftp
- Scheduled check for new data in the receiving directory
- Preparing data files for upload (splitting into manageable parts,
adding header and footer files)
- Deleting old data in table (Much faster than an update) by now
calling mysql-client
- Inserting new data calling mysql-client (datafile can be up 500 MB big)
The process is covered by error-checking, backup & retry handling and
mail alerts in case of failures and takes about 10 minutes for about
200MB.
If someone needs more infos or some hints, just ask.
This is a very interesting feedback. As Myosotis is adding another
proxying layer, it is hard for me to understand why it performs better
than the SQL console directly plugged into Sequoia. There is then
certainly a problem with the sql client feature that should work like
mysql-client.
By the way, did you check that all databases where properly updated and
that mysql-client was well going through Myosotis and not accessing
directly MySQL?
Could you provide us with rough performance numbers for sql
client/Sequoia vs mysql-client/Myosotis/Sequoia vs mysql-client/MySQL?
Thanks again for your feedback,
Emmanuel
_______________________________________________
Sequoia mailing list
[email protected]
https://forge.continuent.org/mailman/listinfo/sequoia
