Hi all
They say the best way to learn something new is to use it solve a
specific problem. I am wanting to do that with Apache Hop, but am stuck
on a particular part.
Requirement:
I am responsible for managing a number of application databases for our
various business units. We have people who support those applications
who periodically require the ability to update data in specific tables.
To facilitate that I have implemented a system where they can log a
request for update access into a central repository, and periodically
that repository will be synced with the various databases. The access
they are granted is (mostly) temporary, and expires after a number of
days. The system to do the syncing currently uses a mix of shell and
python, and works pretty well. Additionally, the passwords used in each
of the target databases are different.
the basic flow is:
1) get a list of databases which need to be synced (based on recent
requests and/or expiry) from the repository
2) for each database, get a list of what should be in that database from
the repository
3) then in the target database, remove anything that shouldn't be there,
and add anything that isn't there
The bit I am having trouble with is the 'for each database'. I expect I
am going to have to use variables, and thought I might be able to use an
environment per database, but I can't see how (or if) you can tell Hop
to use a different environment once you've started a workflow or a
pipeline. So somehow it will need to be all one environment, but I don't
really want to have a squillion different variables.
So, any guidance in what pattern to use in setting this up would be
appreciated.
Thanks and regards
Phil Brown.
