Hi Phillip,
This is what I would do
Scenario 1
Assumptions:
* You have set a project folder with all your projects, environments,
database connections, etc.
* You have dset either a server or a container with hop
Job:
* Create a job that has the list of projects/environments
* Trigger a container or curl to the server that performs the required jobs
for each of the project/ environment combinations.
i.e. you are running a loop on a multi environment setup.
Scenario 2
Assumptions:
* You have setup some form of keystore
* setup an account that can access the required keys
Job
* job has the list of projects/environments.
* Retrieve the list of keys for the environment combination
* Execute a loop that performs the required actions
i.e. this is just a simple loop that manages the security in a way where you
don't keep things out of the key vault
Diego
[ https://www.bizcubed.com.au/ |
] Diego Mainou
Product Manager
M. +61 415 152 091
E. [ mailto:[email protected] | [email protected] ]
[ https://www.bizcubed.com.au/ | www.bizcubed.com.au ] [
https://www.bizcubed.com.au/ |
]
From: "Phillip Brown" <[email protected]>
To: "users" <[email protected]>
Sent: Thursday, 26 October, 2023 2:53:41 PM
Subject: variables, environments, databases, OH MY!
Hi all
They say the best way to learn something new is to use it solve a
specific problem. I am wanting to do that with Apache Hop, but am stuck
on a particular part.
Requirement:
I am responsible for managing a number of application databases for our
various business units. We have people who support those applications
who periodically require the ability to update data in specific tables.
To facilitate that I have implemented a system where they can log a
request for update access into a central repository, and periodically
that repository will be synced with the various databases. The access
they are granted is (mostly) temporary, and expires after a number of
days. The system to do the syncing currently uses a mix of shell and
python, and works pretty well. Additionally, the passwords used in each
of the target databases are different.
the basic flow is:
1) get a list of databases which need to be synced (based on recent
requests and/or expiry) from the repository
2) for each database, get a list of what should be in that database from
the repository
3) then in the target database, remove anything that shouldn't be there,
and add anything that isn't there
The bit I am having trouble with is the 'for each database'. I expect I
am going to have to use variables, and thought I might be able to use an
environment per database, but I can't see how (or if) you can tell Hop
to use a different environment once you've started a workflow or a
pipeline. So somehow it will need to be all one environment, but I don't
really want to have a squillion different variables.
So, any guidance in what pattern to use in setting this up would be
appreciated.
Thanks and regards
Phil Brown.
