Boy, this sure escalated quickly. Long ago we had similar discussions (for months on end) and decided on the Project Lifecycle hierarchy which is the following:
- A "Project Lifecycle" which has "Project Lifecycle Environments" - A "Project lifecycle Environment" which is linked to a single "Project". The terminology was picked from a standard definition on Wikipedia somewhere. Environments and Projects were exposed in the configuration and user interface. Lifecycles themselves are not yet exposed. The terminology matters. You can find elsewhere on the Internet that folks call these environments "project phases" and some other variants but that's not what we picked and it's not accurate either I think. A lot of environments are IMO not part of a phase but often serve completely separate purposes. A practical example of a Lifecyle would be a project A with a development, test and production environment. These 3 form the lifecycle of project A. The name is on any level intended to be something you can choose and that is the case right now. That's all this is really. Please note that this is a separate thing from the physical reality of these concepts. The project lifecycle could be somehow physically represented on the same machine, server, container, ... but likely not. This is not to say that this could not be the case in the future. Now you can mean "Project Lifecyle" when you say "Project" but that's not what we mean in Hop right now and actually it's kind of inaccurate I think. So when Peter says that he might want to have a single name represent A/Development and A/Test on a single machine and call it "A" then what we're talking about is the implementation of the Project Lifecycle. Typical tools that come with this are comparisons, diff extraction (for operators), promotion of artifacts from one environment to the next (a project lifecycle typically has an order) and so on. So far we haven't had that use-case yet and so we haven't focussed on this but that doesn't mean that class ProjectLifecycle doesn't exist :-) In the same way I think that some people simply didn't understand the (quite simple) hierarchy despite our best attempts at explaining and documenting this. Cheers, Matt On Tue, Nov 2, 2021 at 10:52 PM Brandon Jackson <[email protected]> wrote: > I wanted to put a perspective out there on environments. What I think we > would want to end up with are small collections of environments. > > For example, at Neo Corp, perhaps they have a single collection for their > company consisting of > 1) Production > 2) Development > 3) Test > > For a consultant type user, they work with multiple corporate entities and > have a collection each. > Nord Ltd. may have > 1) Production > 2) Test > Taho US Inc. > 1) Prod > 2) Dev > 3) Test > etc. > > What each user would want to do is have the ability to tell an ETL project > which collection and specific environment profile to use. > It would probably be good to separate out the environment support and > config files to hop/config/environment/(collection name)/(supporting flat > files or configs). > > There could be a special folder hop/config/environment/relations.json which > could keep track of client project / collection ties. This means you could > ship projects with environment collections separately, but the Hop UI could > parse and manage some of these relationships separate of the environment > and projects themselves. > The above would allow Hop to scan for these things. > > One other requirement is I would want some way to scan and report on a > project to understand where all the environmental tentacles are touching. > If I take any project right now, variables could be used or consumed > anywhere. Only design best practices lead one to have a tendency of > defining and using variables in particular places. > It would be cool if Matt's fun reporting project might highlight those > variables in a color, especially for variables with the same name. > (patterned usage). > > Hope this contributes positively to the direction of decision-making. > Another thing that would make Hop infinitely more useful is the return of > syntax highlighting in steps. It is hard to overstate how difficult it is > to write any kind of script in Hop at the moment. I have had to resort to > outside tools for preparing and formatting scripts and queries. A prior > ETL tool spoiled me. > > > > On Tue, Nov 2, 2021 at 3:19 AM Hans Van Akelyen < > [email protected]> > wrote: > > > Hi Hop enthusiasts, developers and philosophers > > > > I have had an interesting discussion with Sergio on Jira ticket HOP-3471 > > [1] but we would like to get a bit of a broader consensus. > > > > So the main question is: do we see an environment as something linked to > a > > single project or are they objects that can be reused for several > projects? > > Both approaches have pro's and con's but we need to agree on this as it > > would change user experience and maybe how they are stored. > > > > Currently an environment is a semi-standalone object, it is not linked > to a > > project in the hop-project.json but it is linked to a project in the > > hop-config.json. In the GUI all environments are shown, not only the > > environments linked to a project. When using hop-run I *think* you can > > point to an environment that is not linked to the project in the > > hop-config.json. > > > > My personal preference would be to unlink them completely and have > > environments that could be used by multiple projects. This gives greater > > flexibility when you would want to divide your projects on a lower level. > > con would be that you would always see all available environments defined > > in your hop-config.json. > > > > I would love for some opinions on the matter so it can be set in stone > once > > and for all. And we can update our docs to reflect this decision [2][3]. > > > > Cheers, > > Hans > > > > > > [1] https://issues.apache.org/jira/browse/HOP-3471 > > [2] > > > > > https://hop.apache.org/manual/latest/projects/projects-environments.html#top > > [3] > https://hop.apache.org/manual/latest/projects/index.html#_environments > > > -- Neo4j Chief Solutions Architect *✉ *[email protected]
