Hi Patrick, I really like the new structure and documentation, especially the help functionality for the commands is cool.
I tested the new CLI on a linux machine and everything worked as expected. There is a new variable in the .env file “SP_SUBNET”. Which service requires this environment variable? Is there anything I need to be aware of during an installation? Philipp > On 17. Aug 2020, at 21:40, Patrick Wiener <[email protected]> wrote: > > Good point - I guess both work. So it’s the user’s/developer’s choice. > > For the CLI, I currently modeled it in a flexible way, such that one could > also use the CLI > in order to start the services specified in the environment template in so > called „user“ mode. > > $ streampipes start —help > Start StreamPipes environment > > Usage: streampipes start [OPTIONS] > > Examples: > streampipes start > streampipes start --user > > Options: > -d, --dev Map docker service ports to host in dev mode (default) > -u, --user No docker service ports mapped to host (user mode) > > Thereby, only the UI port gets mapped to the host. Technically, I leverage a > multi-compose file deployment > where there always exists a `docker-compose.yml` and a > `docker-compose.dev.yml`, see here for reference [1]. > > The dev yaml „extends“ the corresponding base yaml service with additional > features, e.g. port mappings. > > When now selecting —user we only load the base yaml service description > ignoring the port mappings since the > corresponding `docker-compose.dev.yml` files are not part of the deployment > description. > > For the UI this is a little bit different [1] - we always have the ports > mapped and have an empty `dev` yaml thus > not extending anything - we only have it for ease of loading the files in the > for loop of the bash script, see > concatenate_compose_files() [3]. > > Patrick > > [1] > https://github.com/apache/incubator-streampipes-installer/tree/new_installer/cli/deploy/standalone/backend > > <https://github.com/apache/incubator-streampipes-installer/tree/new_installer/cli/deploy/standalone/backend> > > <https://github.com/apache/incubator-streampipes-installer/tree/new_installer/cli/deploy/standalone/backend > > <https://github.com/apache/incubator-streampipes-installer/tree/new_installer/cli/deploy/standalone/backend>> > [2] > https://github.com/apache/incubator-streampipes-installer/tree/new_installer/cli/deploy/standalone/ui > > <https://github.com/apache/incubator-streampipes-installer/tree/new_installer/cli/deploy/standalone/ui> > > <https://github.com/apache/incubator-streampipes-installer/tree/new_installer/cli/deploy/standalone/ui > > <https://github.com/apache/incubator-streampipes-installer/tree/new_installer/cli/deploy/standalone/ui>> > [3] > https://github.com/apache/incubator-streampipes-installer/blob/new_installer/cli/bin/common > > <https://github.com/apache/incubator-streampipes-installer/blob/new_installer/cli/bin/common> > > <https://github.com/apache/incubator-streampipes-installer/blob/new_installer/cli/bin/common > > <https://github.com/apache/incubator-streampipes-installer/blob/new_installer/cli/bin/common>> > > >> Am 17.08.2020 um 19:32 schrieb Dominik Riemer <[email protected] >> <mailto:[email protected]>>: >> >> Hi Patrick, >> >> very cool! I also like the new README a lot. >> I think it's a good idea to have a very easy-to-use docker-compose files for >> users. This also reduces the maintenance effort of the current installation >> scripts. >> Also, keeping the CLI for developers and separate helm charts makes sense, >> we can think of integrating the helm charts into the CLI for the future, but >> for the upcoming release I think we are totally fine with the new >> docker-compose files, CLI and Helm charts. >> >> One question I have: Which version are users encouraged to use for non-k8s >> production usage? Should we take the standard docker-compose files for that? >> The advantage is that all ports are closed by default there, which is >> probably a rather important security consideration. >> >> Dominik >> >> On 2020/08/17 10:20:49, Patrick Wiener <[email protected] >> <mailto:[email protected]> <mailto:[email protected] >> <mailto:[email protected]>>> wrote: >>> >>> Hi, >>> >>> I was working on completely refactoring the installer on the weekend [1] >>> >>> The problem was always having some sort of too many areas where we needed >>> to touch and >>> that we did not have a clear focus on what kinda user role should use what >>> kinda way to set up his/her >>> environment. >>> >>> Thus, I propose my initial work to discuss with you: >>> >>> - StreamPipes Compose - The User's Choice >>> - StreamPipes CLI - The Developer's Favorite >>> - StreamPipes k8s - The Operator's Dream >>> >>> ====== >>> StreamPipes Compose: User tailored >>> >>> Two compose files with a default minimal version including few pipeline >>> elments and a optional full option including more >>> pipeline elements. Thus this should work for almost all OS platforms having >>> Docker and Docker-Compose installed. So, >>> we do not need to maintain .bat or ps1 files for windows. >>> >>> No ports are mapped except port 80 - so minimal surface for conflicting >>> ports. >>> >>> ====== >>> StreamPipes CLI: Developer oriented >>> >>> Command line interface to easily setup various dev environments locally in >>> order to either write new pipeline elements or >>> new core features. >>> >>> Ports of services are mapped by default as specified in the respective >>> service directory. >>> >>> ====== >>> StreamPipes k8s: Operator oriented >>> >>> Allow StreamPipes to be deployed on k8s using helm. >>> >>> ====== >>> >>> Overall we layed the foundation. However, I guess we can further integrate >>> the k8s option in the CLI in order to deploy our >>> helm chart using the CLI. >>> >>> In addition, the CLI could be easily extended with new features such as >>> creating new project skeletons using our existing maven archetype. >>> >>> >>> Take a look at it, try it out. Discuss what kind of features to include or >>> tell whats your general opinion. >>> >>> >>> Cheers >>> Patrick >>> >>> >>> [1] >>> https://github.com/apache/incubator-streampipes-installer/tree/new_installer/cli >>> >>> <https://github.com/apache/incubator-streampipes-installer/tree/new_installer/cli> >>> >>> <https://github.com/apache/incubator-streampipes-installer/tree/new_installer/cli >>> >>> <https://github.com/apache/incubator-streampipes-installer/tree/new_installer/cli>> >>> >>> <https://github.com/apache/incubator-streampipes-installer/tree/new_installer/cli >>> >>> <https://github.com/apache/incubator-streampipes-installer/tree/new_installer/cli> >>> >>> <https://github.com/apache/incubator-streampipes-installer/tree/new_installer/cli >>> >>> <https://github.com/apache/incubator-streampipes-installer/tree/new_installer/cli>>>
