daniellavoie commented on issue #5580: URL: https://github.com/apache/incubator-pinot/issues/5580#issuecomment-647009056
> **Foreground or background :** > > I feel Foreground option will not be much useful unless for the quick start or someone trying the system for the first timers, because you have to close the services and relaunch in background for long running the system, instead of that we can support only one mode which is background and add the following command for debugging or exploration purpose, I have to disagree here. We should pause this conversation and take a step back and set ourselves goals. One of the big theme that I would like to see improvement is to make Pinot more friendly to Cloud Native deployments. If you want to run something in a container based environnement, it needs to run in foreground to fully embraces all the container lifecycle management offered by the container orchestrator. Logging management is also simplified because your container orchestrator only needs to retreive the stdout and stderr to forward to a logging platform. As for configuration management; environment variables and cli arguments composes the best practices to operate Cloud Native services. I do agree that there is use cases to run Pinot a a traditional Service Daemon. Once you have a well designed foreground service launcher, it is really easy to wrap it with a service package (such as Systemd) that turns it in a background process. Let me be clear, we have to offer an option for operators to run Pinot in foreground. If our base operating model is background, then we have to retrofit foreground semantics and it will an awkward exercice. It a lot easier to turn a foreground process to background. > I feel adding options like --controler.port=<port> or --cluster-name=<cluster-name> to the CLI will be bad design and can’t be scalable, we should restrict the options at service level like. And the options like brokers ulr, hosts or ports etc should be part of the properties/config file, that way you can still parse by args4j annotation properties and validate before passing to the systems, and at CLI level you only need to validate on predefine commands and options. As I explained earlier, property files are not what best practices guide to operate Cloud Native software. Works great for local dev environment but is not what is recommended for operating software on Cloud platforms. This is not bad design. Modern cloud native software all adopt this pattern. I already have a prototype of Pinot that uses this configuration pattern in a scalable way. Maintaining specific binding through `args4j` and manually wiring them the system configuration is, in my opinion, not scalable. Pinot should offer the best "ready to go" default properties and let you overwrite them the way that pleases you. @satishbellapu, my understanding is that you wish to offer a better "Pinot in a box" experience for developers who wish to iterate quickly on their environment. I think there is a lot of value there. My conclusion is that maybe we should have separate story to align when these two different goals. I honestly think they should be materialized in two separate tools. The operator focused `pinot` cli that aims to offer maximum flexibility for operators to run Pinot Services with freedom of choice regarding their target platform. And a `pinot-dev` cli that downloads, install dependencies and runs Pinot Services in an opinionated way to get productive. Under the hood, it would make a lot of sense for it to run pinot with the "lower level" `pinot` CLI. What do you think of that suggestion? ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
