Dear devs, During the summer of 2019 a long thread <https://groups.google.com/forum/#!topic/prometheus-developers/F1Vp0rLk3TQ> on this mailing list discussed the various issues that the Prometheus project was facing with the adoption of the new Go modules dependency management system. Based on the various arguments and facts that came out of this thread and a discussion with the project's maintainers I offered a potential way out in the form of a generic tool <https://docs.google.com/document/d/1g7wIlxn9JBJkc-2lHAb2MfCYLltWZKyNlIO6SBW1x-8/edit?usp=sharing> .
This tool, dubbed 'modularise', has now left its initial development stages and is ready for wider use and experimentation by Go projects including, but not limited to, Prometheus. The constraints for a solution to the "modules problem" were, on the various discussions, as follows: 1. Keep Prometheus's codebase in a single repository to allow for quick iteration cycles. 2. Continue to use the existing semantic versioning for Prometheus releases. 3. Align with the strong semantic versioning requirements that Go modules puts on the contents of any given module. 4. Allow for the large ecosystem of projects around Prometheus to consume the "internal" implementation APIs (tsdb, promql, etc.) with which Prometheus is built in a developer-friendly manner. As with most problems involving dependency management and versioning, many potential solutions exist. However all the ones that had previously been evoked in the context of Go modules, inside or outside of the Prometheus project, are based on hard tradeoffs. One or more of the above 4 points is required to give in and in practice its usually the first and most important constraint, which in my eyes is ease-of-development, that is sacrificed. The 'modularise' tool provides a new solution, one that aims to not suffer from such a compromise. The simplified version of what 'modularise' does is to: - Copy a defined set of packages from an existing Go module to a separate directory. - Rewrite / rewire import paths where necessary. - Initialise a Go module in the newly created directory. - Publish the result to a git repository. As a result, external projects can depend on these automatically maintained & independent modules without depending on the core project. These modules can be versioned separately from the main Prometheus project with their own dedicated release cadence. These separate modules also have the benefit of not depending on the project from which they are extracted. This introduces the potential for considerably reducing the size of your downstream project's dependency graph. All this might look nice on paper (or maybe not if you dislike the idea ;) ) but nothing gives a better idea of what this means than a good example. Besides the 'modularise' source code now being available <https://github.com/modularise/modularise> there is also a secondary project <https://github.com/modularise/splitcron> which performs a daily run on the latest master commit of Prometheus (and soon other projects as well). The current setup produces two new modules: - prometheus-tsdb <https://github.com/modularise/prometheus-tsdb> - which contains the 'tsdb' subtree and some required associated packages. - prometheus-promql <https://github.com/modularise/prometheus-promql> - which contains the 'promql' subtree and depends on prometheus-tsdb. Although these projects, which are meant to be simple demos, will not be tagged or go through other forms of manual curation they should be consumable by other Go projects without issues. As the title of this message puts it: this is meant to be an experiment, so do share your thoughts, your ideas and experiment yourself by using the demo projects (although please don't yet deploy the result to production :) ). Last but not least I'd like to extend a special thank you to +Bartłomiej Płotka <[email protected]> for bringing me on to the original issue and discussion! Thank you for reading up to this point and happy developing! Duco -- You received this message because you are subscribed to the Google Groups "Prometheus Developers" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/prometheus-developers/CAP8_5EBj08U__jGx-Evu-p0As6zbgSsY3hKcK1dmoiy_d69ndg%40mail.gmail.com.
