Re: Composing Stuart Sierra's components
Hi Dan,
The key to understanding what's happening here is to
remember that `component/start` combines both dependency
ordering *and* dependency injection.
Your super system looks like this just after it is
constructed:
{:system {:foo {}, :bar {}},
:system2 {:baz {}, :qux {}}}
When you call `component/start` on this system, it
dispatches to `start-system`, which will build the
dependency graph to see that :system must be started before
:system2.
Next, start-system will call `component/start` on the
component at :system. That component is itself a SystemMap,
so it dispatches to `start-system`, which starts :foo and
:bar.
Now it's time to start :system2. start-system sees that
:system2 has a dependency (`component/using`) on :system. So
it will `assoc` :system into the component at :system2. Now
the system looks like this:
{:system {:foo {:started true},
:bar {:started true,
:foo {:started true}}},
:system2 {:baz {},
:qux {},
:system {:foo {:started true},
:bar {:started true,
:foo {:started true}
Notice that :system2 now contains a complete copy of
:system.
The rest should be obvious. Starting :system2 will start
:baz, :qux, *and* the nested copy of :system. So :system
gets started again. :foo and :bar were already started, but
`component/start` doesn't know that, so it starts them
again.
This is another reason I don't recommend nesting systems:
the behavior is not obvious unless you deeply understand the
model.
There are 2 ways to prevent the repeated starting of :foo
and :bar in this example.
1. Define a custom record for your super system
implementing component/Lifecycle in a way that knows how
to start the subsystems in the correct order without
`assoc`ing their dependencies.
2. Define the `start` and `stop` methods of :foo and :bar to
check if they have already been started before starting
them again. (i.e. make them idempotent)
-S
On Wednesday, April 15, 2015 at 7:52:42 PM UTC+1, Dan Kee wrote:
Sorry to resurrect an old thread with a somewhat tangential question but...
I'm seeing strange behavior in nesting systems that I am hoping someone
can explain. I have two independent systems as components of a super
system, with an artificial dependency to attempt to enforce ordering of
starting them. When I start the super system, the first system starts,
then the second, then first system is started again and I don't understand
why. I've since realized much simpler, more obvious ways to accomplish
this, but I'd like to understand what I'm seeing.
Code (apologies for the macro, but it keeps things brief):
```
(ns component-debug
(:require [com.stuartsierra.component :as c]) )
(defn capitalize-symbol [s]
(symbol (clojure.string/capitalize (str s))) )
(defmacro make-example-component [name]
(let [capitalized-name (capitalize-symbol name)]
`(do
(defrecord ~capitalized-name []
c/Lifecycle
(start [~name]
(println (str Starting '~name))
(assoc ~name :started true) )
(stop [~name]
(println (str Stopping '~name))
(assoc ~name :started false) ) )
(defn ~(symbol (str new- name)) []
(~(symbol (str map- capitalized-name)) {}) ) ) ) )
(make-example-component foo)
(make-example-component bar)
(make-example-component baz)
(make-example-component qux)
(defn new-system []
(c/system-map
:foo (new-foo)
:bar (c/using
(new-bar)
[:foo] ) ) )
(defn new-system2 []
(c/system-map
Re: Composing Stuart Sierra's components
Ah, that makes sense. Thanks for the thorough response!
On Thursday, April 16, 2015 at 7:00:16 AM UTC-5, Stuart Sierra wrote:
Hi Dan,
The key to understanding what's happening here is to
remember that `component/start` combines both dependency
ordering *and* dependency injection.
Your super system looks like this just after it is
constructed:
{:system {:foo {}, :bar {}},
:system2 {:baz {}, :qux {}}}
When you call `component/start` on this system, it
dispatches to `start-system`, which will build the
dependency graph to see that :system must be started before
:system2.
Next, start-system will call `component/start` on the
component at :system. That component is itself a SystemMap,
so it dispatches to `start-system`, which starts :foo and
:bar.
Now it's time to start :system2. start-system sees that
:system2 has a dependency (`component/using`) on :system. So
it will `assoc` :system into the component at :system2. Now
the system looks like this:
{:system {:foo {:started true},
:bar {:started true,
:foo {:started true}}},
:system2 {:baz {},
:qux {},
:system {:foo {:started true},
:bar {:started true,
:foo {:started true}
Notice that :system2 now contains a complete copy of
:system.
The rest should be obvious. Starting :system2 will start
:baz, :qux, *and* the nested copy of :system. So :system
gets started again. :foo and :bar were already started, but
`component/start` doesn't know that, so it starts them
again.
This is another reason I don't recommend nesting systems:
the behavior is not obvious unless you deeply understand the
model.
There are 2 ways to prevent the repeated starting of :foo
and :bar in this example.
1. Define a custom record for your super system
implementing component/Lifecycle in a way that knows how
to start the subsystems in the correct order without
`assoc`ing their dependencies.
2. Define the `start` and `stop` methods of :foo and :bar to
check if they have already been started before starting
them again. (i.e. make them idempotent)
-S
On Wednesday, April 15, 2015 at 7:52:42 PM UTC+1, Dan Kee wrote:
Sorry to resurrect an old thread with a somewhat tangential question
but...
I'm seeing strange behavior in nesting systems that I am hoping someone
can explain. I have two independent systems as components of a super
system, with an artificial dependency to attempt to enforce ordering of
starting them. When I start the super system, the first system starts,
then the second, then first system is started again and I don't understand
why. I've since realized much simpler, more obvious ways to accomplish
this, but I'd like to understand what I'm seeing.
Code (apologies for the macro, but it keeps things brief):
```
(ns component-debug
(:require [com.stuartsierra.component :as c]) )
(defn capitalize-symbol [s]
(symbol (clojure.string/capitalize (str s))) )
(defmacro make-example-component [name]
(let [capitalized-name (capitalize-symbol name)]
`(do
(defrecord ~capitalized-name []
c/Lifecycle
(start [~name]
(println (str Starting '~name))
(assoc ~name :started true) )
(stop [~name]
(println (str Stopping '~name))
(assoc ~name :started false) ) )
(defn ~(symbol (str new- name)) []
(~(symbol (str map- capitalized-name)) {}) ) ) ) )
(make-example-component foo)
(make-example-component bar)
(make-example-component baz)
(make-example-component qux)
(defn new-system []
(c/system-map
:foo (new-foo)
:bar (c/using
(new-bar)
[:foo] ) ) )
Re: Composing Stuart Sierra's components
Sorry to resurrect an old thread with a somewhat tangential question but...
I'm seeing strange behavior in nesting systems that I am hoping someone can
explain. I have two independent systems as components of a super system,
with an artificial dependency to attempt to enforce ordering of starting
them. When I start the super system, the first system starts, then the
second, then first system is started again and I don't understand why.
I've since realized much simpler, more obvious ways to accomplish this,
but I'd like to understand what I'm seeing.
Code (apologies for the macro, but it keeps things brief):
```
(ns component-debug
(:require [com.stuartsierra.component :as c]) )
(defn capitalize-symbol [s]
(symbol (clojure.string/capitalize (str s))) )
(defmacro make-example-component [name]
(let [capitalized-name (capitalize-symbol name)]
`(do
(defrecord ~capitalized-name []
c/Lifecycle
(start [~name]
(println (str Starting '~name))
(assoc ~name :started true) )
(stop [~name]
(println (str Stopping '~name))
(assoc ~name :started false) ) )
(defn ~(symbol (str new- name)) []
(~(symbol (str map- capitalized-name)) {}) ) ) ) )
(make-example-component foo)
(make-example-component bar)
(make-example-component baz)
(make-example-component qux)
(defn new-system []
(c/system-map
:foo (new-foo)
:bar (c/using
(new-bar)
[:foo] ) ) )
(defn new-system2 []
(c/system-map
:baz (new-baz)
:qux (c/using
(new-qux)
[:baz] ) ) )
(defn new-super-system []
(c/system-map
:system (new-system)
:system2 (c/using
(new-system2)
[:system] ) ) )
(defn -main [ args]
(c/start (new-super-system)) )
```
Output:
```
Starting foo
Starting bar
Starting baz
Starting qux
Starting foo
Starting bar
```
Thank you!
--Dan
On Wednesday, March 18, 2015 at 8:51:17 AM UTC-5, platon...@gmail.com wrote:
A possible implementation for the idea expressed in the previous post -
https://github.com/stuartsierra/component/pull/25
On Wednesday, March 18, 2015 at 2:41:46 PM UTC+1, platon...@gmail.com
wrote:
I've also been investigating the nested system approach/problem.
The primary use case that I have is composing subsystems which are mostly
independent modules using a higher order system to run in one process. The
modules themselves can be easily extracted into separate applications thus
becoming their own top level systems which makes it desirable to keep their
system maps intact.
Components inside modules might depend on the *whole* modules, not their
constituent parts. This allows to have modules call each other through the
API's in-process as well as being easily replaced by remote endpoints when
separated into multiple processes.
This mostly works except for the components depending on other
modules/systems, e.g.:
(require '[com.stuartsierra.component :as cmp])
(defrecord X [x
Re: Composing Stuart Sierra's components
On Tue, Mar 17, 2015 at 5:47 PM, James Gatannah james.gatan...@gmail.com wrote: FWIW, we've been using something that smells an awful lot like nested systems for months now. I never realized we weren't supposed to. It's not that nested systems *never* work, but from what I've seen they cause more complications than they're worth. The 'component' model doesn't forbid it, but it does not support dependencies between components in different subsystems. I've found it easier to keep system maps flat and use namespaced keywords to distinguish subsystem groups, even in large systems with 30+ components. –S -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups Clojure group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: Composing Stuart Sierra's components
I've also been investigating the nested system approach/problem.
The primary use case that I have is composing subsystems which are mostly
independent modules using a higher order system to run in one process. The
modules themselves can be easily extracted into separate applications thus
becoming their own top level systems which makes it desirable to keep their
system maps intact.
Components inside modules might depend on the *whole* modules, not their
constituent parts. This allows to have modules call each other through the
API's in-process as well as being easily replaced by remote endpoints when
separated into multiple processes.
This mostly works except for the components depending on other
modules/systems, e.g.:
(require '[com.stuartsierra.component :as cmp])
(defrecord X [x started]
cmp/Lifecycle
(start [this] (if started (println Already started x) (println
Starting x this)) (assoc this :started true))
(stop [this] (println Stopping x this) this))
(def sys1 (cmp/system-map :x (cmp/using (X. depends on dep nil) [:dep])))
(def sys2 (cmp/system-map :y (cmp/using (X. depends on sys1 nil)
[:sys1])))
(def hsys (cmp/system-map :sys1 (cmp/using sys1 [:dep]), :sys2 (cmp/using
sys2 [:sys1]) :dep (X. dependency nil)))
(cmp/start hsys)
Starting dependency #user.X{:x dependency, :started nil}
Already started dependency
Starting depends on dep #user.X{:x depends on dep, :started nil, :dep
#user.X{:x dependency, :started true}}
clojure.lang.ExceptionInfo: Error in component :sys2 in system
com.stuartsierra.component.SystemMap calling
#'com.stuartsierra.component/start
clojure.lang.ExceptionInfo: Missing dependency :dep of clojure.lang.Keyword
expected in system at :dep
This happens because of the following:
1. Dependency :*dep* of *sys1* is started in *hsys*
2. *sys1* is started (:*dep* is started again, so the start/stop should be
idempotent)
3. Dependency :*sys1* of *sys2* is started in *hsys*
4. :*sys1* cannot be started as it depends on :*dep* which isn't present in
*sys2*
This scenario could be supported by the Component library in several ways:
1. introduce an IdempotentLifecycle protocol which will be respected by the
Component library. Implement the protocol for the SystemMap.
IdempotentLifecycles will not be started or stopped for the second time,
also their dependencies will not be updated if they are already started.
2. do not fail if a component already has a dependency under the specified
key. This is a hack compared to the first solution, but I might go with it
in the short term.
Stuart, what do you think about that? Would you consider a PR implementing
the first proposal?
On Wednesday, March 18, 2015 at 10:18:36 AM UTC+1, Stuart Sierra wrote:
On Tue, Mar 17, 2015 at 5:47 PM, James Gatannah james.g...@gmail.com
javascript: wrote:
FWIW, we've been using something that smells an awful lot like nested
systems for months now. I never realized we weren't supposed to.
It's not that nested systems *never* work, but from what I've seen they
cause more complications than they're worth. The 'component' model doesn't
forbid it, but it does not support dependencies between components in
different subsystems.
I've found it easier to keep system maps flat and use namespaced
keywords to distinguish subsystem groups, even in large systems with 30+
components.
–S
--
You received this message because you are subscribed to the Google
Groups Clojure group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
---
You received this message because you are subscribed to the Google Groups
Clojure group.
To unsubscribe from this group and stop receiving emails from it, send an email
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Re: Composing Stuart Sierra's components
A possible implementation for the idea expressed in the previous post -
https://github.com/stuartsierra/component/pull/25
On Wednesday, March 18, 2015 at 2:41:46 PM UTC+1, platon...@gmail.com wrote:
I've also been investigating the nested system approach/problem.
The primary use case that I have is composing subsystems which are mostly
independent modules using a higher order system to run in one process. The
modules themselves can be easily extracted into separate applications thus
becoming their own top level systems which makes it desirable to keep their
system maps intact.
Components inside modules might depend on the *whole* modules, not their
constituent parts. This allows to have modules call each other through the
API's in-process as well as being easily replaced by remote endpoints when
separated into multiple processes.
This mostly works except for the components depending on other
modules/systems, e.g.:
(require '[com.stuartsierra.component :as cmp])
(defrecord X [x started]
cmp/Lifecycle
(start [this] (if started (println Already started x) (println
Starting x this)) (assoc this :started true))
(stop [this] (println Stopping x this) this))
(def sys1 (cmp/system-map :x (cmp/using (X. depends on dep nil) [:dep])))
(def sys2 (cmp/system-map :y (cmp/using (X. depends on sys1 nil)
[:sys1])))
(def hsys (cmp/system-map :sys1 (cmp/using sys1 [:dep]), :sys2 (cmp/using
sys2 [:sys1]) :dep (X. dependency nil)))
(cmp/start hsys)
Starting dependency #user.X{:x dependency, :started nil}
Already started dependency
Starting depends on dep #user.X{:x depends on dep, :started nil, :dep
#user.X{:x dependency, :started true}}
clojure.lang.ExceptionInfo: Error in component :sys2 in system
com.stuartsierra.component.SystemMap calling
#'com.stuartsierra.component/start
clojure.lang.ExceptionInfo: Missing dependency :dep of
clojure.lang.Keyword expected in system at :dep
This happens because of the following:
1. Dependency :*dep* of *sys1* is started in *hsys*
2. *sys1* is started (:*dep* is started again, so the start/stop should
be idempotent)
3. Dependency :*sys1* of *sys2* is started in *hsys*
4. :*sys1* cannot be started as it depends on :*dep* which isn't present
in *sys2*
This scenario could be supported by the Component library in several ways:
1. introduce an IdempotentLifecycle protocol which will be respected by
the Component library. Implement the protocol for the SystemMap.
IdempotentLifecycles will not be started or stopped for the second time,
also their dependencies will not be updated if they are already started.
2. do not fail if a component already has a dependency under the specified
key. This is a hack compared to the first solution, but I might go with it
in the short term.
Stuart, what do you think about that? Would you consider a PR implementing
the first proposal?
On Wednesday, March 18, 2015 at 10:18:36 AM UTC+1, Stuart Sierra wrote:
On Tue, Mar 17, 2015 at 5:47 PM, James Gatannah james.g...@gmail.com
wrote:
FWIW, we've been using something that smells an awful lot like nested
systems for months now. I never realized we weren't supposed to.
It's not that nested systems *never* work, but from what I've seen they
cause more complications than they're worth. The 'component' model doesn't
forbid it, but it does not support dependencies between components in
different subsystems.
I've found it easier to keep system maps flat and use namespaced
keywords to distinguish subsystem groups, even in large systems with 30+
components.
–S
--
You received this message because you are subscribed to the Google
Groups Clojure group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
---
You received this message because you are subscribed to the Google Groups
Clojure group.
To unsubscribe from this group and stop receiving emails from it, send an email
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Re: Composing Stuart Sierra's components
On Thursday, March 12, 2015 at 10:16:10 AM UTC-5, Stuart Sierra wrote: On Wednesday, March 11, 2015, Colin Yates wrote: Nested systems don't really work. FWIW, we've been using something that smells an awful lot like nested systems for months now. I never realized we weren't supposed to. Maybe we just got lucky? - James -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups Clojure group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: Composing Stuart Sierra's components
Hi James, Do you have a code fragment/gist for the glue? On 17 March 2015 at 17:47, James Gatannah james.gatan...@gmail.com wrote: On Thursday, March 12, 2015 at 10:16:10 AM UTC-5, Stuart Sierra wrote: On Wednesday, March 11, 2015, Colin Yates wrote: Nested systems don't really work. FWIW, we've been using something that smells an awful lot like nested systems for months now. I never realized we weren't supposed to. Maybe we just got lucky? - James -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups Clojure group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout. -- You received this message because you are subscribed to the Google Groups Clojure group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups Clojure group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: Composing Stuart Sierra's components
Hi Jonah,
This is quite comparable to micro-services - each service is an abstraction
or at least a facade and wants to play in a bigger system, but each
micro-service may itself have its own stateful graph to maintain.
I think I will explore my original direction of having a
AComponentWhichDrivesAnEntireSystem and see how far I get with that.
On Wednesday, 11 March 2015 20:53:45 UTC, jonah wrote:
Hey Colin, it sounds like:
* if the 2 systems really can't function without each other, and their
start/stop lifecycles are tightly bound, then somehow they have to be
merged into a single system
or
* if the 2 systems can't be merged into a single system because of true
functional or lifecycle independence and non-overlapping dependencies, then
semantics for what it means for each system to not have the other available
at lifecycle event times have to be established
It smells like the latter may be the way to go. So in pseudocode, the
reusable-component-system that owns the reusable-component would not
receive an instance of the external-collaborator at creation time. Instead,
the reusable-component would have a somewhat weaker definition of its start
semantics, and perhaps would have a way to refer to the
external-collaborator only by name.
That is, perhaps the external-collaborator provided to the
reusable-component is really an external-collaborator-client that is
configured to know the name of the external-collaborator- which is owned by
larger-system- and shields the reusable-component in cases where the larger
system has not been started or the collaborator isn't available.
And perhaps similarly the larger-system has a reusable-component-client or
reusable-client-proxy that is able to bridge appropriately from
larger-system to the reusable-component.
And maybe both client components are configured with the same :bus for
cross-system communication (and perhaps the cross-system bus is owned by a
bridge system). Just riffing, as it's not precisely clear what the
semantics of the systems are.
Stuart's comment that he hasn't run into a need for systems of systems is
coming to mind. Perhaps it makes sense to break these apart more explicitly
in accordance with guidelines around managing micro-services. Easy to say,
of course. :)
Not sure if that's helpful
Jonah
On Wed, Mar 11, 2015 at 3:01 PM, Colin Yates colin...@gmail.com
javascript: wrote:
merge won't help as there will be name space clashes.
I wonder if a more elegant approach would be to construct the 'inner'
system and then assoc onto it the external dependencies it needs
before calling start.
On 11 March 2015 at 18:49, adrian...@mail.yu.edu javascript: wrote:
I believe I misunderstood your question; I didn't realize it was system
(as
opposed to any general component) specific. I think systems can be
merged
together (via 'merge'). Would that help?
On Wednesday, March 11, 2015 at 2:40:14 PM UTC-4, Colin Yates wrote:
Hi Adrian - I don't follow how that helps integrate two different
systems - I wonder if my question was unclear or I am missing
something in your answer. Would you mind posting some pseudo code to
clarify please?
On 11 March 2015 at 18:32, adrian...@mail.yu.edu wrote:
You can specify component dependencies using the 'using' function as
you
know. As long as you know the key of the component in the system you
can
specify this dependency wherever you construct the component. If you
want to
parameterize dependencies, write a constructor function which takes
the
external dependency as a value.
On Wednesday, March 11, 2015 at 2:17:12 PM UTC-4, Colin Yates wrote:
I have a non-trivial component which requires a bunch of internal
and
external collaborators to work. This component is itself re-usable.
What I really want to do is have ReusableComponent be a component
in a
system so it can pull its external collaborators. However,
ReusableComponent
constructs its own services etc. so it really want to be (or at
least
have
access to) a system.
For example, let's say I have the following:
(defrecord InternalToReusableComponent [bus]
(component/Lifecycle
(start [this]...))
(defrecord ReusableComponent [bus logger]
(component/Lifecycle
(start [this]
this)
))
(defn reusable-component-system [external-collaborator]
(component/system-map
:bus ()
:logger ()
:reusable-component (component/using (map-ReusableComponent {})
[:bus
:logger external-collaborator]))
Fine - I now have a system from which I can pull the reusable
component.
However, where does 'external-collaborator' come from? Obviously
there
is a
larger system which I want this component to be part of so I can do:
(defn larger-system []
(component/system-map
Re: Composing Stuart Sierra's components
merge won't help as there will be name space clashes.
I wonder if a more elegant approach would be to construct the 'inner'
system and then assoc onto it the external dependencies it needs
before calling start.
On 11 March 2015 at 18:49, adrian.med...@mail.yu.edu wrote:
I believe I misunderstood your question; I didn't realize it was system (as
opposed to any general component) specific. I think systems can be merged
together (via 'merge'). Would that help?
On Wednesday, March 11, 2015 at 2:40:14 PM UTC-4, Colin Yates wrote:
Hi Adrian - I don't follow how that helps integrate two different
systems - I wonder if my question was unclear or I am missing
something in your answer. Would you mind posting some pseudo code to
clarify please?
On 11 March 2015 at 18:32, adrian...@mail.yu.edu wrote:
You can specify component dependencies using the 'using' function as you
know. As long as you know the key of the component in the system you can
specify this dependency wherever you construct the component. If you
want to
parameterize dependencies, write a constructor function which takes the
external dependency as a value.
On Wednesday, March 11, 2015 at 2:17:12 PM UTC-4, Colin Yates wrote:
I have a non-trivial component which requires a bunch of internal and
external collaborators to work. This component is itself re-usable.
What I really want to do is have ReusableComponent be a component in a
system so it can pull its external collaborators. However,
ReusableComponent
constructs its own services etc. so it really want to be (or at least
have
access to) a system.
For example, let's say I have the following:
(defrecord InternalToReusableComponent [bus]
(component/Lifecycle
(start [this]...))
(defrecord ReusableComponent [bus logger]
(component/Lifecycle
(start [this]
this)
))
(defn reusable-component-system [external-collaborator]
(component/system-map
:bus ()
:logger ()
:reusable-component (component/using (map-ReusableComponent {})
[:bus
:logger external-collaborator]))
Fine - I now have a system from which I can pull the reusable
component.
However, where does 'external-collaborator' come from? Obviously there
is a
larger system which I want this component to be part of so I can do:
(defn larger-system []
(component/system-map
:external-collaborator (...)
:reusable-component (component/using (some-magic-glue)
[:external-collaborator])))
I am struggling to see what (some-magic-glue) should be. I imagine it
needs to be something like:
(defrecord SystemAdaptor [external-collaborator internal-system]
component/Lifecycle
(start [this]
(let [internal-system (or internal-system
(reusable-component-system
external-collaborator))
internal-system (component/start internal-system)]
(assoc this :internal-system internal-system)))
(stop [this]
(let [internal-system (:internal-system this)
internal-system (component/stop internal-system]
(assoc this :internal-system internal-system)))
but it all feels a bit yuck.
I can't merge the two systems because the reusable component is chocka
full of very fine grained command handlers and both the internal and
external systems will have their own 'bus' for example. I could
namespace
the keys but that again feels painful...
Hope that is clear - and I look forward to your thoughts :).
--
You received this message because you are subscribed to the Google
Groups Clojure group.
To post to this group, send email to clo...@googlegroups.com
Note that posts from new members are moderated - please be patient with
your
first post.
To unsubscribe from this group, send email to
clojure+u...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
---
You received this message because you are subscribed to the Google
Groups
Clojure group.
To unsubscribe from this group and stop receiving emails from it, send
an
email to clojure+u...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
--
You received this message because you are subscribed to the Google
Groups Clojure group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
---
You received this message because you are subscribed to the Google Groups
Clojure group.
To unsubscribe from this group and stop receiving emails from it, send an
email to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
--
You received this message because you are
Re: Composing Stuart Sierra's components
I like the idea of passing in the *key* of the external collaborator -
that's nice. Thanks Stuart.
I am surprised there isn't more call for nested systems - maybe there
is and this solution is sufficient-enough...
Again - thanks Stuart!
On 12 March 2015 at 15:16, Stuart Sierra the.stuart.sie...@gmail.com wrote:
On Wednesday, March 11, 2015, Colin Yates wrote:
I can't merge the two systems because the reusable
component is chocka full of very fine grained command
handlers and both the internal and external systems will
have their own 'bus' for example. I could namespace the
keys but that again feels painful...
That's exactly how I would do it.
Nested systems don't really work.
But one system can contain implicit groups of components.
Create each group as a map, then `merge` them into the
system.
Use namespaced keys to prevent clashes.
The constructor functions for each component (or component
group) can take an argument which tells them the names of their
dependencies in the system.
Pseudo example:
(defn reusable-component [collaborator-key]
(component/using (map-ReusableComponent {})
[:my/logger :my/bus collaborator-key]))
(defn reusable-group [collaborator-key]
{:my/logger ...
:my/bus ...
:my/reusable-component
(reusable-component collaborator-key)})
(defn new-system []
(merge (system-map :main/logger ...
:main/bus ...
:main/collaborator ...)
(reusable-group :main/collaborator)))
-S
--
You received this message because you are subscribed to the Google
Groups Clojure group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
---
You received this message because you are subscribed to the Google Groups
Clojure group.
To unsubscribe from this group and stop receiving emails from it, send an
email to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
--
You received this message because you are subscribed to the Google
Groups Clojure group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
---
You received this message because you are subscribed to the Google Groups
Clojure group.
To unsubscribe from this group and stop receiving emails from it, send an email
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Re: Composing Stuart Sierra's components
On Wednesday, March 11, 2015, Colin Yates wrote:
I can't merge the two systems because the reusable
component is chocka full of very fine grained command
handlers and both the internal and external systems will
have their own 'bus' for example. I could namespace the
keys but that again feels painful...
That's exactly how I would do it.
Nested systems don't really work.
But one system can contain implicit groups of components.
Create each group as a map, then `merge` them into the
system.
Use namespaced keys to prevent clashes.
The constructor functions for each component (or component
group) can take an argument which tells them the names of their
dependencies in the system.
Pseudo example:
(defn reusable-component [collaborator-key]
(component/using (map-ReusableComponent {})
[:my/logger :my/bus collaborator-key]))
(defn reusable-group [collaborator-key]
{:my/logger ...
:my/bus ...
:my/reusable-component
(reusable-component collaborator-key)})
(defn new-system []
(merge (system-map :main/logger ...
:main/bus ...
:main/collaborator ...)
(reusable-group :main/collaborator)))
-S
--
You received this message because you are subscribed to the Google
Groups Clojure group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
---
You received this message because you are subscribed to the Google Groups
Clojure group.
To unsubscribe from this group and stop receiving emails from it, send an email
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Re: Composing Stuart Sierra's components
Hey Colin, it sounds like:
* if the 2 systems really can't function without each other, and their
start/stop lifecycles are tightly bound, then somehow they have to be
merged into a single system
or
* if the 2 systems can't be merged into a single system because of true
functional or lifecycle independence and non-overlapping dependencies, then
semantics for what it means for each system to not have the other available
at lifecycle event times have to be established
It smells like the latter may be the way to go. So in pseudocode, the
reusable-component-system that owns the reusable-component would not
receive an instance of the external-collaborator at creation time. Instead,
the reusable-component would have a somewhat weaker definition of its start
semantics, and perhaps would have a way to refer to the
external-collaborator only by name.
That is, perhaps the external-collaborator provided to the
reusable-component is really an external-collaborator-client that is
configured to know the name of the external-collaborator- which is owned by
larger-system- and shields the reusable-component in cases where the larger
system has not been started or the collaborator isn't available.
And perhaps similarly the larger-system has a reusable-component-client or
reusable-client-proxy that is able to bridge appropriately from
larger-system to the reusable-component.
And maybe both client components are configured with the same :bus for
cross-system communication (and perhaps the cross-system bus is owned by a
bridge system). Just riffing, as it's not precisely clear what the
semantics of the systems are.
Stuart's comment that he hasn't run into a need for systems of systems is
coming to mind. Perhaps it makes sense to break these apart more explicitly
in accordance with guidelines around managing micro-services. Easy to say,
of course. :)
Not sure if that's helpful
Jonah
On Wed, Mar 11, 2015 at 3:01 PM, Colin Yates colin.ya...@gmail.com wrote:
merge won't help as there will be name space clashes.
I wonder if a more elegant approach would be to construct the 'inner'
system and then assoc onto it the external dependencies it needs
before calling start.
On 11 March 2015 at 18:49, adrian.med...@mail.yu.edu wrote:
I believe I misunderstood your question; I didn't realize it was system
(as
opposed to any general component) specific. I think systems can be merged
together (via 'merge'). Would that help?
On Wednesday, March 11, 2015 at 2:40:14 PM UTC-4, Colin Yates wrote:
Hi Adrian - I don't follow how that helps integrate two different
systems - I wonder if my question was unclear or I am missing
something in your answer. Would you mind posting some pseudo code to
clarify please?
On 11 March 2015 at 18:32, adrian...@mail.yu.edu wrote:
You can specify component dependencies using the 'using' function as
you
know. As long as you know the key of the component in the system you
can
specify this dependency wherever you construct the component. If you
want to
parameterize dependencies, write a constructor function which takes
the
external dependency as a value.
On Wednesday, March 11, 2015 at 2:17:12 PM UTC-4, Colin Yates wrote:
I have a non-trivial component which requires a bunch of internal and
external collaborators to work. This component is itself re-usable.
What I really want to do is have ReusableComponent be a component in
a
system so it can pull its external collaborators. However,
ReusableComponent
constructs its own services etc. so it really want to be (or at least
have
access to) a system.
For example, let's say I have the following:
(defrecord InternalToReusableComponent [bus]
(component/Lifecycle
(start [this]...))
(defrecord ReusableComponent [bus logger]
(component/Lifecycle
(start [this]
this)
))
(defn reusable-component-system [external-collaborator]
(component/system-map
:bus ()
:logger ()
:reusable-component (component/using (map-ReusableComponent {})
[:bus
:logger external-collaborator]))
Fine - I now have a system from which I can pull the reusable
component.
However, where does 'external-collaborator' come from? Obviously
there
is a
larger system which I want this component to be part of so I can do:
(defn larger-system []
(component/system-map
:external-collaborator (...)
:reusable-component (component/using (some-magic-glue)
[:external-collaborator])))
I am struggling to see what (some-magic-glue) should be. I imagine it
needs to be something like:
(defrecord SystemAdaptor [external-collaborator internal-system]
component/Lifecycle
(start [this]
(let [internal-system (or internal-system
(reusable-component-system
external-collaborator))
internal-system (component/start
Re: Composing Stuart Sierra's components
Hi Adrian - I don't follow how that helps integrate two different
systems - I wonder if my question was unclear or I am missing
something in your answer. Would you mind posting some pseudo code to
clarify please?
On 11 March 2015 at 18:32, adrian.med...@mail.yu.edu wrote:
You can specify component dependencies using the 'using' function as you
know. As long as you know the key of the component in the system you can
specify this dependency wherever you construct the component. If you want to
parameterize dependencies, write a constructor function which takes the
external dependency as a value.
On Wednesday, March 11, 2015 at 2:17:12 PM UTC-4, Colin Yates wrote:
I have a non-trivial component which requires a bunch of internal and
external collaborators to work. This component is itself re-usable.
What I really want to do is have ReusableComponent be a component in a
system so it can pull its external collaborators. However, ReusableComponent
constructs its own services etc. so it really want to be (or at least have
access to) a system.
For example, let's say I have the following:
(defrecord InternalToReusableComponent [bus]
(component/Lifecycle
(start [this]...))
(defrecord ReusableComponent [bus logger]
(component/Lifecycle
(start [this]
this)
))
(defn reusable-component-system [external-collaborator]
(component/system-map
:bus ()
:logger ()
:reusable-component (component/using (map-ReusableComponent {}) [:bus
:logger external-collaborator]))
Fine - I now have a system from which I can pull the reusable component.
However, where does 'external-collaborator' come from? Obviously there is a
larger system which I want this component to be part of so I can do:
(defn larger-system []
(component/system-map
:external-collaborator (...)
:reusable-component (component/using (some-magic-glue)
[:external-collaborator])))
I am struggling to see what (some-magic-glue) should be. I imagine it
needs to be something like:
(defrecord SystemAdaptor [external-collaborator internal-system]
component/Lifecycle
(start [this]
(let [internal-system (or internal-system (reusable-component-system
external-collaborator))
internal-system (component/start internal-system)]
(assoc this :internal-system internal-system)))
(stop [this]
(let [internal-system (:internal-system this)
internal-system (component/stop internal-system]
(assoc this :internal-system internal-system)))
but it all feels a bit yuck.
I can't merge the two systems because the reusable component is chocka
full of very fine grained command handlers and both the internal and
external systems will have their own 'bus' for example. I could namespace
the keys but that again feels painful...
Hope that is clear - and I look forward to your thoughts :).
--
You received this message because you are subscribed to the Google
Groups Clojure group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
---
You received this message because you are subscribed to the Google Groups
Clojure group.
To unsubscribe from this group and stop receiving emails from it, send an
email to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
--
You received this message because you are subscribed to the Google
Groups Clojure group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
---
You received this message because you are subscribed to the Google Groups
Clojure group.
To unsubscribe from this group and stop receiving emails from it, send an email
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Re: Composing Stuart Sierra's components
I believe I misunderstood your question; I didn't realize it was system (as
opposed to any general component) specific. I think systems can be merged
together (via 'merge'). Would that help?
On Wednesday, March 11, 2015 at 2:40:14 PM UTC-4, Colin Yates wrote:
Hi Adrian - I don't follow how that helps integrate two different
systems - I wonder if my question was unclear or I am missing
something in your answer. Would you mind posting some pseudo code to
clarify please?
On 11 March 2015 at 18:32, adrian...@mail.yu.edu javascript: wrote:
You can specify component dependencies using the 'using' function as you
know. As long as you know the key of the component in the system you can
specify this dependency wherever you construct the component. If you
want to
parameterize dependencies, write a constructor function which takes the
external dependency as a value.
On Wednesday, March 11, 2015 at 2:17:12 PM UTC-4, Colin Yates wrote:
I have a non-trivial component which requires a bunch of internal and
external collaborators to work. This component is itself re-usable.
What I really want to do is have ReusableComponent be a component in a
system so it can pull its external collaborators. However,
ReusableComponent
constructs its own services etc. so it really want to be (or at least
have
access to) a system.
For example, let's say I have the following:
(defrecord InternalToReusableComponent [bus]
(component/Lifecycle
(start [this]...))
(defrecord ReusableComponent [bus logger]
(component/Lifecycle
(start [this]
this)
))
(defn reusable-component-system [external-collaborator]
(component/system-map
:bus ()
:logger ()
:reusable-component (component/using (map-ReusableComponent {})
[:bus
:logger external-collaborator]))
Fine - I now have a system from which I can pull the reusable
component.
However, where does 'external-collaborator' come from? Obviously there
is a
larger system which I want this component to be part of so I can do:
(defn larger-system []
(component/system-map
:external-collaborator (...)
:reusable-component (component/using (some-magic-glue)
[:external-collaborator])))
I am struggling to see what (some-magic-glue) should be. I imagine it
needs to be something like:
(defrecord SystemAdaptor [external-collaborator internal-system]
component/Lifecycle
(start [this]
(let [internal-system (or internal-system
(reusable-component-system
external-collaborator))
internal-system (component/start internal-system)]
(assoc this :internal-system internal-system)))
(stop [this]
(let [internal-system (:internal-system this)
internal-system (component/stop internal-system]
(assoc this :internal-system internal-system)))
but it all feels a bit yuck.
I can't merge the two systems because the reusable component is chocka
full of very fine grained command handlers and both the internal and
external systems will have their own 'bus' for example. I could
namespace
the keys but that again feels painful...
Hope that is clear - and I look forward to your thoughts :).
--
You received this message because you are subscribed to the Google
Groups Clojure group.
To post to this group, send email to clo...@googlegroups.com
javascript:
Note that posts from new members are moderated - please be patient with
your
first post.
To unsubscribe from this group, send email to
clojure+u...@googlegroups.com javascript:
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
---
You received this message because you are subscribed to the Google
Groups
Clojure group.
To unsubscribe from this group and stop receiving emails from it, send
an
email to clojure+u...@googlegroups.com javascript:.
For more options, visit https://groups.google.com/d/optout.
--
You received this message because you are subscribed to the Google
Groups Clojure group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
---
You received this message because you are subscribed to the Google Groups
Clojure group.
To unsubscribe from this group and stop receiving emails from it, send an email
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Composing Stuart Sierra's components
I have a non-trivial component which requires a bunch of internal and
external collaborators to work. This component is itself re-usable.
What I really want to do is have ReusableComponent be a component in a
system so it can pull its external collaborators. However,
ReusableComponent constructs its own services etc. so it really want to be
(or at least have access to) a system.
For example, let's say I have the following:
(defrecord InternalToReusableComponent [bus]
(component/Lifecycle
(start [this]...))
(defrecord ReusableComponent [bus logger]
(component/Lifecycle
(start [this]
this)
))
(defn reusable-component-system [external-collaborator]
(component/system-map
:bus ()
:logger ()
:reusable-component (component/using (map-ReusableComponent {}) [:bus
:logger external-collaborator]))
Fine - I now have a system from which I can pull the reusable component.
However, where does 'external-collaborator' come from? Obviously there is a
larger system which I want this component to be part of so I can do:
(defn larger-system []
(component/system-map
:external-collaborator (...)
:reusable-component (component/using (some-magic-glue)
[:external-collaborator])))
I am struggling to see what (some-magic-glue) should be. I imagine it needs
to be something like:
(defrecord SystemAdaptor [external-collaborator internal-system]
component/Lifecycle
(start [this]
(let [internal-system (or internal-system (reusable-component-system
external-collaborator))
internal-system (component/start internal-system)]
(assoc this :internal-system internal-system)))
(stop [this]
(let [internal-system (:internal-system this)
internal-system (component/stop internal-system]
(assoc this :internal-system internal-system)))
but it all feels a bit yuck.
I can't merge the two systems because the reusable component is chocka full
of very fine grained command handlers and both the internal and external
systems will have their own 'bus' for example. I could namespace the keys
but that again feels painful...
Hope that is clear - and I look forward to your thoughts :).
--
You received this message because you are subscribed to the Google
Groups Clojure group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
---
You received this message because you are subscribed to the Google Groups
Clojure group.
To unsubscribe from this group and stop receiving emails from it, send an email
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Re: Composing Stuart Sierra's components
You can specify component dependencies using the 'using' function as you
know. As long as you know the key of the component in the system you can
specify this dependency wherever you construct the component. If you want
to parameterize dependencies, write a constructor function which takes the
external dependency as a value.
On Wednesday, March 11, 2015 at 2:17:12 PM UTC-4, Colin Yates wrote:
I have a non-trivial component which requires a bunch of internal and
external collaborators to work. This component is itself re-usable.
What I really want to do is have ReusableComponent be a component in a
system so it can pull its external collaborators. However,
ReusableComponent constructs its own services etc. so it really want to be
(or at least have access to) a system.
For example, let's say I have the following:
(defrecord InternalToReusableComponent [bus]
(component/Lifecycle
(start [this]...))
(defrecord ReusableComponent [bus logger]
(component/Lifecycle
(start [this]
this)
))
(defn reusable-component-system [external-collaborator]
(component/system-map
:bus ()
:logger ()
:reusable-component (component/using (map-ReusableComponent {}) [:bus
:logger external-collaborator]))
Fine - I now have a system from which I can pull the reusable component.
However, where does 'external-collaborator' come from? Obviously there is a
larger system which I want this component to be part of so I can do:
(defn larger-system []
(component/system-map
:external-collaborator (...)
:reusable-component (component/using (some-magic-glue)
[:external-collaborator])))
I am struggling to see what (some-magic-glue) should be. I imagine it
needs to be something like:
(defrecord SystemAdaptor [external-collaborator internal-system]
component/Lifecycle
(start [this]
(let [internal-system (or internal-system (reusable-component-system
external-collaborator))
internal-system (component/start internal-system)]
(assoc this :internal-system internal-system)))
(stop [this]
(let [internal-system (:internal-system this)
internal-system (component/stop internal-system]
(assoc this :internal-system internal-system)))
but it all feels a bit yuck.
I can't merge the two systems because the reusable component is chocka
full of very fine grained command handlers and both the internal and
external systems will have their own 'bus' for example. I could namespace
the keys but that again feels painful...
Hope that is clear - and I look forward to your thoughts :).
--
You received this message because you are subscribed to the Google
Groups Clojure group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
---
You received this message because you are subscribed to the Google Groups
Clojure group.
To unsubscribe from this group and stop receiving emails from it, send an email
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
