I doubt I can understand the whole of your idea and would aver that a working model / code is lot better for anyone equally interested to load , try and comment on it.
The goal is big and nice, utility not yet comprehensible fully, but perhaps a working code will clarify a bit more again to some one in the similar line of action. Go ahead and let the code + screen shots do the talk.. you might have good feedback On Tue, Jul 24, 2012 at 11:20 PM, Patrik Sundberg <[email protected] > wrote: > On Tue, Jul 10, 2012 at 8:15 PM, Patrik Sundberg wrote: > >> I've done a bit more studying and I think a combination of >> SystemAnnouncer events to catch method changes, pragmas to mark "graph >> nodes" and use of method wrappers could create a very nice way to shield >> the user of my graph API from thinking of identity caches and graph node >> details. >> >> The other cool feature of use is the new class builder and custom slots. >> For nodes that are not dynamic/calculations (what I call properties or >> relationship nodes) I think custom slots could be interesting but need to >> test out some ideas to organize thoughts. Obviously won't work for >> calculated/dynamic nodes since I need ability to take arguments for those >> and can't map to a slot. In principle all I'm after is to have an identity >> map and a graph (DAG) in the background whose management is hidden away >> from user so he can focus on domain logic and not graph stuff. >> >> Going on holiday for a week and a bit, shall start experimenting after >> that. >> > > Ok, so I'm back on this. My first idea is to do this: > > - Every node in my graph is either a piece of data (I call it a leaf or a > property node) or a node depending on other nodes that perform a > calculation (I call it a calc node for now) > > - All the nodes are part of an identity map which means a node can only > exist in one copy and any relations and dependencies will sit between these > unique nodes > > - I'd like to shield the user from all the identify map stuff etc as much > as possible. > > - A smalltalk object is an instance of a class, and it can participate in > the graph by providing leaf nodes and calc nodes. Those nodes are > associated with the messages of the object. > > - I have some ideas for how to bootstrap the dependency graph between > nodes etc and I'm not worried about that bit (on first calculation create > structure via reflection and a dependency stack) > > - The explicit and uggly way I can do things are like this: > Foo>>addOneTo: aNumberNode > ^GraphManager instance > getCalcNode: #addOneTo: > for: self > withArgs: #( aNumberNode ) > block: [ :node | > node value + 1 > ] > > - pretend this goes and asks the identity cache and either gets an > existing leaf node for the addOneTo: for this object for a particular > argument, or if none exist it creates it and attaches the block as the > calculation to perform when #value is sent to the node > > - let's say 'foo' is an instance of Foo class, then 'foo addOneTo: baz' > refers to the calc node, and '(foo addOneTo: baz) value' refers to the > value of the calculation > > - What I was thinking here was that I could use method wrappers to hide > all the junk, like this: > Foo>>addOneTo: aNode > <calcNode> > aNode value + 1 > > - I listen to all method add/update/remove announcements, and when a > method that is marked with the <calcNode> pragma is involved, what I do is > create a MethodWrapper that takes care of all the uggly stuff to get/create > the graph node via the identity cache and as #value method for the node > uses the method implementation the user originally provided (and replaces > the entry in the method dictionary) > > Does that sound doable? The browsers etc wont go nuts? > > If I do that, will system browsers still show the original source for the > method even though I've replaced it via the method dictionary and > MethodWrapper? > > I'm assuming stack traces and debugging etc will be just fine, will just > have the "hidden" node logic via the MethodWrappers included in the call > stack. > > I'll give that type of setup a go, just wanted to throw it out there and > hear if it's an obviously terrible idea first. > > > P.S. for leaf nodes (or any node not requiring an argument to be provided) > I think I could use custom slots instead of method wrappers and do the same > kind of logic as the method wrapper would do in the slot instead. I'll > worry about that later. > > > > On Jul 6, 2012 1:16 PM, "Patrik Sundberg" wrote: > >> On Fri, Jul 6, 2012 at 12:38 PM, S Krish wrote: >>> >>> You should read as much as you can about Kapital of JPMC. It does more >>>> or less what you describe for esoteric Prime Interest Derivatives and very >>>> scalably .. >>>> >>>> >>> I've heard of that one. Never worked at JPM so haven't seen it first >>> hand though. I'll see what I can find. In terms of concepts I'm fully on >>> top of it (have a background with other companies using similar ideas, but >>> not in smalltalk but their own in-house language). So the good part is that >>> I know exactly what I want conceptually, but I'm not stuck into smalltalk >>> enough to know the natural way to implement the concepts. It may sound like >>> a huge endeavour but I've done a lot of it in ruby and even as a side >>> project to my real job we're talking about a couple of months to have >>> something useful. >>> >>> >>>> I doubt if there is much in the public domain, but the fact that >>>> Kapital is the golden standard in this product line talks highly of why >>>> Smalltalk is THE platform for this kind of product. >>>> >>>> >>> I wouldn't say it's the golden standard, that's taking it a bit far, >>> there are others that I rate higher bit I'm also biased :) But I do agree >>> that smalltalk is a reasonably natural platform for these types of >>> concepts, hence why I'm exploring it. >>> >>> >>>> It would be nice to have a Pharo based derivative product that can >>>> easily beat Ruby/ Java / .Net at this dynamic visualization using standard >>>> browsers, where Nautilus too can serve your purpose.. or a home grown tree >>>> based browser.../ Grids.. >>>> >>>> >>> That's my hunch, that the tooling work would be much cut down. In my >>> current ruby mockup I'm kind of creating a poor man's image like experience >>> - it's jruby on the JVM, vim + vim-slime to send code fragments to the >>> "runtime", git, and most likely homegrown GUI to navigate the graph and >>> play with it (not done anything on that yet). I'll be keeping jruby as a >>> dear tool as right now it's a fantastic glue tool in terms of interacting >>> with anything out there in the world (i.e. I can easily work with the >>> bloomberg java API), but I'm very keen to explore pharo for my graph + >>> tooling around that. >>> >>> (slight tangent - coming from outside the smalltalk world it'd be HUGE >>> to me to have git backend for packages and use of github for collaboration. >>> seen that mentioned in other threads. I think it'd be HUGE for community's >>> ability to attract more people). >>> >>> >>> >>>> Once I have completed my run at a PharoTabletIDE/ PharoMorphicView >>>> framework, I would love to collaborate on your endeavour to see if it can >>>> be modelled on it and exposed.. >>>> >>>> >>> Cool. Early days yet and we'll see where I end up. Not familiar with >>> those projects, I shall take a look. >>> >>> >>> Patrik >>> >>> >>> >>> >>>> On Fri, Jul 6, 2012 at 4:08 PM, Patrik Sundberg wrote: >>>> >>>>> Hi, >>>>> >>>>> First of all, sorry for the long email as my first email to the list. >>>>> Hopefully some people will find it an interesting discussion :) >>>>> >>>>> I'm a long time programmer that has been studying smalltalk and pharo >>>>> over >>>>> the last year. It's a beautiful language I'm liking the image style of >>>>> development a whole lot. I've been looking for a good test case to try >>>>> out >>>>> in pharo and I've got something in mind. I've used ruby since around >>>>> 2000 >>>>> and given how much it has been inspired by smalltalk it's not a big >>>>> leap. >>>>> >>>>> My real job is being a commodities trader but I build my own tools and >>>>> something I'm always working on is improved risk and pricing tools. >>>>> Lots of >>>>> that comes down to 2 things: >>>>> 1. evaluating a "function" (in a general sense) with inputs that are >>>>> picked >>>>> depending on time. e.g. I want to price a security and I want to pick >>>>> up >>>>> prices for time X as inputs to the pricing model. >>>>> 2. calculating a derivative of the "function" above with respect to >>>>> inputs >>>>> >>>>> What I tend to do in order to keep things consistent by design, plus >>>>> add >>>>> natural caching, is create a calculation graph. I define all the >>>>> dependencies and create an in-memory graph where each node is unique >>>>> using >>>>> an identity map. I then perform a calculation where the calculations >>>>> trickle >>>>> down the graph starting from the node I'm interested in down all it's >>>>> dependencies. To calculate the derivative I then find the node >>>>> representing >>>>> the input I want the derivative with respect to and use finite >>>>> differences, >>>>> e.g. move it's value, and recalculate the top level node. On the second >>>>> valuation of the graph only the parts that are affected by me changing >>>>> that >>>>> 1 value will need to be evaluated, the rest is cached an unaffected. It >>>>> makes it very easy to ask questions like "How much will I be affected >>>>> by X >>>>> changing by Y?" since I can take the top level node of the graph, >>>>> search for >>>>> X, and if found change it by Y and recalculate. >>>>> >>>>> How the graph is constructed in memory depends on time (and a few other >>>>> things). I always store all previous states of the world so that I can >>>>> recreate any calculation I did in the past. Very useful for diagnosing >>>>> problem. Hence if I for example change what model I use to model >>>>> something, >>>>> that change is recorded in a way such that if I set my "time" to >>>>> before the >>>>> change happened the graph will get created as it would have with the >>>>> old >>>>> model, and for a time after the change it'll create a different graph >>>>> reflecting the new model. Hence the graph can only be known at runtime >>>>> when >>>>> the "time" etc is known. >>>>> >>>>> I currently have a ruby mockup of this. It's a DSL that looks like >>>>> this: >>>>> >>>>> ----------- EXAMPLE >>>>> >>>>> class ExampleObject >>>>> include GraphEntity # includes a module with functionality for >>>>> objects >>>>> participating in the graph, including #property and #calc used below >>>>> >>>>> property :foo do >>>>> 123 # this is the default value for this property which will be >>>>> used >>>>> before a value has been set and saved >>>>> end >>>>> >>>>> calc :bar do |arg| >>>>> foo.value + arg # take the value of the arg property and add >>>>> the >>>>> argument given to the calculation node >>>>> end >>>>> end >>>>> >>>>> o = ExampleObject.new >>>>> # this will kick of the graph being built in memory and setup the >>>>> dependency >>>>> between bar and foo nodes >>>>> o.bar(1).value # -> 124 >>>>> # this will be a "cache lookup" since nothing in the graph that bar >>>>> depends >>>>> on has changed (i.e. the "expensive" calculation is not performed again >>>>> o.bar(1).value # -> 124 >>>>> >>>>> o.foo.set_value(2) >>>>> o.bar(1).value # -> 125, realizes that foo changed and performs a >>>>> recalc >>>>> -------------------------------------------------------- >>>>> >>>>> To accomplish this I use dynamic code generation like the below for >>>>> #property and similar for other types of nodes: >>>>> >>>>> # NOTE: it's a class level method, hence how I can call it when >>>>> defining the class in the example >>>>> >>>>> def property(name, time = CdrEnv.instance.time, &block) >>>>> clear_property(name) >>>>> getter_method_body = <<-EOF >>>>> def #{name} >>>>> init_block = self.class.property_init_block_for(:#{name}) >>>>> time_for_property = >>>>> self.class.property_time_dependency_for(:#{name}) >>>>> if init_block.nil? >>>>> CdrEnv.instance.get_property_node(self, :#{name}, >>>>> time_for_property) >>>>> else >>>>> CdrEnv.instance.get_property_node(self, :#{name}, >>>>> time_for_property, &init_block) >>>>> end >>>>> end >>>>> EOF >>>>> setter_method_body = <<-EOF >>>>> def #{name.to_s}=(value) >>>>> #{name.to_s}.mutate(value) >>>>> end >>>>> EOF >>>>> class_eval getter_method_body >>>>> class_eval setter_method_body >>>>> register_property_node(name, time, &block) >>>>> end >>>>> >>>>> Don't worry about the details or all the unfamiliar ruby, the main >>>>> point is >>>>> that it creates an instance method that uses the singleton >>>>> CdrEnv.instance >>>>> to either get or create the node representing the property from the >>>>> graph >>>>> identity cache depending on if it exists or not. >>>>> >>>>> From a tooling point of view I think I'd love to work with this kind of >>>>> thing in pharo. Building my own browsers for inspecting and debugging >>>>> my >>>>> dynamic graph should be very good fit. However, I'd appreciate some >>>>> pointers >>>>> to idiomatic smalltalk to attack this kind of problem in terms of >>>>> implementing the graph itself - I obviously want the user (even if >>>>> it's me) >>>>> to just have to focus on the domain model and hide as much as possible >>>>> of >>>>> the graph bits under the covers the same way I've done with the code >>>>> generation stuff in my ruby example. >>>>> >>>>> Any input into this would be much appreciated, >>>>> Patrik >>>>> >>>>> P.S. >>>>> Btw, the persistence backend for this is the neo4j graph database, but >>>>> it's >>>>> fronted by a service slotting into my own service framework built using >>>>> ZeroMQ and services exchanging messages in protobuf format. One can >>>>> use it >>>>> from any language as long as one can send protobuf messages over >>>>> zeromq. I >>>>> see there's a zeromq ffi library available on SS that I'll check out, >>>>> but >>>>> I'm not finding a protobuf implementation. It'd be easy enough for me >>>>> to >>>>> port a ruby protobuf implementation but I may as well ask if someone >>>>> has >>>>> already done any work on protobuf for smalltalk? >>>>> >>>>> >>>>> -- >>>>> View this message in context: >>>>> http://forum.world.st/Dynamic-graph-exploration-seeking-input-on-idiomatic-smalltalk-LONG-tp4638702.html >>>>> Sent from the Pharo Smalltalk Users mailing list archive at Nabble.com. >>>>> >>>>> >>>> >>> >
