Am 17.08.2012 um 02:45 schrieb Kent A. Reed: > > <I deleted the earlier exchanges> > > Michael: > > I think your alternative proposal is great. I hope you proceed > irrespective of any decision about the proposed extension to emcmodule.cc.
I thought through this and I think I agree, so read the below as 'proposed addendum to master'. mechanics & plumbing: - the redis-server package in the 10.04 package stream is very old (which would be ok) - the libhiredis-dev C client API (needed) starts with Precise only - the python-redis Python bindings is very old (needs checking) With respect to package availability we have: - 8.04 - no off-the-shelf parts - 10.04 - some parts off the shelf - precsie would have all parts needed This means we're better off with a self-contained fork for now. My experience is that the builds and self-tests for all three parts are harmless and free of surprises. When all required packages are available for all supported platforms, the code and build instructions can be taken out and replaced by package requirements. build requirement: I think redis shall be a required part of linuxcnc eventually, which is why I would unconditionally build except if all of the three above packages are available. Another reason why I'd unconditionally build: I would like folks to warm up to the concepts - it's new and will take a while until it sinks in. Nobody warms up to optional packages which arent included by default. Sizes: - redis server is some 25kloc C - hiredis API: 4kloc C - redis-py: 2.7kloc Python Build time is maybe 20secs walltime for the whole thing on my Mac. community dependency: there are many animals in the 'noSQL' jungle, coming from different genetic backgrounds, many from web applications. I chose redis among other reasons because it isnt tainted by Java, XML and other socially incompatible technologies. I dont think the redis work is endangered to fall out of favor soon. It's quite healthy, and the lead guy, Salvatore Sanfilippo, was hired by VMware to work on redis, which gives me some confidence that redis is here to stay. startup: the only open question IMO is whether the redis server should be unconditionally started by all configurations (the redis process does next to nothing on no traffic, and the memory requirements can be scaled down substantially). My approach would be to be: yes, started by default, with a default configuration, until explicitly disabled in the ini. The reason for that is: going through all configs and adding an explicit startup requirement is a mess. runtime dependency: parts desiring to use redis could either: - check if redis is disabled in the ini, and not connect in that case - initially connect to redis (default to localhost). If the redis server isnt running, this would result in an immediate connection reject telling the client that redis isnt available. licensing: I read this as ok, I still would appreciate a second look at: - redis: https://github.com/antirez/redis/blob/unstable/COPYING - hiredis: https://github.com/antirez/redis/blob/unstable/deps/hiredis/COPYING - redis-py: https://github.com/andymccurdy/redis-py/blob/master/LICENSE data model: redis being one of the 'NoSQL' type databases needs a bit of a data model, which in our case translates pretty much into 'organizing a namespace' (eg interp/interp.params/tool/config etc). This isnt area where I have great expertise but I'd try to organize the redis namespace in a single include file and python predefined query strings such that a data model reorganisation doesnt become a code-hunting safari throughout the source tree. features to be used: Redis does many things, but the ones relevant for linuxcnc are: - networked data structure storage with persistence : that I see as the core feature to be used - publish/subscribe interaction pattern: this is a very much needed communications pattern not supported in a useful way by NML, and my code example explored that. I would prefer that to be part of the messaging layer; assuming that going forward we have a vehicle replacing NML which supports that pattern it would not be a bright idea to have two different messaging layers providing similar functionality. That said, I wouldnt prohibit its use, but I suggest such use be thought through such that is isolated (say in a module) and can easily be replaced. - persistence: some thought has to be given to import of existing emc.var files, and manipulation/export from redis to an external file. Same goes for tool table (no tool table format discussions in this thread, pleaaaaase.) areas of applicability: - my original motivation to introduce redis was to replace the constrained tooltable communicated by Emcstat. This can be taken up again in a more determined way. - the UI/interpreter parameter range idea recently proposed by Daniel Rogge - if we are heading back to a fully distributed model in LinuxCNC, assuming we have support by a decent middleware layer (this will be my suggestion for LinuxCNC3), it does make sense to drop the 'all parts of LinuxCNC can access an inifile' assumption because some parts of it, say a realtime embedded backend, might not have a filesystem worth speaking of after all. In that case, retrieving config information through redis is the way to go - on startup, load the inifile into a redis namespace, done. A redis client api can run on even very restricted environments. - userland HAL components come to my mind. git mechanics: I will start a redis-integration branch off master, with the goal of bringing in the redis parts, and creating configuration and build support, configuration documentation, data model template, and maybe some usage examples. Until merged into master, folks desiring to work with redis should either merge that branch, or work with a local redis/hireds/redis-py installation. how does that sound? anything essential missing? - Michael > > When you introduced the idea of redis "or something like it" last fall I > was too wrapped up in my wife's ongoing difficulties to pay attention, > but looking back on your messages now I think it would make a great > addition to LinuxCNC. I'd be carrying coals to Newcastle to tell you how > convenient it is to be able, at run time, to define, stash, and retrieve > data in such an accessible container. > > In the past you've alluded to the fact that LinuxCNC would actually need > to use little of the growing redis API. My first thought was, we (that's > the editorial "we", meaning someone very much like you) could implement > the few needed capabilities. They aren't much different from what we > were doing in a pub/sub project I was working on in the early part of > this century. Then I got real. Why spend any time/effort at all to > reimplement and test what is already available under a *very* liberal > open-source license. We've got other problems to solve. > > Given the terms of the redis license, we could use either of the schemes > you outlined (Ubuntu package manager or git) unless/until either the > redis project dies or its further development is taken under wraps by > VMware, at which time we could simply distribute the last publicly > available source package as part of our source distribution (with proper > attribution, of course). > > Since the Ubuntu package manager currently offers "redis 2:1.2" and > early versions of its language bindings, I downloaded the redis 2.4.6 > server/client software directly from the project site this afternoon to > play with (my testcase is a simplified bit of the pub/sub project I > mentioned). Compared to the LinuxCNC distribution, it took no time at > all to build on my machine. I'll try your tool-table example later. > > Regards, > Kent > > PS - I know it doesn't matter in the great scheme of things, but it > would be nice if we could wean ourselves from automatically naming the > parameter file "emc.var". > > ------------------------------------------------------------------------------ > Live Security Virtual Conference > Exclusive live event will cover all the ways today's security and > threat landscape has changed and how IT managers can respond. Discussions > will include endpoint security, mobile security and the latest in malware > threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/ > _______________________________________________ > Emc-developers mailing list > [email protected] > https://lists.sourceforge.net/lists/listinfo/emc-developers ------------------------------------------------------------------------------ Live Security Virtual Conference Exclusive live event will cover all the ways today's security and threat landscape has changed and how IT managers can respond. Discussions will include endpoint security, mobile security and the latest in malware threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/ _______________________________________________ Emc-developers mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/emc-developers
