Hi Peter, see below. Sent from my iPhone
> On Oct 30, 2017, at 5:23 AM, Peter Tribble <[email protected]> wrote: > > David, > > Thanks for the thoughts. I have both general and specific comments. > >> When I built systems under ATT SVR3, SVR4, Solaris Zones, and Solaris LDoms, >> I was always concerned about rules for scalability & self configuration. >> >> There is so much effort around Orchestration today, adding intelligence to >> central points, and not enough around intelligent systems which self >> configure. > > Quite. What worries me is that configuration management is now so concerned > with systems being configured > 100% perfectly that systems are no longer robust enough to be able to > continue if things aren't quite right. They've > become increasingly fragile as a result. > >> When Sun was talking about clustered filesystems merging with ZFS - I >> thought this could have been revolutionary. Plug in a new box, have it join >> the pool of redundant resources. Pull out a box, no loss of functionality. >> The metadata servers are a point of failure, but that was acceptable 10 >> years ago. >> >> Orchestration is too complex. Self configuration & communication beyond >> point-to-point is where there is space for innovation. > > So things like bonjour. Or even revisit the likes of Jxta and Jini. Light, tight, with as little code & underlying prerequisites as possible - to reduce platform resource consumption & security vulnerabilities. >> Think about our past: >> - Right thinking was unloading a disk less workstation off the dock & >> placing a MAC address in a boot server to make a desktop work. > > And servers. Take the little yellow slip off a Sun server, set up jumpstart, > and your compute farm is good > to go. Many of my Sun servers *never* had a serial console plugged in. > > You can largely do that now, still. > >> - Right thinking was taking a Sun Ray out of the closet and plugging it in >> to make a workable desktop. > > The whole SunRay area is another one lost completely. There are still people > who do > some sort of zero-admin desktop experience, but there's clearly room to > improve. A Raspberry Pi implementation of a light weight Illumos system would bring us right back to this point - bolt it on the back of an old SunRay or monitor, Bluetooth keyboards & pointer functionality. >> - Right thinking was the example of loading Solaris 10 with dual identical >> drives with zfs root - it knows mirroring with boot block replication was >> the right thing to do. >> - Right thinking was commands like "ruptime" - where the health of the >> network was readily known so people & apps could automatically decide where >> to put load. Worked out of the box. > > Sort of. Clearly of its time. The whole idea was broken, but worked well > enough for some purposes that > people didn't worry too much. The protocol is broadcast, exposes a limited > set of statistics of dubious > utility, and has no knowledge of sensible workload placement. > > I used to use that sort of thing back in the 1980s, but it's essentially > useless today. We all know the limitations from that period of time, but the point is that it was a reasonable start. I used to distribute workloads, it was lightweight & helpful. > Thinking sideways, where are the tools like Grid Engine and NQS? Are they > interesting, or does everyone > think that Mesos or K8S is the way to schedule workloads across a distributed > system? I would like something which could be distributed across Illumos on future version of Pi Zeros. Very lightweight, very few pre-requisites. >> - Right thinking was network capable "talk" commands, where users could talk >> to users everywhere, transparently in the network. Worked out of the box. >> - Right thinking was "finger", where people understood presence of >> individuals, and could communicate freely, transparently in the network. >> Worked out of the box. > > Never worked sanely on a distributed system, where users might be in multiple > locations. I used to use it between sites on the internet, to identify old colleagues, chat, etc. across the continent. We were on different systems, some not tied to Solaris, but stuck on older systems (i.e. Ultrix) - it worked well for us. It was annoying that the SVR4 talk was not cleanly compatible with BSD - we had to use the right command, for the odd colleague out. Of course, there are security concerns with the old implementations - but the concepts were solid. >> - Right thinking was "X", where application servers were decoupled from >> display servers... where a single multithreaded application easily attached >> to multiple users on a network and served up app displays while >> communicating over simple in-memory constructs. (Games like xtank & xbattle >> demonstrated how easy realtime multiuser collaboration with multiple >> displays & a single running image was with less than a floppy worth of >> binaries.) > > While the trend might be towards distributed *computation*, the trend is > towards centralized *interfaces* to that work. > >> - Right thinking was network capable store-and-forward mail built into the >> box with relatively simple ability to mail to other boxes over a network. >> Worked out of the box. Still used. >> - Right thinking was queuing via "lp" to printers over a network with >> adapters to push to/from anything. (People buy very complex publish & >> subscribe systems today, but it is the same concept.) >> - Right thinking was Threaded "NNTP" readers, to subscribe and push content >> to the desktop... content could go anywhere. RSS grew from Microsoft axing >> this protocol with IE. (This need will not go away - think: orbit, moon, >> mars, asteroid belt, etc.) >> - Right thinking was bundling NFS & automounting. Just plug in & access it, >> if you know the path. >> - Right thinking was ubiquitous http and web browsers with search engines... >> which we had long before that with anonymous ftp sites and query hosts. >> >> If I had 1 hour in a room with Illumos developers - I would appeal >> innovation should ONLY be around 1:m protocols on networks for self >> configuration... without central orchestration, without TCP, without >> reinventing protocols, without central directories, with dynamic distributed >> & replicated directories. > > You need a single source of truth, but I would agree that it should be a > distributed implementation. The single source of truth is always an API. I think we are too bogged down in centralized databases. The database can be distributed, as well, access via API. > I've been wondering about things like using AWS metadata to autoconfigure > Amazon instances as a specific example of this class of asking the environment > around you to decide what to do next. > >> Data replication of critical data (to many nodes), metadata replication (all >> OS configuration), read-only OS snapshots, no dependency on ZFS only (revive >> UFS with snapshots for limited use cases, other future open source FS's), > > Tribblix supports UFS root. It's fine if you're resource constrained, but has > serious > limitation as other parts of the OS make assumptions about the root > filesystem. Understood. Raspberry Pi target, again. Constrained distributed systems... Push smaller to a future generation of ESP8266... it is all about IoT. Think about the laterals... - Illumos on hard drive firmware? - RAIDZ in a distributed system where drives talk to each other without a controller? - RAIDZ distributed between SSD's without a controller - RAIDZ on top of RAIDZ in SSD's in a distributed environment I don't think there needs to be a single source of truth. Just a majority of nodes, with enough error correction to make up the difference. There is room for radical architecture vision with immediate implementation opportunities. (We lost decades with the exit of Solaris from MPP systems.) >> self-replication of packages across a cluster (like pkgadd on a global zone >> to guest zones), cheap flash for volatile local data with ability to check & >> lock out bad spots (instead of crash & burn with junk USB sticks today), >> self load balance everything, run apps anywhere & display anywhere (X >> revival), run on a SPARC or Intel or Raspberry Pi (roadmap of past to >> present to future... or scalability from massive to medium to small if you >> prefer.) > > Distributed seems to be a theme. Along with autonomous behaviour. Very much so. IoT is the next wave - Illumos should be there. Ironically, Distributed Systems was the beginning of SunOS becoming relevant. The massive centralization, loss of simple management frameworks or extensible management frameworks (Admin Tool, FMLI, lack of porting XFMLI, etc.) seemed to coincide with decline from third-party integrators. Oracle's push toward centralized stacks caused further integrator decline. We see IoT explosion now, with similar needs & requirements that existed in the beginning of "our great journey." I hope you were able to see the parallel... a vision for the future based upon an imprint from the past, using a "fuzzy logic". Reasonably Autonomous Systems must become synonymous with Illumos to remain relevant and break into a new world. There is external funding available there, for people maintaining core system components and building higher level frameworks. Being the odd OS out with a legacy of security could give us added slideware benefit... since we will not be the popular target to hack. IoT with Autonomous Systems could explode our ecosystem overnight. An extensible management framework, with widgets, would be required... to ease adoption. People don't work with shells any longer. >> Honestly, everything exists, just tweaking is needed with combining things >> into core bundles, and then tweak a few things. Connect everything by >> network. Scale is not Google, it is the depths of the ocean, our Solar >> System (tcp is insufficient.) Think: Robotics. Apply: Farming with planting >> & weeding & harvesting, ad-hoc drone WiFi networks in natural disasters, >> pollinating bee barren fields in china. Imagine: Ender's Game. >> >> We've lived in a decade of little innovation. We know where to go. It is not >> hard. We base too much work on high level frameworks & languages with lots >> of bugs & significant dependencies... while lower level, well known, well >> debugged, well defined protocols should be the preference. With the end of >> heterogeneous computing, it is now time to end central orchestration & >> central directories, do it anywhere & everywhere, and allow others to join >> us for the fun. Illumos can be this. >> >> Thanks, David Halko >> http://www.netmgt.blogspot.com/ >> >> Sent from my iPhone >> >>> On Sep 13, 2017, at 4:01 PM, Peter Tribble <[email protected]> wrote: >>> >>> In my OpenSolaris t-shirt collection, I have one with the slogan: >>> >>> "Innovation happens everywhere" >>> >>> I'm not sure this is *entirely* true; Solaris 10 was a massive nexus >>> of innovation that has proliferated out to other operating systems >>> over the last decade. Frankly, there's not much else been happening >>> in systems development. >>> >>> From what I can see, between the cloying boredom of Linux monoculture >>> and the dead hand of POSIX "standardisation", systems have stagnated. >>> >>> Even in illumos, we're largely doing a bit of light gardening - a bit >>> of weeding here, a bit of pruning there, replanting the odd bush. But >>> no real landscaping is being done. >>> >>> Which begs the question - is systems innovation done and dusted? >>> >>> Or is there more to come? >>> >>> And if there is more, what sort of new features are wanted? >>> >>> At which point I open up the floor to anyone who wants to contribute. >>> >>> (Note: I'm not talking about a gaps analysis. We [illumos] need more >>> drivers, more applications ported. We already know that, and it's just >>> copying, not innovation. So there is an interesting subject there, but >>> if someone wants to follow that then please create a new thread.) >>> >>> Cheers, >>> >>> -- >>> -Peter Tribble >>> http://www.petertribble.co.uk/ - http://ptribble.blogspot.com/ > > -- > -Peter Tribble > http://www.petertribble.co.uk/ - http://ptribble.blogspot.com/ > illumos-discuss | Archives | Powered by Topicbox Without vision, the people perish. Thanks, David Halko http://www.netmgt.blogspot.com/ ------------------------------------------ illumos-discuss Archives: https://illumos.topicbox.com/groups/discuss/discussions/T784fc87098d66577-M4274499798bbfa8b945a13d0 Powered by Topicbox: https://topicbox.com
