Dear David,
I am seriously interested in collaborating with you!
I especially like the following points:
1) Programming by text manipulation is not the only way to do programming
I actually tend to have the more "iconoclastic" view that text-based
programming is "harmful" -- see my previous rant on FONC, but you mentioned
what should be done, whereas I only managed to point out what should not be
done.
2) I like the tacit idea. I always considered the omnipresent reliance on
names as means of binding things together as extremely fragile. Do you
think one could treat the names as annotations with documentation purpose,
without them being the binding mechanism?
3) Last but not least: There is no fundamental difference between
programmers and users. Both groups are just using computers to create some
digital content. Any sharp boundary between the way the two groups work is
maybe unnatural. I think psychology is an important factor here. I actually
do think that many programmers actually like the existence of such boundary
and are not motivated to make it disappear, but this is really just an
opinion.
On Fri, Sep 20, 2013 at 7:35 AM, David Barbour <dmbarb...@gmail.com> wrote:
> Over the last month, I feel like I stumbled into something very simple and
> profound: a new perspective on an old idea, with consequences deeper and
> more pervasive than I had imagined.
>
> The idea is simply this: every user action is an act of meta-programming.
>
> More precisely:
> (1) Each user event addends a tacit concatenative program.
> (2) The output of the tacit concatenative program is another program.
> (3) We can understand the former as rewriting parts of the latter.
> (4) These rewrites include the user-model - navigation, clipboard, etc.
>
> I will further explain this idea, why it is powerful, how it is different.
>
> To clarify, this isn't another hand-wavy 'shalt' and 'must' proposal with
> no idea of how to achieve it. Hammering at a huge list of requirements for
> eight years got me to RDP. At this point, I have concrete ideas on how to
> accomplish everything I'm about to describe.
>
> Users Are Programmers.
>
> The TUNES vision is revived, and better than ever.
>
> *WHY TACIT CONCATENATIVE?*
>
> Concatenative programming is perhaps best known through FORTH. Most
> concatenative languages have followed in Charles Moore's forthsteps,
> sticking with the basic stack concept but focusing on higher-order
> programming, types, and other features.
>
> A stack would be an extremely impoverished and cramped environment for a
> user; even many programmers would not tolerate it. Fortunately, we can move
> beyond the stack environment. And I insist that we do! Concatenative
> programming can also be based upon such structures as trees, Huet zippers,
> and graphs. This proposal is based primarily on tree-structured data and
> zippers, with just a little indirect graph modeling through shared state or
> explicit labels (details later).
>
> A 'tacit' programming language is one that does not mention names for
> parameters or local variables. Many concatenative programming languages are
> also tacit, though the concepts don't fully intersect.
>
> A weakness of tacit concatenative programming is that, in a traditional
> text-based programming environment, users must visualize the environment
> (stack or other structure) in their head, and that they must memorize a
> bunch of arcane 'stack shuffling' words. By comparison, variable names in
> text are easy to visualize and review.
>
> My answer: change programming environments!
>
> Powerful advantages of tacit concatenative programming include:
> 1. the environment has a precisely defined, visualizable value
> 2. short strings of tacit concatenative code are easy to generate
> 3. concatenative code is sequential, forming an implicit timeline
> 4. code also subject to learning, pattern recognition, and rewrites
> 5. every step, small and large, is precisely defined and typed
>
> Instead of an impoverished, text-based programming environment, we should
> offer continuous automatic visualization. Rather than asking users to
> memorize arcane words, we should offer direct manipulation: e.g. take, put,
> slide, toss, drag, drop, copy, paste. Appropriate tacit concatenative code
> is generated at every step, for every gesture. This code is easy to
> generate because generators can focus on short vectors of learned 'known
> useful' words without syntactic noise; this is subject to a variety of
> well-known solutions (logical searches, genetic programming,
> hill-climbing).
>
> And then there are benefits that move beyond anything offered today, UI or
> PL.
>
> Not only can we visualize the environment, we can animate it. Users can
> review and replay their actions, potentially from different perspectives or
> highlighting different objects. Since even the smallest dataflow steps are
> well defined, users can review at different temporal scales, based on the
> granularity of their actions - zooming in to see precisely what taking an
> object entails, or zooming out to see broad changes in an environment.
>
> Rewrites can be used to make these animations smoother, more efficient,
> and perhaps more aesthetically pleasing. And, in addition to undo, users
> can rewrite parts of their history to better understand a decision or to
> fix a mistake.
>
> The programming environment can also help users construct macros: pattern
> recognition is easy with tacit programming even if it were just in terms of
> sequences of words. However, patterns are augmented further by looking at
> context, the environment at the time a word was used. Proposed words can be
> refined with very simple decision procedures to account for slight
> context-sensitive variations. Discovered patterns can be used for simple
> compression of history, or be used for programming-by-example.
>
> An environment that recognizes a pattern might quietly and unobtrusively
> offer a constructed tool or macro, that the user might refine a little
> (e.g. clarifying the decision procedure) before using. The notion of
> 'dialecting' and 'DSLs' is replaced by problem-specific toolboxes and
> macros, where a tool may act a lot like a paintbrush.
>
> Further, there are advantages from the formalism and typing!
>
> For one example, it is to guide user actions relevant to the typeful
> context - i.e. making appropriate suggestions. Also, multiple actions can
> be assigned to a single gesture or voice command, so long as they are
> distinguishable in most typeful contexts. (When there seems to be
> ambiguity, the environment can ask for clarification. Not a problem so long
> as it's rare.)
>
> By introspecting the environment, we can also create words that are
> 'smart' about their application, i.e. automatically performing a search of
> the local environment to find an appropriate target, and perhaps validate
> that it is a unique target. This ability to be selectively imprecise can
> greatly reduce the burden on users and developers. (Usefully, we can
> separate the 'search' and 'apply' patterns such that augmenting any action
> with search is a simple composition.)
>
> Tacit concatenative programming is *safer* than names.
>
> With parameter based programming, the data-plumbing is untyped and ad-hoc.
> Further, captured names are almost never visible in the 'type' of a
> function or closure. This can lead to unsafe or inefficient behaviors,
> where names are captured in a closure that is then communicated, or shared
> by multiple threads. Essentially, the problem is that names are *too*
> expressive. We can use references in ways their referents cannot be used.
> We can put the "gorilla" in the mailbox, but not the gorilla.
>
> This safety issue is especially relevant for RDP. I make heavy use of both
> location types ('where' is the value) and substructural types (functions
> that cannot be dropped, or cannot be copied, or both). Tacit concatenative
> makes safety-by-construction much easier.
>
> Tacit concatenative programming CAN model use of names, i.e. in terms of
> lookup in an association list. My proposal will use this technique on
> occasion. But there is a very strong, visibly obvious distinction between
> the reference and the referent - i.e. the reference is a text value, while
> the referent is a gorilla!
>
> *THE USER MODEL*
> *
> *
> The tacit concatenative program can be understood as an unbounded stream
> of pure `state -> state` rewrite operations. In addition to these
> operations, users have freedom to undo, review, replay, and even rewrite
> their recent history of actions. Undo can be accomplished by the normal
> snapshot-replay mechanisms.
>
> But we don't model the user as awkwardly 'above' the state, apart from it.
> Instead, we model the user within the state. Literally.
>
> (world * user) -> (world * user)
>
> One might think of the 'user' here as the hero of a video-game, and the
> 'world' as a complex environment that can be navigated or manipulated. The
> hero will have hands to carry things, an inventory of loot and weapons,
> perhaps a list of special skills. The hero is so important and central to
> our model, that navigation is actually modeled by rolling the world under
> the hero.
>
> Of course, a user environment isn't a video game. (Or at least it
> shouldn't be used that way at all times!) But the same ideas hold.
>
> We may have 'take' and 'put' actions to move objects from the world to the
> user. Navigation is often modeled using zipper-like operations through a
> document structure, or occasionally by something closer to a hyperlink
> (searching for an object by index). Instead of special skills, we have
> macros and a powerbox. Instead of loot and weapons, we have projects and
> domain-specific toolkits (e.g. paintbrushes, geometry manipulators).
>
> In addition to 'hands', a rather interesting possibility is to have 'eyes'
> - programmable lenses that affect how we view, influence, and navigate the
> world. Through lenses we might introduce overlays, highlight important
> objects, gain x-ray vision for geometries, introduce a head's-up display,
> or collapse irrelevant structures.
>
> (NOTE: This hand-and-eye concept - where the hand is programmable by
> composable tools, and the eye is programmable by composable lenses, and
> this programmability is readily accessible to users - is one I've had in
> mind since about 2003.)
>
> *SHARE VALUES NOT ENVIRONMENTS*
>
> A user's environment is extremely personal and personalizable.
>
> Between pattern recognition, code generation, and programming by example,
> I imagine that the user and environment will tend to 'grow up' together,
> developing a private language specific to each human. In addition, the
> environment will acquire a great deal of private information about a user -
> e.g. relationships, financial information, pictures and messages.
>
> So users won't want to share their personal environments at that
> granularity. And this is fortunate, because they can't. In general, there
> is no safe or sensible way to compose independent command streams from
> multiple users.
>
> But users can share:
>
> * values - numbers, text, and composites that may represent geometries,
> diagrams, documents, graphs, tables, recorded images, sounds, measurements
> * behavior-specifying values - e.g. representing macros, lenses, tools,
> and authorities
> * reactive values - normal or behavioral, time-varying with hidden
> dependencies
>
> In RDP systems, sharing between agents occurs via an intermediate
> resource. Agents include other humans, but also sensors, databases, and
> actuators. The support for reactive time-varying values is a feature
> provided by RDP, and involves remaining attached to the value source to
> track updates.
>
> To share a value, we publish into some space shared with friends or
> customers, or a more global space (like a wiki). Private spaces can be
> established by a variety of protocols with trusted intermediaries, though
> they often must be bootstrapped in physical space.
>
> Not every user thinks about programming, or makes an effort to create
> something reusable. But I think most people will fiddle, find interesting
> ways to arrange lenses, rearrange documents, smash values together to
> create new value. Mashups will be the norm. And even people who aren't
> making any effort might be provided useful tools
>
> Everyone is a programmer some of the time.
>
>
> *ENVIRONMENT METAPHORS FOR USERS?*
>
> I haven't started on the details for a user environment metaphor.
>
> The environments I've developed so far are still aimed at programmers in a
> text-based environment. I would probably be focused on a single stack if
> RDP didn't have declarative concurrency properties. (A single stack is
> painful for modeling concurrent tasks or workflows that must join or synch
> at some steps.)
>
> But, based on my interests, I would focus on the following features:
>
> * zoomable user interfaces with live documents
> * diagrams, geometries, images, graphs, scene-graphs
> * animated non-reactive values (video, GIFs, sound, etc.)
> * widgets, variations suitable for use in RDP
> * augmented reality systems (visual fingerprints, etc.)
>
> What I can say is this: expressiveness will not be the issue here. We
> could model hypermedia systems, desktop metaphor, or whatever else we
> decide.
>
> The main difference from today's design would be that these are now
> constructed of fine-grained values, subject to introspection and
> reorganization and mashup, accessible for macro programming, and coexisting
> in a common language-based security model.
>
> *ENVIRONMENT IS ALSO A LIVE PROGRAM*
>
> Macros, tools, and so on are designed for volatile manipulation of state.
> But that manipulation of state should be meaningful! And to provide meaning
> to state, we must use an interpreter. But this interpretation should be
> live: as we continue to maintain the state, the meaning should be
> propagated automatically.
>
> Here are a few principles that are guiding my thoughts on this subject.
>
> (1) Users must be able to assign their own, private meanings to state in
> their personal environments. Each graph, diagram, document, geometry, and
> so on can have a different meaning. Some of those meanings will be realized
> by programmatic interpretation.
>
> (2) ALL long-running behaviors and policies should have corresponding
> state in the environment. Every relationship, shared value, observation on
> reactive state, and so on should be accessible in this manner. This is
> essential for visibility, maintenance, and for revocation.
>
> (3) Failure is ideally very coarse-grained. Dealing with partial-success
> is painful, complicated, and error-prone; we would greatly benefit from
> precise atomic success/fail boundaries.
>
> It's addressing these principles where RDP really shines. RDP is based
> upon continuous influence and observation, and also has very nice
> properties for runtime update and revocation. For clean failure, RDP
> enables time-warp style 'undo' even in an open system. Of course, there are
> practical latency limits on this (can't always correct the past), but those
> are partially addressed: RDP also enables speculative evaluation, so we can
> tentatively feel out 'what would happen if'.
>
> So, how do we model this separation?
>
> My current thought is that, since meaning is private to the user, the
> association between meanings and objects in the environment should be
> maintained as part of the user-model. I'm currently envisioning a very KISS
> model: there is an association list at a standard location in the
> user-model of a form similar to:
>
> ("@foo" * [block interpreting foo])
>
> Then, in the environment, users will have ("@foo" * fooStructure) objects
> scattered around with no particular organization. If the whole foo object
> is inside some larger structure, like "@bar", then it would be the
> prerogative of the bar interpreter to either ask for a foo interpreter or
> provide its own interpretation.
>
> In order to enforce the "ALL long running behaviors are modeled in visible
> state" principle, the initial program has no authority; it's ultimately
> just a sequence of pure state->state transforms. Capabilities are
> introduced only the second phase. The real argument to the block
> interpreting foo is a pair: (powerblock * fooStructure).
>
> (I'm not entirely satisfied here. In particular, I'd like to have more
> precise understanding of source-stable uniqueness for the powerblock.)
>
> Potentially, this entire process might staged, e.g. if the *output* of
> interpreting foo contains an ("@baz" * bazStructure)
>
> In this design, text-based programming can still be supported, but
> certainly isn't necessary.
>
>
>
>
> *SERIALIZATION: ONE CODE TO BIND THEM*
> *
> *
> I propose that all values be shared by a pure, tacit concatenative
> bytecode. There are several reasons for this.
>
> (1) a uniform serialization model will avoid a lot of redundant parser
> code and discontinuity spikes. And in practice, a tacit concatenative
> bytecode is likely to operate more efficiently than most parsers: it
> reduces to a simple series of table lookups (or even a switch expression)
> and a small state machine to deal with text and blocks.
>
> (2) in a reactive model like RDP, we often have large structural values
> (like an array or scene-graph) where only a few values change. Rather than
> sending the whole structure to communicate a small change, this is easily
> modeled in terms of streaming more bytecode to operate on the original
> value.
>
> (3) we can gain a lot of efficiency by a very simple trick: instead of
> just a value, we can operate on a `(value*context)` pair. The context is a
> 'communication context' that can hold a small library of functions, some
> memoized computations, and so on. Functions in the context can be compiled
> by the recipient.
>
> (4) code can contain useful assertions, self-validation.
>
> (5) a high level of semantic compression can be achieved without any
> additional designs or layers. Though, if semantic compression isn't used,
> then regular streaming compression should work pretty well.
>
> I am developing Awelon Bytecode (ABC) for this purpose.
>
> ABC is a typed, tacit, concatenative bytecode for an idealized RDP system.
> ABC has very little structure; it is a UTF-8 stream with very few parse
> modes:
>
> slsls - (START) normal bytecode mode
> {text goes here} - text mode
> \}, \\, etc. - escapes in text require a mode
> [slsls] - block mode, forming a function
>
> In this case `slsls` means `swap assocl swap assocl swap` - a fairly
> common operation that I usually give the name `assocr`.
>
> ABC has a minimal set of primitives and very few types. It's up to a
> decent compiler or interpreter to simplify data-plumbing like slsls. The
> goal with ABC is not a most efficient direct-interpretation. I am more
> interested in keeping things minimal, easy to prove, easy to generate, and
> easy to optimize.
>
> ABC has only one syntax-layer value type, which is text. Numbers
> (rationals) are specified in ABC by first using text then translating it to
> a number. This isn't the ideal representation for efficiency, but I feel
> that legibility and simplicity has greater value.
>
> {text} :: x -> (text*x)
> # :: (text*x)->num*x
>
> {Hello, World!}
> {42}#
>
> Structured values can be formed by constructing elements and organizing
> them in a streaming fashion. There are some simple strategies to achieve
> this.
>
> (42,108) => {108}#se{42}#
>
> Text can also be used as a comment. I can think of a few reasons this
> might be done - e.g. to provide optimizer suggestions, record profiling
> information, or potential hints for a theorem prover.
>
> % :: (text*x) -> x
> {this is a comment}%
>
> ABC is designed for capability-based languages. I.e. there is no ambient
> authority (except for 'error'). Developers can't even create 'unique'
> objects (or local state) without a capability.
>
> $ :: (text*x)->cap*x
>
> The interpretation of the text within a serialized capability is entirely
> up to the provider of the capability. It could be encrypted code. It could
> be HMAC authenticated code. It could be a random GUID to a stored value.
> And so on.
>
> In RDP systems, all capabilities are implicitly revocable: to 'grant' a
> capability is a continuous action, so to revoke you simply stop granting.
> No state is required, and this can be implemented by a variety of
> strategies.
>
> ABC doesn't track any pure/impure type. However, RDP has a concept of
> location, called 'partition type', which can be used to isolate some
> subprograms.
>
> ABC can have spaces, tabs, newlines, and carriage returns. Those all have
> the same meaning: identity function.
>
>
> *WHAT FAILED BEFORE?*
>
> Similar efforts have come and gone, often with some small success that
> could not be scaled. My hypothesis is that the following have been points
> of failure:
>
> 1. Did not model user/programmer. User sits awkwardly above model, no
> semantic-layer ability to manipulate it or program-by-example. Wall of
> syntax.
>
> 2. Second-class extensions. Brushes, tools, lenses, views are not
> first-class objects that can be carried and composed. Boiler-plate
> namespace management to reuse tools from one task to another.
>
> 3. Did not effectively address value sharing, independent maintenance,
> security properties. Programmers forced to "ship the IDE" to share behavior.
>
> I believe that all three points must be addressed simultaneously to have
> any hope for success. If we address 1,2 we have isolated users - a powerful
> environment but no leverage. If we address 3, we have more effective
> programmers, but it's all arcane knowledge and hidden APIs.
>
> In my design, points 1,2 are addressed by the tacit concatenative model of
> programmer manipulating environment. Point 3 is handled by RDP and
> capability security.
>
>
> _______________________________________________
> fonc mailing list
> fonc@vpri.org
> http://vpri.org/mailman/listinfo/fonc
>
>
_______________________________________________
fonc mailing list
fonc@vpri.org
http://vpri.org/mailman/listinfo/fonc
