On Sat, Sep 21, 2013 at 6:01 AM, Matt McLelland <mclelland.m...@gmail.com>wrote:
> The artifact being constructed in a naive image editor is an image and > it's not particularly fruitful to view that image as a program. > It's very fruitful! An image could be interpreted as a high level world-map to support procedural generation with colors indicating terrain types and heights. Or the image could be interpreted as a template for a cellular automaton. And, of course, the image could be a simple icon or texture. Granted, I can't think of many uses for *raster* images. But if we extend this to the broader concept of image artifacts - diagrams, graphs, and so on there are many effective ways to interpret them as programs. There's a whole bunch of low-hanging fruit here. Don't let go to waste. As you're saying, you can view the operations of building an image as > defining a programming language and more advanced image editors show you a > pipeline of such operations and let you edit it in ways that are starting > to feel like programming. But when you look at things that way, the image > is not the program -- the pipeline is the program. > The pipeline is ALSO a program, and a much more implicit one. Every user action is an act of *meta*-programming. Every artifact has behind it history of actions associated with its construction. If relevant parts of this history can be extracted, perhaps tuned a little for reuse, then developers can build many useful tools without ever explicitly thinking about it as programming. And the best way to look at and understand that pipeline is probably > structured text. Think of how the pipeline is usually presented: a stack of > text descriptions of each operation (probably combined with a thumbnail of > the image after that stage). > I think structured text - in particular, a tacit concatenative program - is a great way to *formally* understand the pipeline. Having this structure is essential. It enables formal approaches to extracting and delimiting the history associated with an artifact. Irrelevant actions (anything not contributing to the artifact's value or type) can be eliminated. I would want the formal descriptions to also be there, kind of omnipresent in the background (like formula in a spreadsheet), if only so that regular users learn a little by osmosis and curious fiddling. As I said, programming should be basic computer literacy. But I think presenting the pipeline would often be done in other terms of informal text (e.g. output of an "explain" method), in addition to the icons. (cf. Bret Victor's Drawing Dynamic Visualizations - http://vimeo.com/66085662). > > We can consider an arbitrary UI to be building a program, but you aren't > really programming unless you're interested in the whole program at once. > Users should be enabled to: * easily think in terms of capabilities not apps * extract and tune action into reusable tools * organize, maintain, and share sets of tools * easily compose tools when they wish * understand artifacts as long-running behavior * manipulate artifacts as acts of live programming * treat artifact manipulation as UI widgets There is no "the whole program", just myriad composable, sharable subprograms. What you today call "real programming" should be looked at in hindsight as caveman programming - i.e. those dark ages when programmers huddled into dank caverns and cubicles with a keyboard and monitor and those ancient towers rather than the ubiquitous, augmented reality. Much like we regard assembly language programming today. Related: http://xkcd.com/378/ The environment should help users a great deal: * pattern recognition, programming-by-example * automatic example-input generation for testing and tuning * automatic search, genetic programming * Markov models to unobtrusively predict user action Visualization is not the only area where tacit concatenative helps a great deal. while this is really just terminology we're hammering out, I would prefer > to say that 95% of UI users aren't programming. Programming is when you > care about the whole program. > To understand users as programmers - the 95% case - isn't "just terminology". It's a new way of thinking. A new perspective can be extremely valuable. As a concrete example, a recent physics article<https://www.simonsfoundation.org/quanta/20130917-a-jewel-at-the-heart-of-quantum-physics/> describes quantum interactions as a higher-dimensional timeless geometry. By doing so, it computes on a napkin interactions that took hundreds of pages with Feynman diagrams. To perceive users as programmers seems, to me, just as valuable for simplifying PL as that quantum jewel is for physics. I ask that you reconsider your current views, and whether what you call 'real programming' has real intrinsic value. > So now consider the special case where the UI we're interacting with is an > IDE > Why should this be a special case? The user environment I described is always an IDE. With a much broader 'I'. I see no special relationship between that primary program and the other > program > we're implicitly building when we look at the IDE's UI as a programming > language. > Some useful relationships: (1) The value models are the same in both cases. (2) Subprograms extracted from the latter are first-class values in the former. (3) Subprograms developed in the former can be used as tools in the latter. (4) The set of user capabilities is exactly equal to the set of program capabilities. (5) Intuitions developed based on usage are directly effective for automation. These shouldn't be "special" relationships. > > My point is that if you start with a named compositional (values can only > be composed) language, it looks like it would be easy to convert to tactic > concatenative. It's easy to replace uses of names with nonsense words that > achieve the same data plumbing. > That's only true if you *assume* every dataflow you can express with the names can also be expressed with concatenative 'nonsense' words. My point is that concatenative allows for degrees of precise, typeful control over dataflow that names do not. I can create tacit concatenative models for which your named applicative (aka monadic) is generally too expressive for translation, yet which are significantly more expressive than pure applicative. Further, it's quite useful to do so for correctness-by-construction involving substructural and modal types (affine, relevant, linear, regional, staging) which are in turn useful for security, resource control, and modeling heterogeneous and distributed systems. Concatenative with first-class names has the exact same problems. That's why I argued even against John Purdy's use of local names. Granted, his language doesn't have the greater expressiveness of substructural and modal types. But now he couldn't add them even if he wanted to. My own modeling of names using explicit association lists only avoids this issue due to its second-class nature, a formal indirection between reference and referent (such that capturing a reference does not imply capturing the referent). Anyhow, the issue here is certainly named vs. tacit, not applicative vs. concatenative. > > > > On Fri, Sep 20, 2013 at 8:28 PM, David Barbour <dmbarb...@gmail.com>wrote: > >> >> On Fri, Sep 20, 2013 at 2:57 PM, Matt McLelland <mclelland.m...@gmail.com >> > wrote: >> >>> >>> I would say that in my experience text is a much better construct form >>> for most programs than those other forms, so I would expect text to be the >>> 95% case. I'm including in that more structured forms of text like >>> tables. >>> >> >> If you look at PL, text has been more effective. Graphical >> programming is historically very first-order and ineffective at addressing >> a variety of problems. >> >> If you look at UI, text input for control has been much less effective, >> and even text output is often augmented by icons or images. The common case >> is buttons, sliders, pointing, and so on. Some applications also use >> concepts of tooled pointers like brushes, or tooled views like layering. >> >> I've tried to explain this before: I see UI as a form of PL, and vice >> versa. Thus, to me, the 95% case is certainly not text. Rather, most users >> today are using a really bad PL (<- the 95% case), and most programmers >> today are using a really unnatural UI (<- and thus need to be really >> serious about it), and this gap is not essential. >> >> >>> >>> What I'm still not understanding is how viewing the editing of an image >>> or graph as a tacit concatenative program is a big win. >>> >> >> Have you ever used a professional image editing tool? >> >> If you haven't, the process actually does involve quite a bit of >> automation. The artist implicitly constructs a small pipeline of layers and >> filters based on the actions they perform. This pipeline can often then be >> separated from the current image and applied to another. Essentially, you >> have implicit macros and a limited form of programming-by-example. >> >> But, with the way applications are designed today, this pipeline is >> trapped within the image editing application. You cannot, for example, >> casually apply a pipeline of filters to a view of a website. Conversely, >> you cannot casually incorporate an image search into layers or elements in >> an image. Either of these efforts would require a lot of file manipulation >> by hand, but that effort would not be reusable. >> >> If you are an artist with a hobby of hacking code, you could: >> * don your programmer hat >> * fire up your favorite IDE and textual PL >> * go through the steps of starting a new project >> * review the APIs for HTTP loading >> * review the APIs for Filesystem operations >> * review the APIs for your image-editing app >> * oh bleep! doesn't have one! >> * review the APIs for image processing >> * export your pipeline >> * begin implementing a parser/interpreter for them >> * who the bleep designed this pipeline language?! >> * abort your effort to implement a general interpreter/parser >> * re-implement your pipeline in your language >> * ugh... who designed these validation tools? >> * 'Image'? type system ain't worth a bleep here... >> * unit tests on images? how does that even work? >> * it's a wall of text! I can't see what's going on!! >> * review your API for image display >> * integrate image display into your tests >> * edit test edit test edit test edit test >> * woohoo! It works! >> * but the test-case is hard-coded in :( >> * contemplate building a configuration language >> * bleep that! I'm a hacker. >> * just edit and hard-code in the next use case... >> * and the next, and the next >> >> (years of bitrot later) >> * why don't these APIs work anymore?! >> * (I really don't want to review them again) >> * where are those configuration variables scattered? >> >> And that's only if you happen to be in the small intersection of artists >> who have a hobby of hacking. If you're a 'serious' programmer you might >> have gone the extra steps to import the pipeline and build a configuration >> file. But the overall experience wouldn't be that much different. >> >> If you're anyone else, you might contemplate hiring a programmer. But you >> think: that's expensive, and I don't off hand know any programmer with >> skills in image-processing who is also looking for work, and I don't want >> to pay a programmer to self-educate. So you end up addressing the specific >> problem by hand. Or you just abandon the effort. >> >> That experience shouldn't need to happen. >> >> But because UIs are bad PLs today, it is very difficult to integrate the >> capabilities of different services, toolkits, and APIs. Conversely, because >> PLs are bad UIs today, we build these thick layers we call 'applications' >> between users and the underlying capabilities. Of course, most programmers >> don't think about UI as PL. And even if they do, most are bad PL designers. >> And even those who aren't, don't have access to or examples of UI toolkits >> designed to support UI as an effective PL. It's a vicious cycle that has >> only been broken by a few small niche communities (like REBOL). >> >> >> >>> I'm skeptical that non-programmers will ever do serious programming. >>> >> >> I don't expect non-programmers to do "serious programming". I expect to >> lower barriers so that "serious programming" is very rarely needed, and >> such that when it is needed it can be handled as a tiny extension to an >> app, or a small composition, rather than a full new app. The goal: Serious >> programming is only needed 5% as often. And, when needed, costs only 5% as >> much. >> >> Programming should not be a career. >> >> Programming should be the most basic form of computer literacy - such >> that people don't even think about it as "programming". >> >> A scientist who knows how to get big-data into one application and >> process it in another should be able to build a direct pipeline - one that >> optimizes away the intermediate loading - without ever peeking under the >> hood, without learning an API. >> >> A musician who knows how to watch YouTube videos, and who has learned of >> a cool new machine-learning tool to extract and characterize microsounds, >> should be able to apply the latter to the sounds from the former without >> learning about HTTP and video transfer and how to scrape sounds from a >> video. Further, it's better for both the artist and servers if this >> processing can automatically be shifted close to the resources and >> eliminates the irrelevant video rendering. >> >> Artists, scientists, musicians, anyone should be able to think in terms >> of capabilities: >> >> * I can get an X >> * I can get a Y with an X >> * therefore, I can get a Y >> >> But today, because UIs are bad PLs, they cannot. Instead, we have this >> modal illogic: >> >> * [app1] I can get X >> * [app2] I can get Y with X >> * ??? >> * profit >> >> UI (and programming) is much more difficult today than it should be, or >> can be. >> >> >>> isn't the hard difference more the concatenative vs. applicative than >>> named vs. tacit? >>> >> >> (context: challenge of translating traditional PL to tacit concatenative) >> >> No. The primary challenge is due to named vs. tacit, and the dataflows >> implicitly expressed by use of names. If you have an applicative language >> that doesn't use names, then there is a much more limited dataflow. It is >> really, literally, just Applicative. >> >> class Functor pl where >> -- language supports pure functions >> fmap :: (a -> b) -> pl a -> pl b >> >> class Applicative pl where >> -- language supports pointy values >> pure :: a -> pl a >> >> -- language supports procedural sequencing >> ap :: pl (a -> b) -> pl a -> pl b >> >> -- (some thought has been given to separating pure and ap). >> >> This much more constrained language is easy to express in a concatenative >> language. >> >> * `fmap` is implicit (you can express pure behaviors if you like) >> * `pure` is modeled by literals (e.g. `42` puts a number on the stack) >> * `ap` is a simple combinator. >> >> But introducing names moves expressiveness from `Applicative` to `Monad`, >> which supports ad-hoc deep bindings: >> >> foo >>= \ x -> bar (\y -> x+y) >> >> Concatenative languages are often somewhere in between Applicative and >> Monad, since they require explicitly modeling the data-plumbing and hence >> can control the expressiveness of bindings. >> >
_______________________________________________ fonc mailing list fonc@vpri.org http://vpri.org/mailman/listinfo/fonc
