Author: gbg
Date: Sat Jun 21 12:56:46 2014
New Revision: 1604346
URL: http://svn.apache.org/r1604346
Log:
Add trunk directory and proposal file.
* flexicon
(trunk): New directory.
* flexicon/trunk
(Proposal.txt): Initial version of the Flexicon Proposal.
Added:
labs/flexicon/trunk/
labs/flexicon/trunk/Proposal.txt
Added: labs/flexicon/trunk/Proposal.txt
URL:
http://svn.apache.org/viewvc/labs/flexicon/trunk/Proposal.txt?rev=1604346&view=auto
==============================================================================
--- labs/flexicon/trunk/Proposal.txt (added)
+++ labs/flexicon/trunk/Proposal.txt Sat Jun 21 12:56:46 2014
@@ -0,0 +1,726 @@
+Flexicon
+==========
+
+A tool for individuals, business and crowds to organise and maintain a
+personal or community-owned compendium of compound commands and
+scripts, which can be standardised, shared, tracked and searched due
+to having a general compatible format and inbuilt tests.
+
+It is not tied to any particular operating system or software, it is a
+general tool to keep our working methods organised and to enable us to
+share and preserve them.
+
+=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=
+
+Index
+-----
+
+A. Introduction
+ Who uses Flexicons and why?
+ How Tricks are used by the community
+B. Design
+ Rationale
+ Next steps
+ Caveat
+ Future Plans
+ Project Motto
+C. Examples
+ Use Case
+ Sample Trick File: SvnShowRevisionNumber.tr
+ Finding the right trick
+ Checking if you can use a Flexicon
+D. Components
+ List
+ Trick Test Suite
+ Environment/prerequisite checker
+ Functionality checker
+E. Library program definition
+ Basic Concept
+ Commands
+ User Interface
+ Bookkeeping
+ Preparation
+ Shelf Management
+ Testing books and tricks
+ Producing books
+F. Data Fields
+ Tricks
+ Flexicon/Library
+
+=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=
+
+A. Introduction
+===============
+
+Flexicon is tool to organise collections of compound commands
+(called tricks for short) that use one or more software applications
+to complete complex tasks.
+
+Each individual or community collects, writes and maintains their own
+Flexicons and individual tricks, and individual Flexicons are a
+named collection of offical tricks which are loaded by the main
+program, called 'Library' along with and personal tricks that the user
+can select on a case-by case basis.
+
+Tricks and Flexicons are freely interchangable between people, and
+ready to use (provided their integrated test passes). Flexicons and
+tricks can be integrity checked, and failing tricks can disabled until
+they have been repaired and are added back.
+
+
+Who uses Flexicons and why?
+-----------------------------
+
+An individual may use Flexicon to keep a collection of their
+personal tricks, and tricks they obtain from other Phrasebooks or as
+individual items.
+
+A business can use Flexicon to organise tricks pertaining to
+particular IT tasks or positions, and to provide organisational
+memory, and a dictionary of processes used.
+
+A community can use Flexicon to pool effort and quality check tricks
+for use with their project or the product their community uses or
+produces. It is a communication tool & vault and specifically aims to
+provide a structure that prevents loss of information and obsolence
+via 'software rot' or lossage through time-induced obscurity, and
+enables a community to keep track of, and organise crowdsourced
+tricks.
+
+A project may use Flexicon to enable users to build a richer user
+interface than the project itself can offer. One such example is
+Apache Subversion which has a plethora of options that can be combined
+to do complex tasks, some of which are not immediately obvious even to
+expert users.
+
+This allows the dev team to add more bells and whistles to their
+project without the overhead of attempting to design how people will
+use their software easily, and allows users to standardise idioms they
+use and swap, or to create personal collections. A user community can
+produce a fully tested, validated Flexicon for every version of the
+projects software they use.
+
+Who better to design an user interface than the user community
+themselves?
+
+
+How Tricks are used by the community
+------------------------------------
+
+Tricks can be shared via the internet, and any trick from anywhere can
+be added to any Flexicon collection according to user-set
+conditions.
+
+Tricks may be publicly shared or kept private, they may be encrypted
+or even carry licenses. It's no different to what you find in the
+wild on the Internet, but with the added bonus of a set format and an
+attempt to ensure functional integrity.
+
+
+B. Design
+=========
+
+Rationale
+---------
+
+Flexicon provides a formal method of organising tricks into
+Flexicons which themselves can be collected into the library program
+that runs on the users' platform.
+
+It's a very general tool and not dependent on any particular usage,
+operating system or software. You can create, distribute and collect
+tricks for your windows system, your Linux/FreeBSD/Macintosh or
+whatever other system you're working on, provided the Flexicon
+software can run on your system.
+
+An individual trick may be a complex script which combines many
+different softwares, or it may be a simple one liner that just uses
+GNU tools for a particular task.
+
+Individual users can pick and mix tricks they like, and communities
+can choose to apply a voting process or another decision process as
+a quality control tool for their official compendiums.
+
+Most tricks come with a test suite that can be run individually or as
+part of an automated batch check to ensure that it works correctly in
+it's current setting. This ensures that the trick remains current
+despite a software upgrade or a project release, and that tricks can
+be selected correctly for particual software versions. Users can of
+course omit this for their personal tricks, or choose to download
+tricks without tests, at their own risk, which is not much different
+to the current situation of finding something useful on the internet.
+
+This may seem a bit complex, but it allows us to share our tricks and
+it is hoped that this gadget will save the Open Source community huge
+amounts of time since we no longer have to each knit our own tricks or
+rely on (and laboriously test) 'cooking recipes' we happen to find
+somewhere that may or may not work.
+
+Also, it will help to standardise what tricks we use. If one of us
+writes a trick to automagically (say) merge an svn trunk to an svn
+branch, everyone can partake if the creator shares this trick -- write
+once, use everywhere (relatively safely...!) is the motto!
+
+
+Next Steps
+----------
+
+I've posted a request to Apache Labs to attract contributers, hoping
+that this project will be promoted to Apache Incubator.
+
+Decisions need to be made what language and database to use, and the
+design idea needs a robust peer review. Many edges of the design are
+ragged, and there are a few parts where the description amounts to
+'and here a miracle happens', in particular the current TrickServer
+concept.
+
+The actual size of the project is rather small --- so far the library,
+which is the actual heart of Flexicon has 27 user commands, most of
+which are classic queries that will be fun and easy to code.
+
+The programming language and other software to use needs to be decided
+upon. The requirements are: Small memory footprint and quick
+execution since were are writing something that actually is a meta
+shell, and as a second tier condition, ubiquity and as little user
+effort needed as is possbible for installation.
+
+Caveat
+------
+
+Thus spake Cassandra: "One user is difficult, two users gets tough,
+an unlimited amount of users with unlimited freedom to do whatever
+they want to do (some of which may be malicious!) is quite the tall
+order."
+
+
+Future Plans
+------------
+
+Once Flexicon is operational and past the initial test phrase, I
+would like to introduce the concept to the Subversion user community,
+in the hope that they indeed adopt Flexicon as a tool to build a
+user designed user interface to Apache Subversion and agree to stress
+test the concept, with a view to set up a voting process to allow the
+existing IRC svn-user community to collaborate on an official
+Apache Subversion Phrasebook.
+
+
+Project Motto
+-------------
+
+Why have one or the other, when you can have one, the other, and both?
+
+
+
+C. Examples
+===========
+
+Use Case
+--------
+
+One typical application (and the orginal motivation) would be for the
+svn-user IRC community to collect and quality check compound commands
+that use svn in conjunction with with other software to produce a
+wrapper application for svn and other applications used in people's
+VCS processes.
+
+The other day on svn-dev mailing list, someone requested an svn
+feature to show the current revision number.
+
+Subversion didn't need a new feature, and whilst the solution _looks_
+simple, it wasn't all that obvious either:
+
+svn info | sed -ne 's/Last Changed Revision: //p'
+
+We helped one user to a solution, but packaging the above trick into
+
+--svn-show-revision-number
+
+would help all users, and also a list of such tricks would inspire
+users to make full use of svn's capabilities they so far have been
+unaware of. The problem is a) how to share this efficiently b) how to
+keep those shared tricks up-to-date!
+
+If we can package the above as a trick together with a test, and
+present it in an accessible form, then all our users will have access
+to this knowledge and the trick can be automatically checked to see if
+it's still functional for every SVN release, and users can build up a
+compendium that educates, and works for, the entire community,
+(reasonably) realiably so.
+
+Ideally, the svn-user community on IRC will be able to use the Apache
+voting process and tools to produce and maintain proven Flexicons
+containing fully tested tricks which pertain to particular svn
+releases.
+
+If a trick fails it's tests, it's automatically removed from the
+official SVN Flexicon and placed back into the pending release
+Flexicon until it's fixed and voted back in again by the community
+members of svn-user.
+
+
+Sample Trick File: SvnShowRevisionNumber_1.tr
+----------------------------------------------
+
+\ID{1}
+\TrickName{SvnShowRevisionNumber}
+\Organisation{svn-user}
+\Flexicon{SVN-1.6-official}
+\Author{Gabriela Gibson}
+\Status{public}
+\Precis{Display current revision number of Subversion}
+\Description{Query the Subversion revision number using 'svn info'}
+\LongSwitch{--svn-show-revision-number}
+\Version{1.0}
+\Software{svn, grep}
+\Shell{bash}
+\Command{c=`$SVN info | grep "Last Changed Rev:"`; echo ${c##*: }}
+\Keywords{svn, revision}
+\TestExpectation{!integer}
+\InitialCreationDate{10 Dec 2013}
+\Last updated{10 Dec 2013}
+
+
+Finding the right trick
+-----------------------
+
+If you perhaps noticed, the example trick file uses 'grep' and bash
+substitutions. What if you actually would prefer to use sh and sed?
+
+Tricks that have different execution methods but the same TrickName
+can be differentiated by searching their properties -- you look for a
+trick with the same name, that uses specific software:
+
+$ library find-software [location] --svn-show-revision-number sh,sed
+
+You can search with any field, or any combination thereof as the key.
+
+Example:
+
+Library also performs searches that help the user to produce a shelf
+with tricks/books that work, by just knowing their software
+requirements. Tricks may have many duplicates, for example:
+
+SvnShowRevisionNumber_1.tr
+(...)
+\Software{svn, grep}
+\Shell{bash}
+\Command{c=`$SVN info | grep "Last Changed Rev:"`; echo ${c##*: }}
+
+SvnShowRevisionNumber_2.tr
+(...)
+\Software{svn, sed}
+\Shell{sh}
+\Command{svn info | sed -ne 's/Last Changed Revision: //p'}
+
+Library will find and select the correct trick by looking at your
+software list, or check that you have the needed tools before it puts
+a trick on the shelf. So the ancient HP system that just has sh will
+have the same interface as someone's Macintosh, Windows or BSD system.
+
+
+Checking if you can use a Flexicon
+------------------------------------
+
+If you find a Flexicon that looks like it's perfect for you, you can
+run a check to see if you have all the required software on your
+system that the tricks in there need to use.
+
+$ library check-software FlexiconName
+
+will produce either a list of everything that is missing, or, inform
+you that it's ready to be used.
+
+
+About TrickServer
+-----------------
+
+TrickServer is a small webhosted database that registers tricks and
+serves unique IDs to new trick authors.
+
+TrickServer can be searched by [TrickName] [keywords] [software] and
+will return ID's of matching tricks. If it cannot find any tricks
+that match, it will inform you that this trick is missing, or if it
+only finds partial matches, it will return all the ID's that match,
+which you then can google for and see if you can find it on some
+mailing list or someone's blog perhaps.
+
+Note: I'm not sure about this design, it looks like it could be
+vandalised. Maybe using an email list and utilising the unique SMPT
+id might just about work, but I think this is beyond my current
+expertise and I would like to invite interested readers to propose a
+better concept here.
+
+
+D. Components
+=============
+
+Trick -- a plain text file in LaTeX macro format.
+
+Flexicon -- a plain text file in a dedicated directory which
+ contains information about the Flexicon and a list of
+ tricks which reside in the same directory.
+
+Library -- command line program that organises them and creates new
+ Flexicons.
+
+TrickServer -- central server that assigns unique ID and collects
+ (some) information about tricks. It also facilitates a
+ voting process in order to purge obsolete tricks.
+
+
+Trick Test Suite:
+-----------------
+
+Tests are run before its added to a Library on a system, or as a
+requested check, for example, after software upgrades. Individuals may
+decide to ignore failing tests, but communities may have their own
+procedures to validate tricks.
+
+
+Environment/prerequisite checker
+--------------------------------
+
+Checks that the test matches the current environment -- if you need
+zsh and svn, this checks for the availability those softwares and
+ensures that the versions thereof are in the range as defined by the
+trick author(s). Similiar in concept to make's configure.
+
+
+Functionality checker
+---------------------
+
+Runs the trick's test (if it exists) that checks whether the trick or
+Flexicon has the correct output on your system.
+
+
+E. Library program definition
+=============================
+
+Basic concept:
+
+Library is the user interface that manages tricks and books and runs
+the commands.
+
+Library has general and optional named shelves on which it stores
+collection of tricks obtained from Flexicons and individual tricks.
+
+A shelf can be emptied, filled, combined and packaged into a new book,
+or dumped out to an alias file.
+
+Library will check the users selection to ensure that every trick on
+the shelf works, and mark broken tricks/books as 'compromised' until
+their tests no longer fails.
+
+
+Commands
+========
+
+There are currently 27 command designs for the library program:
+
+User Interface: library
+
+Bookkeeping: makeAliasFile, make-trick-form
+
+Preparation: check-trick, check-book, check-software, find-software,
+ gather-software, diff-software, select-trick, purge-trick
+
+Shelf management: remove-book, remove-shelf, remove-trick, add-book,
+ add-trick, use-book, use-trick, show-shelf,
+ trick-show-[datafields], book-show-[datafields],
+ move-trick, copy-trick
+
+Testing books and tricks: validate-trick, validate-shelf,
+ validate-book, validate-show-failures
+
+Producing books: write-book, gift-wrap
+
+
+User interface
+---------------
+
+$ library [TrickName|switch]
+
+ run this trick on system.
+
+If library cannot parse your input, the error message is 'Ook?'
+followed by an attempt to explain what went wrong.
+
+library itself can be part of a piped compound command under UNIX --
+tricks can contain tricks that contain tricks which contain
+tricks(...)
+
+
+Bookkeeping:
+------------
+
+List: makeAliasFile, make-trick-form
+
+$ makeAliasFile [shelf|{TrickName1, TrickName2...}] [format] [filename]
+
+ Dump requested library contents into an alias file(s):
+ TrickName=command for unix, not sure what windows needs.
+
+$ make-trick-form [TrickFileName, ID]
+
+ Create TrickFileName_ID.tr containing skeleton of datafields.
+
+
+Preparation:
+------------
+
+List: check-trick, check-book, check-software, find-software,
+gather-software, diff-software, select-trick, purge-trick
+
+$ check-trick [optional: --mandatory]
+
+ reports any missing entries in a trick file. If mandatory is set
+ only missing mandatory entries are shown.
+
+$ check-book [optional: --mandatory] [Flexicon]
+
+ As check-trick, but it check all the tricks of the Flexicon.
+
+$ check-software [TrickName|Flexicon|shelf|all]
+
+ Create list of software required and runs 'configure' to check if
+ the user's system has the required software, report back if 'foo'
+ needs to be installed, disable non-functional tricks, enable all
+ tricks that work.
+
+$ find-software [switch|TrickName|keywords] [software] [location]
+
+ Search location for tricks that conform to software requirements.
+ Do not show duplicates if one trick already matches the
+ requirements. Report tricks/books that are not yet on the shelf,
+ and have software requirements
+
+$ gather-software [switch|TrickName|keywords] [software] [location]
[FlexiconName]
+
+ Search location for tricks that conform to software requirements.
+
+$ diff-software [switch|TrickName] [location]
+
+ Show a numbered list along with the software requirement and command
+ differences of the given TrickName.
+
+$ select-trick [number|ID]
+
+ Sometimes two or more tricks do the same thing in a different way.
+ This command allows you to select the one your prefer.
+
+$ purge-trick [number|ID] [successor ID | alternative ID]
+
+ Schedules an obsolete trick for a deletion vote at the TrickServer.
+ Note that the ID is never reused, but marked obsolete. A duplicated
+ trick that uses a different command is not obsolete, nor is a broken
+ trick that could be repaired. However an identical trick that has
+ two IDs is indeed obsolete -- there can only be one!
+
+
+Shelf management:
+-----------------
+
+List: remove-book, remove-shelf, remove-trick, add-book, add-trick,
+use-book, use-trick, show-shelf, trick-show-[datafields],
+book-show-[datafields], move-trick, copy-trick
+
+Tricks and Flexicons are never actually deleted, just removed from
+the library's data collection. If you want to delete them, you need
+to delete the actual file(s) themselves.
+
+$ remove-book [FlexiconName]
+
+ removes everything from FlexiconName from all shelves.
+
+$ remove-shelf [shelf]
+
+ empty that shelf.
+
+$ remove-trick [TrickName|switch]
+
+ Remove [TrickName|switch] from the shelves.
+
+$ add-book [no-test] [shelf] [FlexiconName]
+
+ If no shelf is given, a new shelf bearing FlexiconName is created.
+
+$ add-trick [no-test] [shelf] [TrickName]
+
+ If no shelf is given, it will be added to 'general'.
+
+ If no-test is set, the addition will not be tested but marked
+ 'compromised'.
+
+$ use-book [FlexiconName]
+
+ Empty all shelves, load FlexiconName
+
+$ use-trick [TrickName]
+
+ Empty all shelves, use TrickName instead.
+
+$ show-shelf [shelf|all] [switches|TrickName]
+
+ show what tricks are on the shelf and the original FlexiconName if
+ applicable.
+
+$ trick-show-[datafields] [switch|TrickName|keywords]
+
+ where [datafields] are any data fields a Trick contains.
+ Example: trick-show-precis-command
+
+$ book-show-[datafields] [Flexicon|shelf|keywords]
+
+ where [datafields] are any data fields a Trick contains.
+ Example: book-show-author-status
+
+$ move-trick [TrickName] [shelf]
+
+ move trick to a shelf.
+
+$ copy-trick [TrickName] [shelf]
+
+ copy trick to a shelf.
+
+
+Testing books and tricks
+------------------------
+
+List: validate-trick, validate-shelf, validate-book,
+validate-show-failures
+
+$ validate-trick [Name]
+$ validate-shelf [shelf]
+$ validate-book [Flexicon]
+
+ run tests on the above. Tricks marked 'Compromised' will be
+ disabled. Show report of disabled items.
+
+$ validate-show-failures
+
+ display complete report of failed tests.
+
+
+Producing books:
+----------------
+
+List: write-book, gift-wrap
+
+$ write-book [shelf]
+
+ write new book from shelf contents.
+
+$ gift-wrap [FlexiconName]
+
+ add author's digital signature and make md5sum (I assume you can
+ securely add the actual md5sum into the signature somehow, so
+ malicious modification is not easily possible despite it being plain
+ text)
+
+
+F. Data Fields
+==============
+
+A trick is a plain text file using the LaTeX macro style format, ie,
+\fieldName{<contents>}
+
+The following fields are defined.
+
+* denotes an mandatory entry to ensure overall consistency
++ denotes user/community set mandatory entry
+- denotes optional entry that users/communities can set.
+& encrypted mandatory field (probably a key for a server?)
+% encrypted optional field (probably a key for a server?)
+# constant mandatory entry
+R range (min, max, numerical, in [])
+
+Absence of a non-mandatory field is interpreted as 'none'.
+
+Tricks
+------
+
++ Flexicon Name of Flexicon this belongs to
++ Organisation Name of community/company
+# Author The original designer
+# ID unique numerical ID
+% AuthorEmail Email of original designer
++ Maintainer Current maintainer
+% MaintainerEmail Email of maintainer
+- MailingList mailing list for this trick
+- Contributers people who made/fixed this
+- PublicKey To sign the trick
+- md5sum Enable safety check of trick
+* Status public, restricted, personal
++ License Licence type
++ Copyright some tricks are propietary
++ Encryption whether the trick is secured
+- Reviewers last person/group to review this
+# TrickName One word alias name
+* Precis Short description (one line)
+* Description Long description
+* LongSwitch official Long switch
++ ShortSwitch official short switch
+- UserLongSwitch user set alias for long switch
+- UserShortSwitch user set alias for short switch
+* Version tricks have versions
++ OperatingSystem, R what OS to use (absence = any)
++ ExportVariables shell scripts may require env vars set
++ ExportVariablesDescription description of req. env vars
+* Software list of req. SW and [version ranges]
+* Shell What shell to use
+* Command script or command of the trick
+* Keywords a list of tags eg, bash, svn, emacs
++ TestShell What driver to use to run the test
++ TestEnvironment environment script for the test
++ TestInput data for the test
++ TestCode code for the test
++ TestExpectation expected output
++ TestResult expected result
++ FunctionalityStatus result of last test run
+- Notes Extra notes
++ Bugs Bugs and such
++ Alternatives Similiar tricks (IDs thereof)
++ Ancestor Trick that was used to make this
++ Successor Newer version of trick (ID thereof)
+* InitialCreationDate When was it made?
+* LastUpdated When was it last fixed?
++ WebAddress Canonical source of this trick
++ UserDefinedFields Some empty spots for future use
++ Compromised true if a test failed
++ Server Origin
+
+Flexicon/Library
+------------------
+
+# Real ID Unique ID
+* Working ID for inofficial/personal copies
+# FlexiconName Name
+- Public Key To sign the Flexicon
+- md5sum Enable safety check of Flexicon
++ OperatingSystem, R what OS to use (absence = any)
+* Software list of req. SW and version ranges
+* Shell,R What shell to use
+* Precis Short description (one line)
+* Description Long description
++ Tricklist list of tricks added
+* InitialCreationDate When was it made?
+* LastUpdated When was it last updated?
++ WebAddress Canonical source of this Flexicon
++ Alternatives Similiar Flexicons (IDs thereof)
++ Ancestor Original Flexicon or None
++ Successor Newer version of Flexicon (ID thereof)
+* Keywords a list of tags eg, bash, svn, emacs
++ Organisation Name of community/company
+# Author The original designer
+% AuthorEmail Email of original designer
++ Maintainer Current maintainer
+% MaintainerEmail Email of maintainer
+- Mailing list mailing list for this trick
+- Contributers people who made/fixed this
+* status public, restricted, personal
++ License Licence type
++ Copyright some Flexicons are propietary
++ Encryption whether the trick is secured
+- Reviewers last person/group to review this
++ Compromised true if a test failed
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscr...@labs.apache.org
For additional commands, e-mail: commits-h...@labs.apache.org
