Re: best practices query: non-emacs packages based on tangled source

2020-10-28 Thread Greg Minshall
Immanuel,

my take

> - Not possible to tangle all code going to a specified file

i'm not sure what this means.

> - Not possible to add line directives without major surgery

yes, that would be a problem (... if i were programming in,
e.g., C, etc., so *is* for other people).

> - Not all language modes do the correct thing

my *sense* here is that the level of standardization across different
babel languages in terms of which header arguments are supported, as
well as their meanings, is moderate.

one solution to this (which seems to have happened, at least to an
extent) is good per-language documentation of which header arguments are
supported, and what they mean within the context of the particular
language.

harder would be agreement and implementation of some "core set" of
header arguments/semantics on which one could count, followed by a list
of per-langugage extensions, or followed by places where "we agree to
disagree", or some such?  anyway, a lot of work.

> - No way to prevent overwriting an unchanged file

yes, i too have a hack i use for this (and, it would be nice if there
were something 'out of the box' that did this).

> - Special casing e.g. Don=E2=80=99t tangle to a file called =E2=80=98no=E2=
> =80=99.

somewhere, some how, fonts were lost. :)

cheers, Greg



Re: best practices query: non-emacs packages based on tangled source

2020-10-28 Thread Immanuel Litzroth
I think the org way of tangling just doesn't work well for compiled languages.
Here are some reasons:
- Not possible to tangle all code going to a specified file
- Not possible to add line directives without major surgery
- Not all language modes do the correct thing
- No way to prevent overwriting an unchanged file
- Special casing e.g. Don’t tangle to a file called ‘no’.
Specific gripe:
- :prologue and :epilogue are not honoured by all modes
>>>
#+ BEGIN_SRC emacs-lisp :prologue "hello" :epilogue "goodbye" :tangle ex1.el
(+ 1 2)
#+ END_SRC


#+ BEGIN_SRC haskell :tangle ex1.hs :prologue "hello" :epilogue "goodbye"
hey
#+ END_SRC

hey

#+HEADER: :tangle ex1.cpp
#+ BEGIN_SRC cpp :prologue "hello" :epilogue "goodbye"
main()
{}
#+ END_SRC

#+ BEGIN_SRC bash :tangle ex1.bash :prologue "hello" :epilogue "goodbye"
aha
#+ END_SRC
>>>

Based on the org machinery I've written a simple tangler which addresses some
of these shortcomings. It can add line directives to recursively expanded noweb
references -- and get that right. It can also take any action at the
end of tangling
like comparing your tangled file to the one on disk and not overwrite
if it's the same,
or just sending your tangled buffer to an interpreter not writing it
to file (remember that
error references point back to the .org file). Also it can tangle all
blocks going to the
same file. This has been tested for Haskell, C++ and elisp.
It's in a private repo on github because it's not ready for release
yet, but if someone
is interested I can give them access.
Immanuel

On Tue, Oct 27, 2020 at 11:30 PM Dr. Arne Babenhauserheide
 wrote:
>
>
> TRS-80  writes:
>
> > Therefore, any stuff I plan on releasing publicly, I do not do in
> > literate style (JMHO).  However if you are dead set on doing literate
> > paradigm, then maybe my experience is invalid for your use-case.
>
> My experience is that literate style works very well for tutorials, but
> whenever I developed a larger program within org-mode, I tangled it once
> I had to start debugging and then kept it as separate source.
>
> Once the code becomes complex, I want to stay in one programming
> language without outside features that also add extra complexity.
>
> That said, for simple code the additional freedom of multi-language
> programming in org-mode is awesome. I’m using that to create random
> roleplaying characters directly from random tables in org-mode, with
> those tables being the primary source (because the book is the most
> important product, not the code).
>
> Best wishes,
> Arne
> --
> Unpolitisch sein
> heißt politisch sein
> ohne es zu merken



-- 
-- Researching the dual problem of finding the function that has a
given point as fixpoint.



Re: best practices query: non-emacs packages based on tangled source

2020-10-27 Thread Dr. Arne Babenhauserheide

TRS-80  writes:

> Therefore, any stuff I plan on releasing publicly, I do not do in
> literate style (JMHO).  However if you are dead set on doing literate
> paradigm, then maybe my experience is invalid for your use-case.

My experience is that literate style works very well for tutorials, but
whenever I developed a larger program within org-mode, I tangled it once
I had to start debugging and then kept it as separate source.

Once the code becomes complex, I want to stay in one programming
language without outside features that also add extra complexity.

That said, for simple code the additional freedom of multi-language
programming in org-mode is awesome. I’m using that to create random
roleplaying characters directly from random tables in org-mode, with
those tables being the primary source (because the book is the most
important product, not the code).

Best wishes,
Arne
-- 
Unpolitisch sein
heißt politisch sein
ohne es zu merken


signature.asc
Description: PGP signature


Re: best practices query: non-emacs packages based on tangled source

2020-10-27 Thread TRS-80

On 2020-10-15 14:11, Greg Minshall wrote:

if anyone has any techniques they've used, liked (or hated), i'd
love to hear.



I am someone who keeps my Emacs config in a literate style in blocks
within an Orgmode outline.  And I enjoy it!

However I somewhat recently came across a project that was developed
with a literate paradigm.  It was an immediate turn off for me.
Because now in addition to learning someone else's code, I have to
learn how their brain works, how they organized the file, etc.  It
just added another layer of complexity on top of something that is
already potentially complex (the code itself).

Maybe I was just immediately soured because I didn't like their
formatting (not leaving enough spaces where I would have, etc.) or how
they organized the project.  And I probably didn't give it a fair
shake if I'm being honest.  But I don't have a lot of time, and things
like that just create too much friction (IMO) when I was only slightly
motivated to make a contribution in the first place.

And remember what I said at first, I am someone who appreciates the
literate paradigm (at least for my own config).  Imagine how someone
who does not even know who Knuth is would feel about it.

Therefore, any stuff I plan on releasing publicly, I do not do in
literate style (JMHO).  However if you are dead set on doing literate
paradigm, then maybe my experience is invalid for your use-case.

Cheers,

TRS-80



Re: best practices query: non-emacs packages based on tangled source

2020-10-27 Thread Greg Minshall
Tom,

thanks very much for your very detailed analysis and explanation of the
issues you've had.  it's taken me this long to read through it
carefully.

the issues with python source blocks [C-c C-c] versus [C-c C-v C-t] are
in fact unfortunate, as it does make it hard to develop without lots of
tangling.  (i've run into that before with Python/org, but without
really understanding what was going on.)

i mostly write R code these days, and it's pretty happy either way.
however, i also write javascript code, and while i do it for front end
(so, don't execute it inside of org mode), i suspect it has the same
issues python does of conflating file names and modules, and might
benefit from some enhancement you suggest about treating source blocks
as "things" to be imported into the running babel session (maybe an
enhancement of <>?).

for me, my ideal will be when the org "language" is documented as an
RFC, and people have produced stand alone tanglers, so i can release
code, and others, without emacs, without org mode, can edit the "source
of truth that is the org file", and run make to tangle and build
whatever random software package i'm releasing.

again, thank you for the reply.

cheers, Greg



Re: best practices query: non-emacs packages based on tangled source

2020-10-18 Thread Tom Gillespie
Hi Greg,

Great question. This came out a bit longer than I anticipated since I
wrote up a couple of relevant workflows. Sync between org source
blocks and tangled code is something that I think needs improvement. I
have covered the difference in semantics between tangled code and
babel evaluated code (along with some others factors) since it can
have an impact on what workflows you might choose.

Following on Timothy's suggestion, one key thing that I think is
needed is the ability to detangle nested and arbitrary code. Detangle
of code tangled with :comments noweb is not fully implemented. A full
detangling implementation would make it easier and safer to
automatically detangle back to the org source blocks using a
pre-commit hook or similar. Detangling from the org sources is also
something that needs to be implemented for this to work. I think that
a more complete detangling implementation could go a long way toward
making it easier for those who are not used to org to commit to a
project. In the absence of this, I have found that most of my existing
workflows actively avoid keeping tangled code and org sources tracked
in git at the same time unless absolutely necessary, and even with git
there to back me up I have shot myself in the foot tangling over files
that I forgot to detangle.

Below are a few examples. In all cases I have had to consciously work
around the issue of having tangled code that is outside the source of
truth that is the org file.

As an additional note before the examples, I have found that the trade
offs when tangling code also depend on the language you are using. For
example, I consider some languages, such as Python, to be obligate
tanglers since their semantics conflate modules and files. Org babel
might be able to work around this in some cases, but it would mean,
for example, that ob-python would have to explicitly compensate for
this deficiency by implementing the ability to treat source blocks as
modules to be loaded into a session or somehow pulled in during the
prologue by pre-parsing blocks to look for import statements, etc. The
deficiencies of a language mean that if you want certain functionality
for that language then org babel can't just treat the code as text,
and might have to go to great lengths to try to keep the semantics of
babel evaluation and of tangled code aligned.

The workflow that I have found to be the most reasonable I developed
while working on an elisp project (it is public but I'm not quite
ready to link it on this list). In this workflow I add a hook via
~(add-hook 'before-save-hook #'org-babel-tangle nil t)~ for any org
file that should be tangled, that way I don't have to worry about
whether I remember to tangle etc. However, there are a couple of
issues. It does not work in reverse, you still always have to edit the
org blocks. It will become annoyingly slow if you have many blocks to
tangle. You really want it to run only when the source blocks change,
not whenever the org files changes. Finally, The exact semantics of
tangling multiple blocks to the same file can have a major impact on
performance. So even this best case is not the greatest and doesn't
enable your specific use case (detangle issues in particular are a
show stopper). Since this is an elisp project I have to run tests on
the tangled file in a separate instance of Emacs to ensure that it
works as expected.

In this project I also have a completely unreadable file that is valid
and executable in 3 languages. Tangled blocks that are easily readable
in the org sources are commented out intentionally in the tangled
file. This is a worst case for detangling. I suspect that it can be
done, but it will push the detangling implementation to the limit. At
the moment, there is no way to detangle this file back to readable
form at all, and it is not clear that anyone should try to edit the
tangled file in the first place. All this to say, if we reason from
this extreme example, maybe the best thing is to tangle at the last
possible moment, never keep the tangled form under version control
etc. Unfortunately the use case for this file is to bootstrap Emacs,
which means that in order for it to be useful it _must_ be tangled and
put under version control since the systems it needs to run on don't
have Emacs.

The worst experience I have had was when I was developing python code
where I needed to capture the output of the block in order to populate
tables. Over time the code grew to the point where it needed to become
a library. This is where Python being an obligate tangler reared its
head, and the differences in semantics between tangled and evaluated
code became a major pain. Combine this with the fact that my testing
workflows in Python essentially require me to edit the tangled code
for me not to lose my mind, but I would also forget to detangle, and
sometimes overwrite on retangle, and I was quite unhappy.

As a result I eventually gave up and moved all python development out
of the 

Re: best practices query: non-emacs packages based on tangled source

2020-10-16 Thread TEC



Hi Greg,

Just one little thing that occurs to me, for accepting PRs if you 
add

the header arg (globally would probably be best) :comments link
That with M-x org-babel-detangle should help with accepting PRs.

Hope that helps,

Timothy.

Greg Minshall  writes:

hi.  i apologize if this has been asked before (especially if by 
me).
but, since i had a question recently about Org Src... buffers, 
this came

up.

i'm wondering what people do who want to release a non-emacs'y 
package
(an R package, say, or ...), and who did their development "from 
within"

a .org file.

i can "build" whatever files are needed to release the package. 
but,
it's nice to be able to let people look at the sources, maybe 
submit

'pull requests', etc.

if anyone has any techniques they've used, liked (or hated), i'd 
love to

hear.

thank you very much, Greg





Re: best practices query: non-emacs packages based on tangled source

2020-10-16 Thread Diego Zamboni
Hi Greg,

What I do with my Elvish modules (https://github.com/zzamboni/elvish-modules
, https://github.com/zzamboni/elvish-completions) is to just include the
Org files together with the tangled .elv files.

--Diego


On Thu, Oct 15, 2020 at 8:28 PM Greg Minshall  wrote:

> hi.  i apologize if this has been asked before (especially if by me).
> but, since i had a question recently about Org Src... buffers, this came
> up.
>
> i'm wondering what people do who want to release a non-emacs'y package
> (an R package, say, or ...), and who did their development "from within"
> a .org file.
>
> i can "build" whatever files are needed to release the package.  but,
> it's nice to be able to let people look at the sources, maybe submit
> 'pull requests', etc.
>
> if anyone has any techniques they've used, liked (or hated), i'd love to
> hear.
>
> thank you very much, Greg
>
>


Re: best practices query: non-emacs packages based on tangled source

2020-10-16 Thread Eric S Fraga
On Thursday, 15 Oct 2020 at 21:11, Greg Minshall wrote:
> i can "build" whatever files are needed to release the package.  but,
> it's nice to be able to let people look at the sources, maybe submit
> 'pull requests', etc.

I have recently done this with a Julia project which I make available at
github.  All code and documentation is developed in a single org and
files tangled etc. to create the github repository for the project.

I include the org file in the repository although the HTML that I
generate from it (as documentation) is hosted on my work web site for
visibility reasons.

If you're interested, you can see all the files including the complete
org file at https://github.com/ericsfraga/Fresa.jl

-- 
: Eric S Fraga via Emacs 28.0.50, Org release_9.4-57-g8402c4



Re: best practices query: non-emacs packages based on tangled source

2020-10-15 Thread Tim Cross


There is no great answer I am aware of. However, I will sometimes
generate a markdown version of the source so that at least non-emacs
users have a slightly better chance of being able to view the source in
a more format friendly manner than a 'raw' org file. However, pull
requests and the like are more likely going to be diffs on the generated
sources as most people will want to use their preferred editor and
that will likely need the generated source file is in order to get their
editor 'IDE' features etc.

This is one reason I tend not to use org's 'literate programming' model
for anything other than documentation, simple examples, configuration
files, sql and basic scripting. I find for more complex development,
especially when it requires multiple files, namespaces/modules, long
running repl sessions, extensive test suits etc, using org adds another
layer of complexity which soon outstrips the benefits of having
documentation and source in one file. Of course, this will also depend
on the development language/platform. I tend to use languages which
involve a fair bit of 'REPL' based development rather than a more
traditional write, generate, compile, debug loop. On the other hand,
when it comes to documentation, tutorials, configuration files and
workflow automation, org is definitely my preferred tool. 

Tim

Greg Minshall  writes:

> hi.  i apologize if this has been asked before (especially if by me).
> but, since i had a question recently about Org Src... buffers, this came
> up.
>
> i'm wondering what people do who want to release a non-emacs'y package
> (an R package, say, or ...), and who did their development "from within"
> a .org file.
>
> i can "build" whatever files are needed to release the package.  but,
> it's nice to be able to let people look at the sources, maybe submit
> 'pull requests', etc.
>
> if anyone has any techniques they've used, liked (or hated), i'd love to
> hear.
>
> thank you very much, Greg


-- 
Tim Cross



best practices query: non-emacs packages based on tangled source

2020-10-15 Thread Greg Minshall
hi.  i apologize if this has been asked before (especially if by me).
but, since i had a question recently about Org Src... buffers, this came
up.

i'm wondering what people do who want to release a non-emacs'y package
(an R package, say, or ...), and who did their development "from within"
a .org file.

i can "build" whatever files are needed to release the package.  but,
it's nice to be able to let people look at the sources, maybe submit
'pull requests', etc.

if anyone has any techniques they've used, liked (or hated), i'd love to
hear.

thank you very much, Greg