Re: literate programming, development log -- ideas? (ominbus reply)

[Tim Cross]

"Samuel Banya"  writes:

> Hey Tim,
>
> Wow that's an awesome idea too aka to create local branches on each machine, 
> and then switch over the master branch to pull them in.
>
> Do you have any video references for this kind of thing?
>
> I like videos first, and then documentation.
>
> It would really help me out since I'd love to try to figure this kind of 
> thing out, since I am pretty close to just using rsync for my Git notes going 
> forward,
> but needed that kind of convincing otherwise.
>

Sorry, no video links/references. There are lots out there, I just don't
watch them. I think one of the best ways to understand git is to just
use it at the command line and experiment. Git can appear daunting, but
it is actually fairly easy. It has a huge number of commands and
options, but 90% of the time, you really only need a very small handful.

The nice thing about learning a VCS is that you can clone a repository,
experiment as much as you like and when you make a mistake, try to use
git to undo it, but if all else fails, blow away your copy and clone and
try again.

Probably the biggest complaint I see about git (or almost any VCS) is
merge hell. However, this can be largely avoided with a few simple rules
and some discipline in your workflow. People often get into this type of
mess because they make lots of changes before committing them and
because they make changes on the master branch rather than a topic
branch.

I almost never make changes directly to the master branch. I will create
a new branch for even small changes and then merge it back into master
once the changes are complete. Sometimes, I forget and realise I've made
changes which should have been done in a branch. When this happens, I
use the 'stash' command to stash my changes, create a topic branch and
then apply the stash to that branch.

If a topic branch has existed for a while and you know the master branch
has diverged significantly since you made your topic branch, it can be
useful to rebase your topic branch against the current master branch.
This will effectively apply all the changes from master to your topic
branch *before* then applying your changes. It is like you have just
forked your topic branch from master. You may still get some conflicts
when re-applying your branch changes, but there will typically be less
and they will be easier to resolve. 

If your working on a project with lots of contributors, where master
changes quite quickly, you may want to do frequent pulls and either
merge or rebase into your topic branch. Merge conflict really only
becomes a pain when there are lots of changes to all apply at once. If
I'm working on a large project with lots of contributors, I might do a
pull and merge or rebase into my topic branch every day (maybe multiple
times per day). On a private repository, I will do this far less often
because I know what has changed and when it is a good idea to merge or
rebase my current topic branch.

Provided some discipline is applied, I find when it comes time to merge
my topic branch back into master, it is a straight forward merge with no
conflicts and changes in master remain clean. 

It is sometimes a good idea to poke around a bit in the .git directory
and get to understand what is in there and how it works. It is
surprisingly simple once you understand the structure. One of the things
I like about git is that it makes simple things simple and hard things
possible.

Sometimes, using some sort of sync software is the right solution. I
regularly use rsync for a number of tasks. However, I use git and remote
git services like gitlab or github for all of my projects and org notes.
Some people prefer Mercurial (hg) over git. I think it is a fine VCS as
well and often use it too. It has many of the same concepts and can be
used in a similar manner. For some people, it sits better with the way
they work/think than git. However, the core concepts are the same.  

Re: [wip-cite-new] Adjust punctuation around citations

[Bruce D'Arcus]

I'll let you two sort it out; I don't have a position.

On Sun, Jun 13, 2021, 3:23 PM Denis Maier  wrote:

>
> Bruce D'Arcus  hat am 14.06.2021 00:04 geschrieben:
>
>
> Nicolas explained the reverse is out of scope,
>
> IIRC, it was out of scope ATM.
>
> and gave a reasonable explanation why (because much harder to reconstruct
> missing information IIRC).
>
> That's where I disagree. I think the opposite is true.
>
> On Sun, Jun 13, 2021, 2:54 PM Denis Maier  wrote:
>
> Am 12.06.2021 um 11:39 schrieb Nicolas Goaziou:
> > Hello,
> >
> > Denis Maier  writes:
> >
> >> Yes, good this is coming.
> >
> > As a step forward, I rebased wip-cite-new branch with more support for
> > note numbers handling.
> >
> > I added three customizable variables:
> >
> > - org-cite-adjust-note-numbers, which simply allows the user to toggle
> >punctuation and note number moving (on by default).
> >
> > - org-cite-note-rules, which defines what rules to apply according to
> >locale, expressed as a language tag, as in RFC 4646.
> >
> > - org-cite-punctuation-marks, which lists strings recognized as
> >punctuation in the process.
> >
> > `csl' and `basic' processors now both make use of this.
> >
> > I'd appreciate some feedback, in particular about the docstrings of the
> > variables above. I focused on the "note numbers" topic instead of
> > "punctuation" since I found the latter too generic.
> >
> > Also, there are some points that may need to be discussed:
> >
> > - I'm not sure about the `org-cite-punctuation-marks' variable being
> >global, i.e., not locale-specific.
> >
> > - There is no support for this in LaTeX-derived back-ends, because
> >I don't know when a citation is going to become a footnote. As
> >a reminder, there is no "\footcite" command in `biblatex' processor.
> >OTOH, users might prefer using a more advanced mechanism, e.g.,
> >csquotes.
> >
> > - It doesn't do anything special in quote blocks, because I'm still not
> >sure there is something to do. AFAIU, special casing there only
> >applies to author-date location, which out of the scope of this code.
> >
> > WDYT?
>
> Ok, I've managed to test this a bit, and I think this looks pretty good
> so far.
>
> The only question I'd still have is if this could somehow also cover the
> reverse situation (going from a note style to author-date). I've noticed
> that simply adding a new language rule doesn't work anymore---as opposed
> to my initial tests with earlier iterations of that mechanism. Seems
> like this mechanism is now only triggered when using a note based style.
>
> Best,
> Denis
>
> >
> > Regards,
> >
>
>

Re: [wip-cite-new] Adjust punctuation around citations

[Denis Maier]

 
 
  
   
  
  
   
Bruce D'Arcus  hat am 14.06.2021 00:04 geschrieben:
   
   

   
   

   
   
Nicolas explained the reverse is out of scope,
   
  
  
   IIRC, it was out of scope ATM.
   
  
  
   
 and gave a reasonable explanation why (because much harder to reconstruct missing information IIRC).
   
  
  
   That's where I disagree. I think the opposite is true.
   
  
  
   
  
  
   

 On Sun, Jun 13, 2021, 2:54 PM Denis Maier  wrote:
 


 Am 12.06.2021 um 11:39 schrieb Nicolas Goaziou:
 > Hello,
 > 
 > Denis Maier  writes:
 > 
 >> Yes, good this is coming.
 > 
 > As a step forward, I rebased wip-cite-new branch with more support for
 > note numbers handling.
 > 
 > I added three customizable variables:
 > 
 > - org-cite-adjust-note-numbers, which simply allows the user to toggle
 >    punctuation and note number moving (on by default).
 > 
 > - org-cite-note-rules, which defines what rules to apply according to
 >    locale, expressed as a language tag, as in RFC 4646.
 > 
 > - org-cite-punctuation-marks, which lists strings recognized as
 >    punctuation in the process.
 > 
 > `csl' and `basic' processors now both make use of this.
 > 
 > I'd appreciate some feedback, in particular about the docstrings of the
 > variables above. I focused on the "note numbers" topic instead of
 > "punctuation" since I found the latter too generic.
 > 
 > Also, there are some points that may need to be discussed:
 > 
 > - I'm not sure about the `org-cite-punctuation-marks' variable being
 >    global, i.e., not locale-specific.
 > 
 > - There is no support for this in LaTeX-derived back-ends, because
 >    I don't know when a citation is going to become a footnote. As
 >    a reminder, there is no "\footcite" command in `biblatex' processor.
 >    OTOH, users might prefer using a more advanced mechanism, e.g.,
 >    csquotes.
 > 
 > - It doesn't do anything special in quote blocks, because I'm still not
 >    sure there is something to do. AFAIU, special casing there only
 >    applies to author-date location, which out of the scope of this code.
 > 
 > WDYT?
 
 Ok, I've managed to test this a bit, and I think this looks pretty good 
 so far.
 
 The only question I'd still have is if this could somehow also cover the 
 reverse situation (going from a note style to author-date). I've noticed 
 that simply adding a new language rule doesn't work anymore---as opposed 
 to my initial tests with earlier iterations of that mechanism. Seems 
 like this mechanism is now only triggered when using a note based style.
 
 Best,
 Denis
 
 > 
 > Regards,
 > 
 
 

   
  
 

Re: [wip-cite-new] Adjust punctuation around citations

[Bruce D'Arcus]

Nicolas explained the reverse is out of scope, and gave a reasonable
explanation why (because much harder to reconstruct missing information
IIRC).

On Sun, Jun 13, 2021, 2:54 PM Denis Maier  wrote:

> Am 12.06.2021 um 11:39 schrieb Nicolas Goaziou:
> > Hello,
> >
> > Denis Maier  writes:
> >
> >> Yes, good this is coming.
> >
> > As a step forward, I rebased wip-cite-new branch with more support for
> > note numbers handling.
> >
> > I added three customizable variables:
> >
> > - org-cite-adjust-note-numbers, which simply allows the user to toggle
> >punctuation and note number moving (on by default).
> >
> > - org-cite-note-rules, which defines what rules to apply according to
> >locale, expressed as a language tag, as in RFC 4646.
> >
> > - org-cite-punctuation-marks, which lists strings recognized as
> >punctuation in the process.
> >
> > `csl' and `basic' processors now both make use of this.
> >
> > I'd appreciate some feedback, in particular about the docstrings of the
> > variables above. I focused on the "note numbers" topic instead of
> > "punctuation" since I found the latter too generic.
> >
> > Also, there are some points that may need to be discussed:
> >
> > - I'm not sure about the `org-cite-punctuation-marks' variable being
> >global, i.e., not locale-specific.
> >
> > - There is no support for this in LaTeX-derived back-ends, because
> >I don't know when a citation is going to become a footnote. As
> >a reminder, there is no "\footcite" command in `biblatex' processor.
> >OTOH, users might prefer using a more advanced mechanism, e.g.,
> >csquotes.
> >
> > - It doesn't do anything special in quote blocks, because I'm still not
> >sure there is something to do. AFAIU, special casing there only
> >applies to author-date location, which out of the scope of this code.
> >
> > WDYT?
>
> Ok, I've managed to test this a bit, and I think this looks pretty good
> so far.
>
> The only question I'd still have is if this could somehow also cover the
> reverse situation (going from a note style to author-date). I've noticed
> that simply adding a new language rule doesn't work anymore---as opposed
> to my initial tests with earlier iterations of that mechanism. Seems
> like this mechanism is now only triggered when using a note based style.
>
> Best,
> Denis
>
> >
> > Regards,
> >
>
>

Re: [wip-cite-new] Adjust punctuation around citations

[Denis Maier]

Am 12.06.2021 um 11:39 schrieb Nicolas Goaziou:

Hello,

Denis Maier  writes:


Yes, good this is coming.


As a step forward, I rebased wip-cite-new branch with more support for
note numbers handling.

I added three customizable variables:

- org-cite-adjust-note-numbers, which simply allows the user to toggle
   punctuation and note number moving (on by default).

- org-cite-note-rules, which defines what rules to apply according to
   locale, expressed as a language tag, as in RFC 4646.

- org-cite-punctuation-marks, which lists strings recognized as
   punctuation in the process.

`csl' and `basic' processors now both make use of this.

I'd appreciate some feedback, in particular about the docstrings of the
variables above. I focused on the "note numbers" topic instead of
"punctuation" since I found the latter too generic.

Also, there are some points that may need to be discussed:

- I'm not sure about the `org-cite-punctuation-marks' variable being
   global, i.e., not locale-specific.

- There is no support for this in LaTeX-derived back-ends, because
   I don't know when a citation is going to become a footnote. As
   a reminder, there is no "\footcite" command in `biblatex' processor.
   OTOH, users might prefer using a more advanced mechanism, e.g.,
   csquotes.

- It doesn't do anything special in quote blocks, because I'm still not
   sure there is something to do. AFAIU, special casing there only
   applies to author-date location, which out of the scope of this code.

WDYT?


Ok, I've managed to test this a bit, and I think this looks pretty good 
so far.


The only question I'd still have is if this could somehow also cover the 
reverse situation (going from a note style to author-date). I've noticed 
that simply adding a new language rule doesn't work anymore---as opposed 
to my initial tests with earlier iterations of that mechanism. Seems 
like this mechanism is now only triggered when using a note based style.


Best,
Denis



Regards,

Re: literate programming, development log -- ideas? (ominbus reply)

[Samuel Banya]

Hey Tim,

Wow that's an awesome idea too aka to create local branches on each machine, 
and then switch over the master branch to pull them in.

Do you have any video references for this kind of thing?

I like videos first, and then documentation.

It would really help me out since I'd love to try to figure this kind of thing 
out, since I am pretty close to just using rsync for my Git notes going 
forward, but needed that kind of convincing otherwise.

On Sun, Jun 13, 2021, at 12:46 AM, Tim Cross wrote:
> 
> "Samuel Banya" mailto:sbanya%40fastmail.com>> writes:
> 
> > Not sure if it counts as off-topic for this thread, but does everyone use 
> > Git to manage their Org docs and notes?
> >
> > I ask because of Greg's previous post.
> >
> > I've noticed that some times after git merge events across a few machines 
> > (ex: I forgot I had already pushed notes for my private notes on one 
> > machine,
> > and had to merge the results from another machine), I'll get weird "HEAD" 
> > and "END" statements inserted by Git.
> >
> > Also, combined with some tasks duplicating as a result was annoying.
> >
> > Was debating if this is just something I'd have to deal with, or if there 
> > might be a better versioning workflow (ex: just using rsync, etc)
> >
> > Would be curious on everyone's thoughts.
> >
> > ~ Sam
> >
> 
> I use git as the master and then checkout to whatever machine I'm working
> on. I tend to have at least 3 different machines I'm working on (home
> Linux, work Linux and Macbook).
> 
> On each machine, I will checkout from master and then create a 'local'
> branch where I make any local changes. When I'm finished working
> locally, I will commit to the local branch, switch to the master branch,
> do a pull. If no changes are pulled, then I will merge in the local
> branch and push up to the master repository. If changes are pulled, then
> I will make a decision whether to use rebase to add those changes to my
> local branch or just merge. Deciding on which depends on the types of
> things changed, size of what has changed etc.
> 
> I find rebasing and merging is often the best approach to keeping commit
> logs fairly clean and linear. However, that will depend on what is being
> changed and the amount of changes. Frequent pulling and either merging
> and rebasing is useful.
> 
> Creating new branches (both just locally and within the master
> repository) is a very lightweight operation. I use lots of branches and
> will regularly go back through and get rid of old branches when no
> longer needed (i.e. changes in the branch have been merged into master
> or the branch topic is no longer relevant/needed etc). Understanding the
> difference between a rebase and a fast-forward merge is important.
> Likewise, using branches effectively is critical. My master branch tends
> to be quite clean - I almost never make changes directly in the master
> branch. Everything happens in another branch and later merged into the
> master when ready. 
> 
> 

Re: literate programming, development log -- ideas?

[Samuel Banya]

Hey everyone, 

I just wanted to put this here in the Literate Programming thread, DistroTube 
just did a video on why Emacs rules, but the following timestamps are pretty 
dang useful for literate programming for still-always learning newbies like 
myself, just wanted to mention them here for reference:
Timestamps:
7:20
11:46
15:35
17:36

What Are The Benefits Of Emacs Over Vim? 
(https://www.youtube.com/watch?v=kRkp-uJTK7s)

On Sun, Jun 13, 2021, at 12:24 AM, Tim Cross wrote:
> 
> Eric S Fraga mailto:e.fraga%40ucl.ac.uk>> writes:
> 
> > On Monday,  7 Jun 2021 at 14:43, Greg Minshall wrote:
> >> i write most of my code in a (per-project) .org file, which is typically
> >> tangled into source or script files.  
> >
> > I do the same.
> >
> >> i'm wondering if people do this, especially the development log, and if
> >> there are any hints or practices people might feel would be of interest
> >> to share.
> >
> > I use version control for this aspect, liberally adding/deleting
> > text/code and relying on the version control system to keep the log for
> > me.  I used to try to keep the log, as you call it, within the org file
> > but that seemed eventually to be both difficult and pointless when there
> > are decent version control tools out there.
> >
> > I use src mostly [1] when everything is going to be in one file.
> >
> > The "current" version of the document will have the code and results
> > that match the text.
> >
> > YMMV, of course.
> >
> > Footnotes:
> > [1]  https://gitlab.com/esr/src
> 
> I do something very similar. I will use org's archive facility as well,
> but git with good commit logs seems to meet most of my needs. The
> current 'master' HEAD is the current 'state' of the code, documentation,
> notes etc.
> 
> 
> -- 
> Tim Cross
> 
> 

Re: [wip-cite-new] Adjust punctuation around citations

[Denis Maier]

 
 
  
   Hi Nicolas,
   
  
  
   thanks for all your work on this. I've been busy lately, but I'll try to test ASAP. (Might take a few though, just to list you know.)
  
  
   Denis
   
  
  
   
Nicolas Goaziou  hat am 13.06.2021 00:04 geschrieben:
   
   

   
   

   
   
Hello,
   
   

   
   
"Bruce D'Arcus"  writes:
   
   

   
   
>> - I'm not sure about the `org-cite-punctuation-marks' variable being
   
   
>> global, i.e., not locale-specific.
   
   
>
   
   

 One of the things I've wondered about, but cannot offer any expertise


 on (really, I have no clue), is how this might work for languages like


 Arabic or Mandarin.

   
   
Of course, different punctuation marks exist, so characters are at least
   
   
locale-dependant. But I don't know if the set is entirely determined by
   
   
the locale or if it also depends on the "style" of the document.
   
   

   
   
>> - There is no support for this in LaTeX-derived back-ends, because
   
   
>> I don't know when a citation is going to become a footnote. As
   
   
>> a reminder, there is no "\footcite" command in `biblatex' processor.
   
   
>> OTOH, users might prefer using a more advanced mechanism, e.g.,
   
   
>> csquotes.
   
   
>
   
   

 So the upshot is if users want this functionality for LaTeX/PDF, they


 should use oc-biblatex for that export target?

   
   
Actually, that's the opposite. I don't know if it is possible, or even
   
   
appropriate, to port that feature to `biblatex' and `natbib' processors.
   
   
So, ATM, if you want it, and target LaTeX/PDF, your only option is
   
   
`csl'.
   
   

   
   
>> - It doesn't do anything special in quote blocks, because I'm still not
   
   
>> sure there is something to do. AFAIU, special casing there only
   
   
>> applies to author-date location, which out of the scope of this code.
   
   
>
   
   

 Here's the scenario I believe Denis was concerned about:

   
   

 #+begin_quote


 ... block with citation at end. [cite:@doe]


 #+end_quote

   
   

 I think in standard author-year styles, we'd want:

   
   

 ... block with citation at end. (Doe, 2020)

   
   

 ... while in note-based and "US" (space removed):

   
   

 ... block with citation at end.[1]

   
   

 Not sure if that has any practical import, but just to clarify.

   
   
I understand this, but, AFAICT, this is the output we get already. So
   
   
I don't think there is anything to do.
   
   

   
   
Thank you for the feedback.
   
   

   
   
Regards,
   
   
--
   
   
Nicolas Goaziou
   
  
 

Re: literate programming, development log -- ideas? (ominbus reply)

[Tim Cross]

Greg Minshall  writes:

> Tim,
>
> thanks for your comments.
>
>> A lot depends on whether what you want is an org file which documents
>> the current state of play or one which is more similar to a lab book
>> which contains a more chronological type evolution of ideas and
>> experiments. I often setup completely separate org 'projects' which will
>> consist of multiple org files and will move things between different
>> files as the project evolves. In some extreme cases, I may even have
>> multiple git branches and will often use git tags to mark specific
>> 'milestones'.
>>
>> How I decide whether to use a date tree with notes, branches, tags,
>> archive sections/files, separate org files etc is typically determined
>> by how likely I might be to want to review or go back through previous
>> work/experiments/decisions. Working this out can take a bit of time and
>> experimentation, but in general, I rarely need to checkout old versions
>> or even open archive trees/files. I do have a journal file for each
>> major project which has lots of ideas, random thoughts and even small
>> experiments (with source blokcs) and I tend ot have a large 'reference'
>> file which contains notes and links to external references and then a
>> 'main' org file, which reflects the current state. 
>
> my current curiosity is in how to integrate lab book, brain storming,
> functionality into my projects.  it seems as if, in an extreme case, you
> might possibly have
> - a lab book sort of file (i.e., date order, minimal "going back and
>   correcting entries")
> - a journal file, unstructured, not-infrequently updated, notes, URLs,
>   etc.
> - the "main" org file -- that which "ships".
> - an archive file (one per each of the other?)
>
> for any given project, is the evolution from =foo.org= to this larger
> number sort of organic, in the sense that you start with one file, then,
> at some point, say, "okay, i need to put these bits in a new journal
> file"?
>
> and, trying to leverage the brain cells of others, have you tended to
> settle on any sort of consistent naming scheme for the different files?
> =-log=, =-notes=, etc.?
>
> i suspect that if my brain were more git-shaped, i'd find the idea of
> separate files easier.  i.e., my instinct is to think of each of these
> files as having a version-path independent of the others.  rather than
> the git-view, which is that the version-path is the commit-path, and
> each commit includes the (mostly-temporally) related versions of each of
> again, thanks!  Greg


For me, the growth was quite organic. I still have quite a few single
file 'projects' where everything is in one file (with top level headings
for the different 'bits').

In fact, nearly all my org usage has followed an organic approach. When
I first started with org, I made the common mistake of getting in and
defining lots of todo states, tags, templates for different structures,
lots of capture templates etc etc. I later realised this was a big
mistake and ended up being a fine example of over engineering.

I ended up going back to a stock standard org setup with next to no
configuration. I then used this 'out of the box' setup for a while and
worked out how to do what I wanted using what org already had. I avoided
installing any additional org related packages. Once I was familiar with
what org has and how to use it, I then began to refine my workflow bit
by bit. I slowly started tweaking my configuration and added some simple
elisp functions to make my life easier. I then systematically looked at
various org packages I had heard about and installed a couple which I
found useful.

I now have a very workable and nice workflow where org is integrated
into most of what I do. There is a bit in my config, but most of it is
just setting various org variables, a few capture templates, a custom
agenda view and some tweaking to some of the export stuff. While I use
org pretty much daily, there is a lot of org I just don't use. For
example, I only use tags very sparingly. I use babel to generate my
init.el, numerous config files (my zsh shell setup, my mbsync and lots
of other config files) and I have some language 'logbook' projects where
I experiment and learn different languages, frameworks etc. I tend not
to use babel/literate programming with full on coding projects. I will
use org for documentation, issue tracking, notes etc. However, for
'real' code projects, I tend to keep the code in 'normal' language
dependent files rather than adopt a full literate/noweb style of
development. I do often use org to document and manage deployment
scripts and some 'devops' type stuff though. 
-- 
Tim Cross