Re: [PATCH] capture: Fix handling of time range for :time-prompt

2021-02-02 Thread Richard Lawrence
Kyle Meyer  writes:

> Pushed (3e64d3475).

Thank you!

>> As far as I can tell, that is not fully possible today, even with this
>> patch. The reason is that time *range* information entered at the prompt
>> generated by :time-prompt gets thrown away. The reason for *that* is
>> that org-read-date is called with t as its to-time argument (the second
>> argument), so that the date is returned in Emacs' internal time
>> representation, which doesn't represent ranges.
>
> Hmm.  Is it really about the TO-TIME argument?  If org-read-date is
> called with TO-TIME as nil, doesn't it still throw away the end of the
> range?
>
>   (let ((org-time-was-given nil)
> (org-end-time-was-given nil))
> (org-read-date nil nil nil "Date for tree entry:"))
>   ;; enter "3pm-4pm" => "2021-02-01 15:00"
>
> Or, actually, it stores the end in org-end-time-was-given, but it does
> that regardless of the TO-TIME argument.

Ah, that's interesting, thanks for pointing it out. I thought I had
checked to see if org-end-time-was-given had a value after org-read-date
gets called for a capture :time-prompt, but now I see that of course
that wouldn't work without the surrounding let binding...   

OK, so with your patch, the way is clear to stick this value in
org-capture-plist and make use of it when filling %T and friends in
templates. I'll see if I can come up with a patch to do that, and
report back.

-- 
Best,
Richard



Re: [PATCH] capture: Fix handling of time range for :time-prompt

2021-01-31 Thread Richard Lawrence
Dear Kyle and all,

Thanks for following up! Sorry it's taken me so long to reply.

Kyle Meyer  writes:

> Testing that against the cases in your initial report, I believe it
> leads to the same results as your patch, so here's a cleaned-up version.

I confirm that this cleaned up version works for me and gets the same
results for my test cases. I think it should be applied (modulo one
nitpick below). Are you willing to apply it, Kyle? I don't have commit
rights myself.

> -- >8 --
> Subject: [PATCH] capture: Fix handling of time range for :time-prompt
>
> * lisp/org-capture.el (org-capture-set-target-location): Bind
> org-end-time-was-given around the org-read-date call to get a return
> value that uses the start time rather than doing custom adjustment of
> the return value.
>
> If org-capture-set-target-location detects that the answer to
> org-read-date has a time range, it strips the end time from the answer
> and calls org-read-date-analyze again.  (org-read-date already calls
> it underneath.)  The regexp it uses, however, can easily match a date,
> leading to a bogus result.
>
> org-read-date-analyze is already capable of processing the time range
> in a way that matches org-capture-set-target-location's intent: when
> org-end-time-was-given is bound, org-read-date-analyze splits off the
> end value of the range and stores it in org-end-time-was-given.  Drop
> the custom logic and let org-read-date-analyze handle the range.
>
> Reported-by: Richard Lawrence 
> Ref: https://orgmode.org/list/87h7obh4ct.fsf@aquinas
> ---
>  lisp/org-capture.el | 35 +++
>  1 file changed, 15 insertions(+), 20 deletions(-)
>
> diff --git a/lisp/org-capture.el b/lisp/org-capture.el
> index f40f2b335..d7b69f228 100644
> --- a/lisp/org-capture.el
> +++ b/lisp/org-capture.el
> @@ -1025,28 +1025,23 @@ (defun org-capture-set-target-location ( 
> target)
>  (time-to-days org-overriding-default-time))
> ((or (org-capture-get :time-prompt)
>  (equal current-prefix-arg 1))
> -;; Prompt for date.
> -(let ((prompt-time (org-read-date
> -nil t nil "Date for tree entry:")))
> +   ;; Prompt for date.  Bind `org-end-time-was-given' so
> +   ;; that `org-read-date-analyze' handles the time range
> +   ;; case and returns `prompt-time' with the start value.
> +   (let* ((org-time-was-given nil)
> +  (org-end-time-was-given nil)
> +  (prompt-time (org-read-date
> + nil t nil "Date for tree entry:")))
>(org-capture-put
> :default-time
> -   (cond ((and (or (not (boundp 'org-time-was-given))
> -   (not org-time-was-given))
> -   (not (= (time-to-days prompt-time) (org-today
> -  ;; Use 00:00 when no time is given for another
> -  ;; date than today?
> -  (apply #'encode-time 0 0
> - org-extend-today-until
> - (cl-cdddr (decode-time prompt-time
> - ((string-match "\\([^ ]+\\)-[^ ]+[ ]+\\(.*\\)"
> -org-read-date-final-answer)
> -  ;; Replace any time range by its start.
> -  (apply #'encode-time
> - (org-read-date-analyze
> -  (replace-match "\\1 \\2" nil nil
> - org-read-date-final-answer)
> -  prompt-time (decode-time prompt-time
> - (t prompt-time)))
> +  (if (and (or (not org-time-was-given))

Nitpick: (or (not org-time-was-given)) is equivalent to (not org-time-was-given)

> +   (not (= (time-to-days prompt-time) (org-today
> +  ;; Use 00:00 when no time is given for another
> +  ;; date than today?
> +  (apply #'encode-time 0 0
> + org-extend-today-until
> + (cl-cdddr (decode-time prompt-time)))
> +prompt-time))
>(time-to-days prompt-time)))
> (t
>  ;; Current date, possibly corrected for late night
>
> base-commit: 9e8215f4a5df7d03ac787da78d28f69a4c18e7d3

As for this:

> The original change came in b61ff117b (org-capture.el:
> Set a correct time value with file+datetree+prompt, 2012-09-24), and I
> believe the related thread is
>

Re: clock-table and hooking that into org-capture file+olp+datetree

2021-01-30 Thread Richard Lawrence
Hi Christopher and all,

"Christopher Causer"  writes:

>> Note that org-datetree-find-date has a slightly annoying interface, in
>> that you need to provide a list of three integers representing a
>> calendar date. 
>
> Yes, that is a little awkward. What I did think of using was substrings to 
> extract the date from the picker interface.
>
> #+BEGIN_SRC emacs-lisp
> (defun org-date-picker-to-list ()
>   (let* ((date-string (org-read-date))
>(year (substring date-string 0 4))
>(month (substring date-string 5 7))
>(day (substring date-string 8 10)))
> (mapcar 'string-to-number (list month day year
> #+END_SRC
>
> Does that look sensible? 

As sensible as anything :) 

I recently added my version, using decode-time and nth, as a helper
function to my .emacs. I feel sure that it must already exist
*somewhere* in Emacs but I searched quite a while for it and didn't find
it. I will use this again below:

#+begin_src emacs-lisp 
  (defun time-as-calendar-date (time)
"Convert time in Emacs' time format to a calendar date list (MONTH DAY 
YEAR)"
(let ((parsed-time (decode-time time)))
  (list
   (nth 4 parsed-time)
   (nth 3 parsed-time)
   (nth 5 parsed-time
#+end_src

> My two next things to tackle are 
>
> 1. A hook to run the function when I run org-capture.

Here you might find it useful to grab the value of the :default-time key
from org-capture-plist (which should contain either the time you entered
at the date prompt during capture, or the current time, in Emacs' time
format). Then, using the functions above, you can say something like
this in your hook:

#+being_src emacs-lisp
(let* ((default-time (plist-get org-capture-plist :default-time))
   (date (time-as-calendar-date default-time)))
  (org-update-clocktable-on-date date))
#+end_src

> 2. Changing the org-clock-report options in your function above, but not the 
> defaults.
>
> For the second point, is there some trick to swap a global variable for the 
> run of a function? The variable in this case would be org-clocktable-defaults.

Emacs Lisp has dynamic scope by default, which makes this is pretty easy
in general: just set the value you want to use in a let form around the
code that uses this variable, like:

(let ((org-clocktable-defaults your-custom-value-here))
  ...)

> Thank you so much Richard. You've probably saved me days of going through the 
> org-mode documentation and source. 

No problem! That's what this list is for. I recently spent a fair amount
of time digging through the datetree stuff myself, so I was glad to have
a chance to share what I learned.

-- 
Best,
Richard



Re: clock-table and hooking that into org-capture file+olp+datetree

2021-01-30 Thread Richard Lawrence
Hi Christopher,

"Christopher Causer"  writes:

> Hello everyone! Here's a reasonably easy (I think) question because I'm quite 
> new to Emacs and org-mode.
>
> I have an org-capture template using file+olp+datetree[1], which works great 
> at filing my thoughts for the day. Separately I know I can generate clock 
> tables[2] based on dynamic blocks to show me what I've been doing with my 
> time for any given period. What I'm struggling with is to glue parts of these 
> together to achieve the following:
>
> 1. I org-capture to a subheading of datetree. When it does so it either 
> creates or updates an org-clock-report just below the datetree header  (the 
> bit that says "2020-11-12 Thursday", for example.) I guess this would be the 
> parent of what I'm capturing.
>
> 2. For all my historical journal entries, if I could move point to a headline 
> with a date such as the example below and it would pull the date out and add 
> a clocktable below via an interactive function that would be my ideal. This 
> is less of a problem for me as I don't have much in the way of history in my 
> diary yet or my other org files.
>

If I understand right, what you need for both of these things is a
function to jump to a date in your diary datetree and update the
clocktable there. Right?

Some functions that will help with this:
- org-datetree-find-date-create
- org-narrow-to-subtree

So, something like this should get you started:

#+begin_src emacs-lisp
(defun org-update-clocktable-on-date (date)
  (save-excursion
;; open the file containing the datetree:
(find-file (concat org-directory "/diary.org"))
;; jump to the subtree for the given date:
;; note: date must look like (m d y) where all three values are integers
(org-datetree-find-date-create date)
;; narrow to the subtree for this date, so we don't update
;; any other clocktables
(org-narrow-to-subtree)
;; update the clock report, or create it if it doesn't exist
;; note: we pass a prefix argument to tell org-clock-report to
;; update the first clocktable it finds in the (narrowed) buffer
(org-clock-report t)
;; widen to the whole buffer again
(widen)))
#+end_src

Then you can call this function, providing the date, in different
contexts where you want to create or update the clocktable.

Note that org-datetree-find-date has a slightly annoying interface, in
that you need to provide a list of three integers representing a
calendar date. One easy way to do that interactively is with
calendar-read-date, which prompts you for the year, month and day, so
you could say

(org-update-clocktable-on-date (calendar-read-date))

calendar-read-date is not as nice to use interactively as org-read-date,
but as far as I know, there is no easy way to get the calendar (m d y) format
out of its return value, which is either a string like "2021-01-30" or a
value in Emacs' internal time representation format.  But you can do
something like

(let*
;; prompt for the date and decode the resulting internal time as a list:
((decoded (decode-time (org-read-date nil t nil "Update on date:")))
;; unpack the date as a list (m d y) from the decoded time:
 (date (list (nth 4 decoded) ; month
 (nth 3 decoded) ; day
 (nth 5 decoded ; year

  (org-update-clocktable-on-date date))

Hope that helps get you to your next step!

-- 
Best,
Richard



[PATCH] incorrect timestamps with :time-prompt and datetrees

2021-01-12 Thread Richard Lawrence
Hi everyone,

Bumping this, since I forgot to put "PATCH" in the subject line before.

Richard Lawrence  writes:

> Here is a patch for this issue. It uses a narrower regex to match a time
> range. This regex requires time ranges to have ":MM" or an AM/PM
> specification in the end time, to prevent mangling strings that are
> interpreted as dates, like "11-12".
>
> This patch is a minimal change that gets the code working in the way
> that seems to have been intended, so it seems worth applying to maint.
>
> However, the way the code is intended to work doesn't seem right to me,
> because it simply throws away time range information at the time prompt.
> If you enter a time range like "13:00-14:00" at the time prompt, you
> will get a timestamp with "13:00" for the time when the %T template is
> expanded. (This is because org-capture-set-target-location uses the
> beginning of the entered time range to set :default-time, which must be
> an encoded time value, and there is no obvious way to set a time range.)
> This is a surprising contrast with the behavior of %^T, which preserves
> the time range information in the timestamp entered. But fixing this
> will be a larger change and possibly requires some discussion.

-- 
Best,
Richard

>From a6c223664aad6096943e236c9a51c30246e57669 Mon Sep 17 00:00:00 2001
From: Richard Lawrence 
Date: Wed, 6 Jan 2021 11:53:42 +0100
Subject: [PATCH] org-capture: fix expansion of %T when capturing to a datetree

* org-capture.el (org-capture-set-target-location): Use a narrower
regular expression to replace a time range by its start time when
setting :default-time, so that dates do not get mangled.

TINYCHANGE
---
 lisp/org-capture.el | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/lisp/org-capture.el b/lisp/org-capture.el
index f40f2b335..df0eccdbb 100644
--- a/lisp/org-capture.el
+++ b/lisp/org-capture.el
@@ -1038,12 +1038,12 @@ Store them in the capture property list."
 			 (apply #'encode-time 0 0
 org-extend-today-until
 (cl-cdddr (decode-time prompt-time
-			((string-match "\\([^ ]+\\)-[^ ]+[ ]+\\(.*\\)"
+			((string-match "\\(--?\\([012]?[0-9]\\)\\(\\(:[0-5][0-9]\\)\\|\\(am\\|AM\\|pm\\|PM\\)\\>\\)\\)\\(.*\\)"
    org-read-date-final-answer)
 			 ;; Replace any time range by its start.
 			 (apply #'encode-time
 (org-read-date-analyze
- (replace-match "\\1 \\2" nil nil
+ (replace-match "\\6" nil nil
 		org-read-date-final-answer)
  prompt-time (decode-time prompt-time
 			(t prompt-time)))
-- 
2.20.1



Re: Bug: incorrect timestamps with :time-prompt and datetrees

2021-01-06 Thread Richard Lawrence
Hi everyone,

Richard Lawrence  writes:

> I can now confirm that this is the problem. It looks like what is happening 
> here
> is that the regex is meant to match a time range, but ends up matching
> the date: thus a string like "12-31 13:00" gets mangled to "12 13:00"
> and sent into org-read-date-analyze.

> So I guess the solution is...a better regex is needed to cause this
> branch to fire only in the intended case, namely, a time range. 

Here is a patch for this issue. It uses a narrower regex to match a time
range. This regex requires time ranges to have ":MM" or an AM/PM
specification in the end time, to prevent mangling strings that are
interpreted as dates, like "11-12".

This patch is a minimal change that gets the code working in the way
that seems to have been intended, so it seems worth applying to maint.

However, the way the code is intended to work doesn't seem right to me,
because it simply throws away time range information at the time prompt.
If you enter a time range like "13:00-14:00" at the time prompt, you
will get a timestamp with "13:00" for the time when the %T template is
expanded. (This is because org-capture-set-target-location uses the
beginning of the entered time range to set :default-time, which must be
an encoded time value, and there is no obvious way to set a time range.)
This is a surprising contrast with the behavior of %^T, which preserves
the time range information in the timestamp entered. But fixing this
will be a larger change and possibly requires some discussion.

-- 
Best,
Richard

>From a41c751f15488a8b0d48d6b1b1744dbbc003f9f0 Mon Sep 17 00:00:00 2001
From: Richard Lawrence 
Date: Wed, 6 Jan 2021 11:53:42 +0100
Subject: [PATCH] org-capture: fix expansion of %T when capturing to a datetree

* org-capture.el (org-capture-set-target-location): Use a narrower
regular expression to replace a time range by its start time when
setting :default-time, so that dates do not get mangled.

TINYCHANGE
---
 lisp/org-capture.el | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/lisp/org-capture.el b/lisp/org-capture.el
index f40f2b335..df0eccdbb 100644
--- a/lisp/org-capture.el
+++ b/lisp/org-capture.el
@@ -1038,12 +1038,12 @@ Store them in the capture property list."
 			 (apply #'encode-time 0 0
 org-extend-today-until
 (cl-cdddr (decode-time prompt-time
-			((string-match "\\([^ ]+\\)-[^ ]+[ ]+\\(.*\\)"
+			((string-match "\\(--?\\([012]?[0-9]\\)\\(\\(:[0-5][0-9]\\)\\|\\(am\\|AM\\|pm\\|PM\\)\\>\\)\\)\\(.*\\)"
    org-read-date-final-answer)
 			 ;; Replace any time range by its start.
 			 (apply #'encode-time
 (org-read-date-analyze
- (replace-match "\\1 \\2" nil nil
+ (replace-match "\\6" nil nil
 		org-read-date-final-answer)
  prompt-time (decode-time prompt-time
 			(t prompt-time)))
-- 
2.20.1



Re: Bug: incorrect timestamps with :time-prompt and datetrees

2020-12-24 Thread Richard Lawrence
Richard Lawrence  writes:

> This is org 9.4 running from maint (commit ab00524fc). I spent a while
> stepping through org-capture and org-read-date but haven't found the
> problem yet. I suspect this snippet from a cond form in the middle of
> org-capture-set-target-location:
>
> #+begin_src
> ((string-match "\\([^ ]+\\)-[^ ]+[ ]+\\(.*\\)"
> org-read-date-final-answer)
> ;; Replace any time range by its start.
> (apply #'encode-time
> (org-read-date-analyze
> ;; it looks to me like this is maybe sending the wrong value
> ;; into org-read-date-analyze: 
> (replace-match "\\1 \\2" nil nil  
>org-read-date-final-answer)
>
> #+end_src

I can now confirm that this is the problem. It looks like what is happening here
is that the regex is meant to match a time range, but ends up matching
the date: thus a string like "12-31 13:00" gets mangled to "12 13:00"
and sent into org-read-date-analyze.  If I comment out this branch of
the cond form, and enter "12-31 13:00" at the org-read-date-prompt, then
both the timestamp and the location in the datetree are correct.

So I guess the solution is...a better regex is needed to cause this
branch to fire only in the intended case, namely, a time range. Should
it be checking for "am"/"pm" or ":" or similar? Otherwise it seems
impossible to distinguish a date specification like "11-12" from a time
range. 

-- 
Best,
Richard



Bug: incorrect timestamps with :time-prompt and datetrees

2020-12-24 Thread Richard Lawrence
Hi all,

I ran into a subtle bug yesterday. Basically, when using org-capture to
capture

  - an entry into a datetree,
  - on a date other than today (using :time-prompt in org-capture-templates)
  - with a capture template that inserts a timestamp (%T)

then I get incorrect results for either the timestamp or the location in
the datetree, depending on how I enter the date.

Here is a minimal working example:

#+begin_src emacs-lisp
  (setq org-capture-templates
  `(
("d" "Diary")
("da" "Appointment/Event" entry
 (file+olp+datetree "/tmp/diary.org")
 " %^{Description}\n %T"
 :time-prompt t)))
#+end_src
Notice that the template contains %T, to expand to a timestamp with a
time, and also that the capture target is a datetree and :time-prompt is
true. My understanding is that this should both file the entry to the
entered date in the datetree, and fill the %T template with a timestamp
for the entered date (and time, if any).
  
Here are the results from running a few captures with this setup on
Dec 24 around 11AM. I tried various ways of scheduling an event on Dec 31
at 13:00; what I entered into the prompt of org-read-date is shown in
the headline:

#+begin_example

* 2020

** 2020-12 December

*** 2020-12-24 Thursday
 Entered "31-12" 
 <2020-12-31 Thu 11:06>

This gets filed to the wrong day in the datetree, but the date in the
timestamp is correct. The time is also correct (it's the current
time, since no time was entered).

 Entered "31-12 13:00" 
 <2020-12-31 Thu 13:00>

This gets filed to the wrong day in the datetree, but the date and time
in the timestamp are correct.
 
*** 2020-12-31 Thursday
 Entered "12-31" 
 <2020-12-31 Thu 00:00>

This gets filed to the correct day in the datetree and the date in the
timestamp is correct. The time is 00:00 because no time was entered.
(Why isn't the time 11:06, though, like in the first example?)

 Entered "12-31 13:00" 
 <2021-01-12 Tue 13:00>

This is the most problematic case: it gets filed to the correct day in
the datetree, but the date in the timestamp is incorrect!

 Entered "2020-12-31 13:00" 
 <2020-12-31 Thu 13:00>

If I enter the date in full ISO format, the location in the datetree and
the timestamp are both correct.

#+end_example

Possibly relevant here is the value of calendar-date-style, which is
'american for me. I tested briefly with 'european but it did not make a
difference for the "31-12" cases.

This is org 9.4 running from maint (commit ab00524fc). I spent a while
stepping through org-capture and org-read-date but haven't found the
problem yet. I suspect this snippet from a cond form in the middle of
org-capture-set-target-location:

#+begin_src
((string-match "\\([^ ]+\\)-[^ ]+[ ]+\\(.*\\)"
org-read-date-final-answer)
;; Replace any time range by its start.
(apply #'encode-time
(org-read-date-analyze
;; it looks to me like this is maybe sending the wrong value
;; into org-read-date-analyze: 
(replace-match "\\1 \\2" nil nil  
   org-read-date-final-answer)

#+end_src

Will report here if I find out more exactly.
Happy holidays, everyone!

-- 
Best,
Richard



Re: typos in doc.org

2020-11-12 Thread Richard Lawrence
hj-orgmod...@hj.proberto.com writes:

> https://www.orgmode.org/worg/sources/doc.org :
>
> availables -> available ,  uesr -> user :

Fixed - thanks for the report!

-- 
Best,
Richard



Re: Bug: unexpected new page started after bullet line [9.1.9 (release_9.1.9-65-g5e4542 @ /usr/share/emacs/26.3/lisp/org/)]

2020-10-10 Thread Richard Lawrence
Hi Rob,

Rob Sargent  writes:

> This construct causes a new page to be started
> * head
> ** subhead
> - bullet

... in which export backend? You say you are using LibreOffice, but you
provided an example Org file with many LaTeX-related options. Do you see
the page break in a LibreOffice document? in a LaTeX-compiled PDF? both?

If in LibreOffice, do you see the same behavior if you don't have all
the LaTeX-related options in place?

FWIW, I tried exporting this construct to both LaTeX and ODT *without*
the LaTeX options on Org 9.4 and do not see the page breaks you are
seeing.

-- 
Best,
Richard



Re: Customize resheduling behaviour

2020-09-30 Thread Richard Lawrence
Hi Stefan,

Stefan Huchler  writes:

> I have a task sheduled like that:
>
> * TODO  Order from a delivery service
>   SCHEDULED: <2020-10-06 Di .+2w>
>
> ...
> Is there a way to reshedule it, if it fails 1 week later and if I
> complete it, reshedule 2 weeks later?

I don't think there's any way to do this automatically when you change
the TODO state of the entry.

The simplest way I can think of is to do C-c C-s +7 (i.e., call
org-schedule and give a new date a week later at the prompt). If doing
this on a different day than the scheduled day, you'll need to change
the interval you enter (e.g., if you missed it by 2 days, enter +5) or
type in the new date manually.

Another option is C-c C-x c (i.e., org-clone-subtree-with-time-shift).
Clone the tree one time, with a shift of one week, then delete the old
tree. The advantage of this option is that the time shift will be
relative to the old scheduled/deadline date, not the date on which you
run the command, so you can always say +1w.

Hope that helps!

-- 
Best,
Richard



Re: Bug: Math mode doesn't work if followed by a dash [9.4 (nil @ /home/gutin/.emacs.d/.local/straight/build/org-mode/)]

2020-09-29 Thread Richard Lawrence
"Berry, Charles"  writes:

> The case Gutin describes conforms to the documentation, viz. `$x\beta$-` 
> should produce math mode LaTeX as I read the next paragraph. 
>
> From (info "(org) LaTeX fragments"):
>
>• Text within the usual LaTeX math delimiters.  To avoid conflicts
>  with currency specifications, single ‘$’ characters are only
>  recognized as math delimiters if the enclosed text contains at most
>  two line breaks, is directly attached to the ‘$’ characters with no
>  whitespace in between, and if the closing ‘$’ is followed by
>  whitespace, punctuation or a dash.

Hmm, good point.

It looks to me like the relevant function is
org-element-latex-fragment-parser, and the code for that hasn't changed
much in several years (last change was 2017). (Also, I was wrong,
parsing latex fragments is not done with just a regexp.) I can confirm
that this function parses "$foo" in "$foo$ bar" as a latex fragment, but
not in "$foo$-bar"

Not sure if the problem here is the code or the documentation. Perhaps
the documentation should be updated to reflect the current behavior?

-- 
Best,
Richard



Re: Bug: Math mode doesn't work if followed by a dash [9.4 (nil @ /home/gutin/.emacs.d/.local/straight/build/org-mode/)]

2020-09-28 Thread Richard Lawrence
Hi Gutin,

gutin  writes:

> What I meant is that if you type
>
>   $*$-algebra
>
> and hit C-c C-x C-l, then the "$*$" doesn't get replaced with a
> mathematical image. A similar problem happens when you export to Latex:
> The dollar signs get escaped.

I believe this is intentional. There are too many other uses of dollar
signs that people might want to use in Org documents that might be
broken if Org treated them as math mode delimiters; so the regexp that
matches math mode delimiters is deliberately limited in scope.

The recommended workaround is to use the \(...\) style of delimiter for
math mode instead, which is much less ambiguous. Here is a function that
makes this easy, by inserting these delimiters when you type "$". It's
been posted on this mailing list before, and I use a version of it
myself; it's very useful!

   ;; from Nicolas Richard 
   ;; Date: Fri, 8 Mar 2013 16:23:02 +0100
   ;; Message-ID: <87vc913oh5@yahoo.fr>
   (defun yf/org-electric-dollar nil
 "When called once, insert \\(\\) and leave point in between.
   When called twice, replace the previously inserted \\(\\) by one $."
  (interactive)
  (if (and (looking-at ")") (looking-back "("))
  (progn (delete-char 2)
 (delete-char -2)
 (insert "$"))
(insert "\\(\\)")
(backward-char 2)))
   (define-key org-mode-map (kbd "$") 'yf/org-electric-dollar

Hope that helps!

-- 
Best,
Richard



Re: Bug: Math mode doesn't work if followed by a dash [9.4 (nil @ /home/gutin/.emacs.d/.local/straight/build/org-mode/)]

2020-09-28 Thread Richard Lawrence
Hi Gutin,

gutin  writes:

> If a pair of dollar signs is followed by a dash, then math mode doesn't
> work.

Can you say a bit more about what you are trying to do, what you are
expecting to happen, and what you are seeing instead?

Since you refer to math mode, I assume you are exporting to LaTeX. I
just tested this:

some text
$$-1$$
other text

and it exports fine to LaTeX on my machine (Emacs 26.1, Org 9.4).
What's broken for you?

-- 
Best,
Richard



Re: newline or no newline at end of capture: expected behavior

2020-09-27 Thread Richard Lawrence
Hi Samuel,

> On 9/23/20, Samuel Wales  wrote:
>> i have the same question for headline-only capture
>> buffers.  i.e. what is the expected result with and without trailing
>> newline.

Just to clarify: are you aware of the :empty-lines, :empty-lines-before
and :empty-lines-after keys that you can set in org-capture-templates?
Are your questions about the default values of these keys?

I just tried a simple setup in emacs -Q with Org 9.4, namely:

(setq org-capture-templates
  '(("s" "simple" entry (file+headline "~/org/notes.org" "Foo")
  "* %?" :empty-lines 1)))

and Org seems to correctly respect the value of these keys when they are
specified explicitly. That is, with :empty-lines 0, no newlines are
added; with :empty-lines 1, a blank line appears on each side of the
captured entry; with :empty-lines-after 1, a blank line appears after,
but not before, the new entry; and so on.

Can you use these keys to get the behavior you are looking for? Or is
your question about something else?

-- 
Best,
Richard



Re: Get Grades Done: the joys of Org's simple power

2020-06-19 Thread Richard Lawrence
Hi Devin,

Devin Prater  writes:

> Ah, I’m using Safari on MacOS 10.15. I can try with Chrome as well, though.

Did it work in Chrome, and/or when you moved the script to the end of
the file?

For what it's worth, here is a new version that should work better
regardless of where it's placed in the document.  The only change is
that the code which replaces the text representation of the checkboxes
with HTML checkboxes has been wrapped in a function that fires on the
"DOMContentLoaded" event.

Hope that's helpful!

-- 
Best,
Richard

#+begin_export html

  function updateCookiesIn(div) {
  const headline = div.querySelector("h1, h2, h3, h4, h5, h6");
  if (!headline) return;
  const cookies = Array.from(headline.querySelectorAll("code"))
.filter(c => c.innerText.startsWith("[") && 
c.innerText.endsWith("]"));
  const fracCookies = cookies.filter(c => c.innerText.includes("/"));
  const pctCookies = cookies.filter(c => c.innerText.includes("%"));

  // The ugly query strings here restrict the selection to checkboxes at 
*this* level of the hierarchy
  const allTasks = div.querySelectorAll(`#${div.id} > div > ul 
input[type=checkbox], #${div.id} > div > ol input[type=checkbox]`);
  const completedTasks = div.querySelectorAll(`#${div.id} > div > ul 
input[type=checkbox]:checked, #${div.id} > div > ol 
input[type=checkbox]:checked`);

  const newFrac = `[${completedTasks.length}/${allTasks.length}]`;
  const newPctText = allTasks.length
? (100 * completedTasks.length / allTasks.length).toFixed(0)
: "100"; // Org shows 100% for a cookie when there are no 
checkboxes 
  const newPct = `[${newPctText}%]`;

  fracCookies.forEach(c => c.innerText = newFrac);
  pctCookies.forEach(c => c.innerText = newPct);
  }

  function replaceWithCheckbox(code) {
  const isChecked = code.innerText.includes("X");

  const checkbox = document.createElement("input");
  checkbox.setAttribute("type", "checkbox");
  if (isChecked) checkbox.setAttribute("checked", "checked");
  checkbox.onclick = function (e) {
  const container = findContainingSection(e.target);
  if (!container) return;
  updateCookiesIn(container);
  };

  code.replaceWith(checkbox);
  }

  function findContainingSection(el) {
  if (!el.parentElement) return null;

  const parent = el.parentElement;
  const classes = Array.from(parent.classList);
  if (classes.some(cl => cl.startsWith("outline") && 
!cl.startsWith("outline-text"))) {
  return parent;
  } else {
  return findContainingSection(parent);
  }
  }

document.addEventListener("DOMContentLoaded", function(event) { 
  const orgCheckboxes = document.querySelectorAll(".off > code, .on > code");
  orgCheckboxes.forEach(replaceWithCheckbox);
  
  const orgSections = document.querySelectorAll("div.outline-1, div.outline-2, 
div.outline-3, div.outline-4, div.outline-5, div.outline-6");
  orgSections.forEach(updateCookiesIn);
});


#+end_export



Re: Get Grades Done: the joys of Org's simple power

2020-06-14 Thread Richard Lawrence
Hi Devin and all,

Devin Prater  writes:

> Yeah, I was hoping to just have an HTML page or something that could
> be put on Github or just sent in an email attachment, where checking
> checkboxes would update the fraction cookie.

I hacked together a quick solution for this. You should be able to just
drop this Javascript into an Org file. When it is included in HTML that
has been exported via the standard Org exporter, it replaces the static
"[ ]" and "[X]" code elements with actual HTML checkboxes, and updates
any progress cookies in the relevant headlines when those boxes are
checked or unchecked.

I haven't tested it extensively, and the code can surely be improved,
but it works for the cases I could think to test.

#+begin_export html

  function updateCookiesIn(div) {
  const headline = div.querySelector("h1, h2, h3, h4, h5, h6");
  if (!headline) return;
  const cookies = Array.from(headline.querySelectorAll("code"))
.filter(c => c.innerText.startsWith("[") && 
c.innerText.endsWith("]"));
  const fracCookies = cookies.filter(c => c.innerText.includes("/"));
  const pctCookies = cookies.filter(c => c.innerText.includes("%"));

  // The ugly query strings here restrict the selection to checkboxes at 
*this* level of the hierarchy
  const allTasks = div.querySelectorAll(`#${div.id} > div > ul 
input[type=checkbox], #${div.id} > div > ol input[type=checkbox]`);
  const completedTasks = div.querySelectorAll(`#${div.id} > div > ul 
input[type=checkbox]:checked, #${div.id} > div > ol 
input[type=checkbox]:checked`);

  const newFrac = `[${completedTasks.length}/${allTasks.length}]`;
  const newPctText = allTasks.length
? (100 * completedTasks.length / allTasks.length).toFixed(0)
: "100"; // Org shows 100% for a cookie when there are no 
checkboxes 
  const newPct = `[${newPctText}%]`;

  fracCookies.forEach(c => c.innerText = newFrac);
  pctCookies.forEach(c => c.innerText = newPct);
  }

  function replaceWithCheckbox(code) {
  const isChecked = code.innerText.includes("X");

  const checkbox = document.createElement("input");
  checkbox.setAttribute("type", "checkbox");
  if (isChecked) checkbox.setAttribute("checked", "checked");
  checkbox.onclick = function (e) {
  const container = findContainingSection(e.target);
  if (!container) return;
  updateCookiesIn(container);
  };

  code.replaceWith(checkbox);
  }

  function findContainingSection(el) {
  if (!el.parentElement) return null;

  const parent = el.parentElement;
  const classes = Array.from(parent.classList);
  if (classes.some(cl => cl.startsWith("outline") && 
!cl.startsWith("outline-text"))) {
  return parent;
  } else {
  return findContainingSection(parent);
  }
  }

  const orgCheckboxes = document.querySelectorAll(".off > code, .on > code");
  orgCheckboxes.forEach(replaceWithCheckbox);
  
  const orgSections = document.querySelectorAll("div.outline-1, div.outline-2, 
div.outline-3, div.outline-4, div.outline-5, div.outline-6");
  orgSections.forEach(updateCookiesIn);

#+end_export

Hope that helps!

-- 
Best,
Richard



Re: Get Grades Done: the joys of Org's simple power

2020-06-14 Thread Richard Lawrence
Hi Devin,

Devin Prater  writes:

> I tried that on my file, but the checkboxes didn’t update. I’ll give you the 
> kind of file I’m working with:

OK, sorry about that!  Two things:

1) I see now that my code assumes the script block runs immediately, so
it doesn't work if the script runs before the document is completely
loaded. I'll fix that and send an update.

2) When I move the script to the end of your file, it works here for me
(on Firefox Nightly). But it could also be that there are browser
differences getting in the way; I used some newer Javascript features, I
think. What browser are you using?

-- 
Best,
Richard



Re: wip-cite status question and feedback

2020-05-02 Thread Richard Lawrence
Nicolas Goaziou  writes:

> I think there are really two paths here: either we only support the
> common denominator between all processors, like, e.g., Pandoc, or we
> handle every possible command, knowing that most of them will not be
> portable anyways.

Yes, I think that is the core issue: which way to do we want to go here?

My opinion has always been that it makes more sense to just support the
common denominator at the level of Org citation syntax, for two reasons:
(1) it makes implementing a good solution that will work for a lot of
cases much more feasible, and (2) anyone who really needs more than the
common denominator -- that is to say, anyone who needs BibLaTeX -- can
already write arbitrary LaTeX snippets directly in an Org document. Thus
the latter group doesn't really lose anything if the syntax only
supports the common denominator, while everyone else has a lot to gain
from an implementation of citation syntax that can be exported on other
backends.

On the other hand, this opinion is narrowly focused on the issue of how
citation syntax can be rendered into citations when exporting a
document. When I think about the other uses for the syntax that e.g.
John Kitchin has talked about in this thread, and that something like a
future org-ref could support, then I see that people who need to export
citations as BibLaTeX *would* still be missing out if they couldn't use
the citation syntax.

So, I think it is good if the syntax can support advanced BibLaTeX users
too, and it looks to me like the "cite/xxx" syntax can do that. I have
no objections to the syntax you've proposed, Nicolas.

I *do* think it's worth marking a clear distinction between citation
syntax that can vs. cannot be expected to export correctly on non-LaTeX
backends. It looks to me at the moment like that distinction will be
expressed as the difference between "cite" and "cite/xxx". For me,
that's a reason to make "cite/text" a special case and give it its own
syntax, since this is such an important and widespread use case, and CSL
supports it. But I don't feel that strongly about this. For me, it would
be fine if it's mentioned as a special case in the manual.

-- 
Best,
Richard



Re: wip-cite status question and feedback

2020-05-01 Thread Richard Lawrence
"Bruce D'Arcus"  writes:

>> > My understanding, though, is that org "cite" would default to your
>> > last example I quote above (in natibib, citep); that there's no need
>> > for a dedicated "cite/paren" command, either reserved or not.
>>
>> Not necessarily. "cite" means default value, whatever that is. It could,
>> for example, mean: "cite/text" for every citation, if that is what you
>> use the most. In that case, "cite/paren" is necessary, to override it
>> locally. It could also be, e.g., "cite/footnote", then both "cite/text"
>> and "cite/paren" could be of some use. That was suggested by Richard
>> Lawrence in this thread, if my memory serves me right.
>>
>> Does that make sense?
>
> I think so. I'll defer to Richard on this, since he was making this point.

Sorry to take so long to reply. The point I made earlier was that, as
far as I understand, the choice of CSL stylesheet is the main factor
determining how a given citation gets rendered into the output (assuming
you process citations with CSL). So yes, it makes sense to have "cite"
mean default value as determined by the choice of stylesheet.

I've been skimming the CSL documentation, and I'm realizing that I
actually don't have a very good understanding of how these different
types of commands would be represented at the level of a CSL processor.

Bruce, is it possible to have a CSL stylesheet that would be able to
accommodate both e.g. "cite/paren" and "cite/footnote" in the same
document? Can a stylesheet support an arbitrary numbers of different
citation types like this, and can a CSL processor choose among those
types based on their *names*?

If so, then I think Nicolas' proposal to have "cite" mean default and
make non-default citations available as "cite/xxx" makes sense
(especially with the other syntax supporting suppress-author, etc.).

If not, then the "cite/xxx" syntax makes less sense to me; it just sort
of looks like a different way of writing BibLaTeX commands, and will be
hard to support when LaTeX is not the output format. I would be hesitant
in that case to make "cite/xxx" the standard way to express "this
citation should be rendered in manner xxx, instead of the default".

Best,
Richard



Re: wip-cite status question and feedback

2020-04-19 Thread Richard Lawrence
"Bruce D'Arcus"  writes:

> I can't see that it's necessary to have a fourth, because I think the
> result of that would be this, which doesn't make any sense.
>
> 4.  "Doe blah blah {2017}"/"Doe blah blah {[3]}" ->
> author-in-text+suppress-author command
>
> Let us know what you think?

I think this could sometimes make sense. Granted, it wouldn't be very
often, but if e.g. you are citing something inside a wider parenthetical
remark, like:

  (Blah blah. However, Doe showed that not-blah; see her -@doe17.)

I can imagine that some style guides might forbid putting nested
parentheses in that position, so having a way to render "2017" instead
of "(2017)" would be useful.

Another case: I can imagine citation styles that use e.g. a work's title
(instead of its year) as the non-author identifier, in which case it
would often make sense to say things like

  Doe depicts blah in her -@doe17

as a way to output things like

  Doe depicts blah in her /Wondrous Novel/

Again, I don't know how important this is, or how widely used it would
be, but those are at least a couple of possibilities.

On the other hand, I notice that pandoc does not distinguish these
cases, at least with the default citation style; pandoc renders both
-@doe17 and [-@doe17] like "(2017)", so maybe it's not that important.

> ... notwithstanding that, I think Nicolas' latest proposed syntax
> would support this anyway.
>
> [citet:-@doe17]

Great. No objections from this corner, then!

-- 
Best,
Richard



Re: wip-cite status question and feedback

2020-04-18 Thread Richard Lawrence
Hi Bruce and all,

"Bruce D'Arcus"  writes:

> Just to align what you're saying and what I'm saying:
>
> I see three commands in the pandoc syntax: standard/parenthetical,
> author-in-text, and suppress-author; that look like so:
>
> [@doe17]
> @doe17
> -@doe17
>
> Implicit in what you wrote is the last one is not needed.
>
> The question, then: Is that what you're saying; we don't need suppress-author?

Ah, no, I didn't intend it like that. I am not very familiar with the
implementation details of pandoc-citeproc and wasn't aware that
suppress-author was a different type of citation command. I was
(vaguely) thinking of the third case as a "variant" of an in-text
citation type, rather than a separate type.

Actually, the Pandoc example you give seems to support this way of
thinking about it:  

> Doe, by contrast, found negative results [-@doe17].

That is a fourth case, right? "[-@doe17]" is not equivalent to "-@doe17"?

In other words, what we have here are two orthogonal distinctions:
parenthetical vs. in-text, and normal vs. author-suppressed. So, at
least on my funny way of counting ;), that's two citation "types", with
two "variants" within each of those types.

> one of the CSL implementers (Frank Bennett) figured out how to make
> the above example an author-in-text variant, so that you don't need
> suppress-author, and the entire sentence is the citation.
>
> He did this by adding an optional "infix" variable to the citation.
>
> So in that example, you would have:
>
> - command: "author-in-text"
> - citekey: "doe17"
> - infix: "by contrast, found negative results"
>
> This is arguably an edge case, but it does relate to the question of
> whether we need two (standard and author-in-text) or three commands
> (adding the suppress-author).
>
> One could make the reasonable argument (I think, though not everyone
> would agree) that the workaround for the above example is to use
> author-in-text command but restructure the sentence:
>
> @doe17, by contrast, found negative results.
>
> From that perspective, I guess we indeed need only two commands:
> standard (parenthetical) and author-in-text.

This way of writing the sentence seems less obvious to me than the
pandoc syntax. It also has the potential disadvantage that the choice
between rendering

"Doe (2017), by contrast, found negative results"

and

"Doe, by contrast, found negative results (2017)"

now has to be made at the level of the stylesheet instead of at the
level of the sentence where that citation appears. My instinct is that
this choice is informed by individual writing style and better made at
the level of the sentence. But you probably have a better sense than I
do of whether this is something that should at least sometimes be
controlled by the stylesheet. (Are there e.g. journals that always want
in-text citations to look like the latter case? I have no idea.)

In any case, if I'm right that this choice is usually better made at the
sentence level, then I think the syntax needs to support all four cases.
.The pandoc syntax does this, and I think the suppress-author variation
is probably needed often enough that we should have something similar.

-- 
Best,
Richard



Re: wip-cite status question and feedback

2020-04-18 Thread Richard Lawrence
Joost Kremers  writes:

> Good points. I guess what this boils down to is whether Org wants 
> to be like LaTeX, where simple things are doable and complicated 
> things possible, or Pandoc, where simple things are simple indeed 
> and complicated things essentially impossible.
>
> To clarify: in LaTeX (biblatex) you can mix footnote and in-text 
> citations in a single document, Pandoc doesn't allow that. 
> Pandoc's functionality is sufficient for a great majority of 
> cases, but if you want or need to go beyond it, things get very 
> difficult.

Right. The Pandoc syntax trades some of the flexibility of (Bib)LaTeX
for the ability to render the citations it *does* support in a whole
bunch of non-LaTeX formats.

I personally think this is a good tradeoff, and one I would like to see
Org adopt. In both Org and Pandoc, you can use embedded LaTeX if you
need it. If you need the full power of BibLaTeX citations, then you are
confined to LaTeX export anyway, so you might as well just use BibLaTeX
commands in your document. But if you fall into the great majority of
use cases, you can use specialized citation syntax, and thereby get
reasonable behavior in other export formats too.

> My suggestion would still be not to hard-code a limit on possible 
> citation commands. Org itself should probably just provide the 
> basics, but users and add-on packages should be allowed to define 
> more specific commands with readable names and there should be a 
> well-defined interface for doing so (just like users and packages 
> can add new link types, for example).

I agree with this. I see no problem with having an analogue of
org-add-link-type for citations, and I think it's reasonable to have the
syntax allow for such extensions, so that e.g.

[cite/my-custom-cite-type: ...]

can still be recognized by the parser as a citation, which extensions
can then give a semantics to. But I think there needs to be a clear
syntactic delimitation between citations that are expected to work "out
of the box" (which to me primarily means: exported correctly in the
built-in backends) vs. those that need some additional extension to
export correctly or support additional behavior (which doesn't need to
be available on all backends, and could e.g. support BibLaTeX-only
users).  Otherwise the problem of getting citations working is too big a
project.

Best,
Richard



Re: wip-cite status question and feedback

2020-04-18 Thread Richard Lawrence
Hi all,

Nice to see this issue being discussed again!

I don't have a lot to add and at the moment I don't have a lot of time
to contribute, but I wanted to make one point about this issue:

Joost Kremers  writes:

> On Mon, Apr 13 2020, Nicolas Goaziou wrote:
>> denis.maier.li...@mailbox.org writes:
>>> What about allowing something more verbose? Perhaps
>>> "cite-intext:" or "cite:intext:"?
> [...]
>>> The simple syntax is great for most cases, but if you want to
>>> support some of those not so common biblatex commands, this might be
>>> better.
>>
>> Alphanumeric suffix provides 62 combinations, which should hopefully
>> be enough for any citation back-end out there (I'm looking at you
>> biblatex). It's not terribly readable, tho, as you point out.
>
> 62 combinations might sound like a lot, but if you want your cite
> commands to be mnemonic, you'll run out of options much more quickly.

This came up in the discussion in 2015, too. So maybe this can help
avoid repeating a long discussion about this:

I think it is worth pointing out to Bib(La)TeX users that it is useful
to avoid a proliferation of citation commands in Org syntax. The syntax
discussed so far achieves this by "factoring out" formatting information
that BibLaTeX puts into the command into other parts of the syntax and
into the choice of citation stylesheet. For example, instead of having
\footnotecite and \parencite as separate commands, you can just have a
single cite command, and the choice of stylesheet determines whether
citations get formatted as footnotes or as in-text parenthetical
citations or as something else.

This obviously has the drawback that if you only have single citation
command, you only get to make the choice about formatting once for the
whole document (via the stylesheet). So, I think the relevant question
is: how many different basic citation types are needed *within a single
document*, keeping in mind that these basic types will be formatted in
different ways, depending on the choice of stylesheet?

My experience is that it's typically just two (e.g. parenthetical and
author-in-text), and my memory of the earlier conversation was that most
people agreed. This is also borne out in the Pandoc syntax. As long as
we have two basic types of citations, the finer points of formatting
them can be achieved via other syntax, including the choice of
stylesheet.

-- 
Best,
Richard



Re: Release 9.3

2019-12-03 Thread Richard Lawrence
Dear Bastien and all,

Bastien  writes:

> I've released Org 9.3.
> ...
> As usual, this is the work of Nicolas acting as a maintainer
> and of many other contributors!

Thank you, Bastien, Nicolas, and everyone else! I haven't been very
active on the list of late, but I am using and loving Org as much as
ever. I appreciate all your hard work, and I wanted to say so.  Thanks,
and congratulations on this milestone!

-- 
Best,
Richard



Re: mobile org

2019-11-19 Thread Richard Lawrence
Neil Jerram  writes:

> If org-web and organice are browser-based, why do they need syncing?  Could
> the server be your regular non-mobile Org machine?

Presumably yes, if you're willing to keep it online and run a WebDAV
server on it.

> On Tue, 19 Nov 2019 at 08:57, Diego Zamboni  wrote:

>> Looking at the github history, organice seems to be a fork of org-web, but
>> with more recent development.

Yes, I think I saw somewhere that it's a "friendly" fork.

-- 
Best,
Richard



Re: mobile org

2019-11-18 Thread Richard Lawrence
Jude DaShiell  writes:

> Will mobile org work in Android PI?  If not, any good alternatives beyond
> orgsly available?  I don't like to have to use dropbox and would like to
> plug a usb cord into my computer and synchtronize my orgmode files that
> way if possible.

I saw this recently, and haven't seen it mentioned on the list:

https://organice.200ok.ch/

Here's an intro video: https://invidio.us/watch?v=aQKc0hcFXCk

It's a browser-based interface rather than an app, so it should work
just about anywhere. It doesn't solve the syncing problem itself;
instead it allows you to sign in with Dropbox, Google Drive, or WebDAV.
Still not great if you're trying to avoid the proprietary services and
sync with a cable, but it looks like an interesting alternative to
Orgzly.

-- 
Best,
Richard



Re: included text

2019-11-04 Thread Richard Lawrence
"Fraga, Eric"  writes:

> Mind you, an alternative could be #+CALL-ing a src block
> that generates the text as output?

On these lines: I've been thinking recently that it would be great to be
able to tangle/export other kinds of blocks (quotes, examples, etc.) via
the same kind of mechanism that src blocks use.

This would be really useful for me for making e.g. handouts from the
file that contains notes for a talk or a paper.

I've done this in the past with src blocks containing Org source code.
So instead of writing

#+begin_quote
Yada yada
#+end_quote

I write

#+begin_src org :tangle handout.org
,#+begin_quote
Yada yada
,#+end_quote
#+end_src

and create the handout.org file via org-babel-tangle.

This works but it is pretty clunky, and you lose a lot of the benefit of
having the original quote block. For one thing, it no longer exports as
a quote from the original file, but as source code. I'd rather just
write:

#+begin_quote :tangle handout.org
Yada yada
#+end_quote

or even better:

#+name: yada-quote
#+begin_quote
Yada yada
#+end_quote

and then in handout.org, something like:

#+include: talk.org::#yada-quote

Is there anyone who would be interested in this besides just me and
Samuel? (Can this already be done somehow in a non-clunky way?? I find
John's suggestion intriguing but too clunky: it seems like I shouldn't
have to pass a block of text through a shell or Python interpreter just
to include it verbatim in another Org file!)

-- 
Best,
Richard



Re: [O] CUSTOM_ID vs ID

2019-08-01 Thread Richard Lawrence
Hi Nate,

Nathan Neff  writes:

> However, I usually *do* manually assign IDs (not CUSTOM_IDs) myself.
> The reason is so I can reasonably search for the ID within my org
> files and that the link ID makes some sense to me.
>
> For example, if I see a link to "id:keyboard_shortcuts" I can tell where
> it's going.

If this is what you want IDs for, then you probably want CUSTOM_ID.
There is built-in Org syntax for links based on CUSTOM_ID: write your
links like [[#some-custom-id]] or [[file:foo.org::#some-custom-id]].

And if you use CUSTOM_ID, then if you ever need ID later (e.g., for
something like org-caldav), you won't have a collision between the two
different properties.

-- 
Best,
Richard



Re: [O] Show weekday in daily agenda view

2019-05-15 Thread Richard Lawrence
Hi Johanna,

"Prof. Dr. Johanna May"  writes:

> thanks for pointing me at the variable org-agenda-format-date. There
> was  a line in my dotemacs that included a formatting without the
> weekday (something like %y-%w-%d).

Ah, ok, great!

> I could not figure out how the name-of-the-weekday format would be
> called. But for now I'm fine with returning to the default which gives
> me an English date including the weekday.

You probably want "%a" or "%A" somewhere in there, if you want a
locale-specific name of the weekday.  See the docstring for the
format-time-string function.

-- 
Best,
Richard



Re: [O] Show weekday in daily agenda view

2019-05-14 Thread Richard Lawrence
Hi Johanna,

johanna@th-koeln.de writes:

> maybe I did not search the right way in google alias startpage. But I
> could not figure out how to write next to the date e.g. 2000-04-01 the
> weekday, i.e. the specific day of the week. It would come very handy to
> have the date displayed as 2019-05-14 Tu and 2019-05-15 We and so on. Or
> even have it spelled out as 2019-05-14 Tuesday.
>
> Probably there is a way of setting this. How does it work?

Hmm.  My daily agenda (the default one) starts like this:

Day-agenda (W20):
Tuesday14 May 2019

and tasks, etc. are listed below that.  I don't think I've ever
customized this.  Are you seeing something different?  Or maybe you mean
something else by "agenda"?

If you're talking about Org's built-in agenda, you might want to take a
look at the `org-agenda-format-date' variable.  Mine is set to the
org-agenda-format-date-aligned function, which I assume is what outputs
the example above.

If you're just talking about timestamps in Org files, you might want to
look at the `org-time-stamp-custom-formats' and
`org-display-custom-times' variables.

Hope that helps!

-- 
Best,
Richard



Re: [O] Order of items after a headline in an Org outline?

2019-02-09 Thread Richard Lawrence
Hi David,

David Masterson  writes:

> Is there a documented requirement of the order of things in a Org
> outline?  

My understanding is that a lot of work in recent years has gone into
making Org's syntax more consistent, and this is an ongoing effort.  So,
sometimes things still work if they're not in the proper order, but I
wouldn't count on them remaining so.  Your best bet is just to use Org's
functions for inserting and modifying these things, e.g., org-schedule
to schedule something.

You might find this helpful:
https://orgmode.org/worg/dev/org-syntax.html

> For instance, do these things have to come directly after the
> headline and, if so, in what order?  Will Org get confused otherwise?

The correct order is:

> * SCHEDULED and DEADLINE
> * PROPERTY drawer

and then notes.  (Timestamps can occur anywhere within notes.)  e.g.:



* Sample Headline
  DEADLINE: <2019-02-11 Mon> SCHEDULED: <2019-02-10 Sun>
  :PROPERTIES:
  :FOO:  bar
  :END:

Here I am typing some notes, late in the evening on [2019-02-09 Sat]



Hope that helps!

-- 
Best,
Richard



Re: [O] Org-Publish of a PDF ??

2018-12-12 Thread Richard Lawrence
Hi David,

David Masterson  writes:

> When I publish my project, I find that my org files are first generated
> into tex and pdf files in directory1 and then the tex/pdf files are
> copied to directory2.  What I would like is for the tex/pdf files to be
> directly generated in directory2 with no "extras" in directory1.

I think you might be thinking about this the wrong way around.  I may be
wrong about this, as I'm not a heavy user of the publishing framework,
but here's my two cents.

As far as I understand it, the publishing framework is basically
designed around the idea of copying finished products to a desired
location (a directory, either locally or on a webserver).  It's built as
a layer on top of Org's exporters, not as a way to make those exporters
behave differently.  If you want to control the *build* process for the
PDF, as opposed to just where it finally ends up, you probably want to
set options for the latex exporter, not the publishing framework.

In particular, you might want something like:

#+EXPORT_FILE_NAME: builddir/file.tex

in the Org files you're exporting, where builddir is somewhere outside
of your directory1.  (I'm *pretty* sure that will work, but I haven't
tested it.)

Hope that helps!

-- 
Best,
Richard



Re: [O] “Fuzzy” times (“evening”, “morning”, “night”…)

2018-12-09 Thread Richard Lawrence
Hi Leo,

Leo Gaspard  writes:

> In the process of migrating all my self-organization to org-mode, I
> noticed there is something that cannot currently be encoded in
> timestamps: fuzzy times, where an appointment is made for “Dec 4, Tue,
> evening” but with the hours not yet fixed.
> ...
> To be perfectly honest, my ulterior motive here is to auto-generate
> tasks “Decide of an exact time for [task]” a few days before the
> timestamp. However, I haven't investigated yet whether that'd actually
> be doable and I'm still pretty new to the lisp/emacs/org-mode
> ecosystems, so this may be completely impossible.
>
> What do you think about this idea? Is there a better way to do what I'm
> trying to do with current tools?

What about just adding a tag (:evening:, :morning:, etc.) to represent a
fuzzy time, with a plain date stamp, like <2018-12-04>?

That would allow you to easily create a custom agenda view containing
just entries with fuzzy times (if they have a timestamp with just a
date, you can even filter for those that are coming up in the next
couple of days).  Then when it's convenient, you can look at this agenda
view and schedule things that are still fuzzy more precisely.

Hope that helps!

-- 
Best,
Richard



Re: [O] In LaTeX export, can I control what heading type a headline goes to?

2018-11-29 Thread Richard Lawrence
Hi Bill,

William Denton  writes:

> Is there a way, exporting as a book, to make Org skip "part" and make a 
> top-level Org headline turn into a chapter?  Is there a built-in way, or do I 
> need to make my own class in org-latex-classes that has the structure I want?

Another simple solution that doesn't require changing the default
configuration is: just don't use top level headlines in your document!
I don't think Org will care if your highest-level headlines start with "**".

That may or may not work for your particular situation, but it seems
like a good option to keep in mind!

-- 
Best,
Richard



Re: [O] [POLL] Should Org tempo be enabled by default? (expand templates thru e.g. "<s[TAB]")

2018-04-30 Thread Richard Lawrence

Jon Snader  writes:

I use the 

Re: [O] Should wip-cite branch be merged to master?

2018-04-25 Thread Richard Lawrence

Hi everyone,

I don't have too much to add to this, though I do want to thank 
everyone for their continued interest in citations, and the work 
that has been done!


Since the question of syntax has come up again, I guess I want to 
address one point that András made:


András Simonyi  writes:

From the citeproc-el/CSL point of view, the current syntax is 
perfect with the notable exception of the provided citation 
commands. Currently only `cite' and `(cite)' are supported, 
where the latter seems to be intended to provide the 
parenthetical version of a basic citation, e.g. in an 
author-date style `cite' would produce something like `Smith 
2018` while `(cite)' `(Smith 2018)'. Now I think that for 
author-date styles `cite' should produce the parenthetical 
version and that `(cite)' probably shouldn't be among the 
commands at all. The main reason is that most citation 
processors (biblatex, CSL processors etc.)  support not only 
author-date citation styles but footnote-based ones as well, and 
the concept of a `parenthetical citation' doesn't really make 
sense for the latter. 


I think this actually gets at one of the fundamental problems with 
citations, namely, they resist attempts to separate 'content' from 
'style'.  It is hard to insert citations into a document without 
making any assumptions about how they will be rendered by the 
citation processor.  And so it is hard to come up with a syntax 
that is general enough to represent widely different citation 
styles, in a way that makes those assumptions explicit in the 
syntax of the unprocessed citation, so that the content of the 
citation can be separated from how it will be rendered.  Thus we 
see the huge proliferation of different citation commands in 
e.g. BibLaTeX, natbib, and similar packages.  András' proposal 
would reintroduce some of the complexity of BibLaTeX citation 
commands at the level of Org syntax.


Maybe having multiple citation commands is ultimately the best way 
to go, but I want to point out that one of the goals of the 
original discussion to come up with an Org syntax that is simpler 
and does a better job of separating content and style.  Anyway, 
that was one of my goals in synthesizing people's various 
suggestions and needs into a concrete syntax proposal.  I think we 
should try to avoid proliferation of citation commands.


If that is a reasonable goal, then I think there is really just 
one distinction that deserves to be represented at the level of 
citation commands: the distinction between 'in text' and 
'parenthetical' citations, that is, the distinction between 
'cite:' and '(cite):' in the current syntax in wip-cite.  That 
brings me to this point:


A more abstract characterization which is applicable to all 
styles is that normally references are not part of the main 
text, they are set off either by brackets or in a note. Since 
this is the most frequent, basic form, I think this the one 
which should be produced by the `cite' syntax, that is, when 
used in normal text `cite' should produce something like `(Smith 
2018)' for author-date styles and a note with the reference for 
note styles.


I think András' abstract characterization is exactly right: the 
issue here is not really about whether the citation has brackets 
around it.  Instead, it is about whether the rendered citation is 
intended to be *read as part of the surrounding sentence* 
('in-text'), or whether it is *grammatically independent* of the 
main text ('parenthetical').  I disagree, however, that 
'parenthetical' citations are the normal or usual case; I think 
that depends completely on one's field, writing style, etc.  (In 
my dissertation, for example, in-text citations outnumber 
parenthetical citations roughly 3:2.) 

The 'in-text' vs. 'parenthetical'/'out-of-text' distinction is 
really the fundamental one that you can't abstract from as an 
author.  And it seems to me that apart from this decision, most of 
the other decisions about styling can handed off to the citation 
processor, via a choice of CSL file, at least for the out-of-text 
case: is it a note-based style, or a Harvard-like one? etc.


The in-text case is a little trickier because you need a way to 
represent the different kinds of data that might be intended to 
fit into the surrounding sentence (author name? year? both? just a 
number? etc.).  But there are many many possibilities here, and 
for that reason I think it is better to avoid making them 
primitive citation commands.  Instead we should just stick to one 
command for the in-text case, and have some extensible way to 
represent the data that should be rendered, e.g. maybe like


[cite: @Jones2018 :year]

or (as I think was proposed at one point):

[cite:author @Jones2018]

Again, maybe it's worth having some shortcuts here for the common 
cases, but I think in general we want to try to avoid 
proliferation of basic citation commands.  So for that reason I 
think we should just stick 

Re: [O] aligning images, html attributes ignored when exporting to html (longish, sorry)

2018-04-23 Thread Richard Lawrence

Hi Andreas,

Andreas Reuleaux  writes:

Leaving out the double quotes may be comfortable, but I am not 
sure if this was a good idea: makes it difficult to see which 
parts belong to the style, and where the alt attribute starts: 

 #+attr_html: :style float: right; border: 1px solid brown; :alt 
 foo  
this would have been easier to read I think 

  #+attr_html: :style "float: right; border: 1px solid brown;" 
  :alt "foo"


This would be a good place to use a CSS class.

In your CSS file (or between 

Re: [O] Org citations, CSL and citeproc-el

2018-01-10 Thread Richard Lawrence
Hi András and all: 

Sorry for the delay.  


András Simonyi  writes:

Another question is the syntax for specifying the bibliography 
to be used and, maybe, the location where the bibliography 
should be placed.  Org-ref (ab)uses links for these purposes as 
well. Was there a syntax proposal for these things too? 


As far as I know, there was not.  This was not settled because in 
order to settle it, you need to know a bit more about the backend. 
(Where is the citation data stored? do you have to get it from an 
external program? if so, which? etc. etc.)


I think there was some agreement that we can do this with option 
keywords.  There is an existing syntax for specifying 
bibliographies in contrib/ox-bibtex.el, which probably provides a 
good model:


#+BIBLIOGRAPHY: bibfilename stylename optional-options  When I was 
playing around with hooking up the citation syntax to an external 
citation processor, I was using something similar: I had keywords 
like


#+CSL_FILE: the-csl-style-file.csl +BIBDB: bibtex testdoc.bib 


in the header, and used

#+BIBLIOGRAPHY: here

to specify where the bibliography should be placed in the 
document.  That is obviously not the nicest solution, but the 
basic idea might be a good starting point. 



--
Best,
Richard



Re: [O] Org citations, CSL and citeproc-el

2018-01-05 Thread Richard Lawrence

Dear Simonyi,

Simonyi András  writes:

a few days ago I've released the first public version of 
citeproc-el (https://github.com/andras-simonyi/citeproc-el), a 
CSL 1.01 citation processor library for Emacs.


Wow!  I don't know if you are aware, but we had discussed the 
possibility of building something like this on the list a few 
years back.  At the time it seemed like a lot more work than 
anyone was willing to do, and so the effort stalled.  Thanks so 
much for working on this -- it looks like you've done a ton of 
work!


The resulting link syntax is rather cumbersome so I'd like to 
ask your opinion about introducing an alternative org-mode 
citation syntax that handles all of these elements.  One option 
would be to use something very similar to pandoc's citation 
syntax (which I tried to follow as much as possible in the cite 
link descriptions of citeproc-orgref). 


We had a (very) long discussion about implementing a new citation 
syntax for Org in the spring of 2015.  Most of it took place in 
February and March; see:


https://lists.gnu.org/archive/html/emacs-orgmode/2015-02/threads.html
https://lists.gnu.org/archive/html/emacs-orgmode/2015-03/threads.html 


And here is how I summarized the state of that discussion in June:

https://lists.gnu.org/archive/html/emacs-orgmode/2015-06/msg00426.html

The upshot was that we came to agreement on quite a few points 
about what citation syntax should look like, and those points have 
been implemented in Org syntax in the wip-cite branch of the Org 
repo.  (This branch is now long out of date and at the very least 
in need of a rebase onto current master, I suspect.)  It would be 
really great to get things moving again.  Using your code to 
provide citation processing during Org export for the syntax 
that's already been implemented would be the place to start.


There are other aspects of the syntax we agreed on that are not 
implemented yet; as I recall, the idea was to get a minimal 
agreeable subset working, and then add to it based on real-world 
experience and feedback.


A more general question I'd like to raise how (or whether) you 
see citeproc-el's (and CSL's) potential place in the org-mode 
ecosystem.  There are a lot of directions which the further 
development could take (BibLaTeX support, citeproc-YAML 
bibliographies, CSL editing and CSL extensions etc.) and I'd be 
grateful to receive your input on which ones I should focus on. 


Here's my two cents on this. From what I recall about the 
discussion (it's hard to believe it's already been almost three 
years...), I would focus on two things:


1) BibLaTeX support.  Most Org users who want citation support 
want it primarily for LaTeX export, and BibLaTeX provides a good 
model of what we should try to achieve in Org citation support.


2) org-bibtex support.  Org-bibtex is a library that represents 
citation data via Org's property syntax.  Some people use this and 
it would be great to have it integrated with the new citation 
syntax and export.  That would provide an end-to-end citation 
solution that is completely native to Org and Emacs. 

Again, thanks for jump-starting this effort!  I look forward to 
contributing to it this spring.


--
Best,
Richard



Re: [O] Properties lost when closing, scheduling, etc.

2017-10-10 Thread Richard Lawrence

Hi John,

John Goerzen  writes:

Unfortunately, org-mode only recognizes the PROPERTIES when they 
occur immediately after the headline.  So whenever I take an 
action on an item like this, the properties become invisible to 
org-mode.   


What version of Org are you running?  This sounds like old 
behavior (or maybe a problematic mix of new and old behavior). 
PROPERTIES drawer syntax changed (incompatibly) in 8.3; see 
http://orgmode.org/Changes_old.html.


If you are running an older version, may want to try upgrading and 
then running the org-repair-property-drawers function defined on 
that page. 


Hope that helps!

Best,
Richard



[O] Agenda filtering (take 2)

2017-10-02 Thread Richard Lawrence

Hi everyone,

About a year ago, I wrote: 


Is there a simple way for me to say to the agenda, "Show me all 
(and only) the NEXT tasks that are part of a project whose 
deadline is before (say) 2016-11-01"?  It seems like this 
should be possible with the built-in agenda but I can't quite 
figure it out.  The tricky thing is filtering by the deadline 
of the parent project, which might be several levels up. 


I'm still stuck on this, so I'd like to ask this question again. 
Bastien responded: 

I would add a category to each project, then use something along 
this: 

("N" "My important tasks" tags-todo 
 "CATEGORY={cat1\\|cat2}+TODO={NEXT}+DEADLINE<=\"<+3d>\"") 


But that doesn't seem to work for me.  The problem is that the 
tasks I want to list in the tags-todo search don't themselves have 
deadlines; only their parent projects do.


I have a file with projects that look like this:

** TODO Foo University 
  DEADLINE: <2017-11-01>

*** NEXT Determine application requirements for Foo U.
*** TODO Submit application

I'd like to see a list of just NEXT tasks in this file, sorted by 
the deadline of their parent projects.


I thought that property inheritance would be the way to do this. 
So I tried: 

   ("jn" "Job application NEXT tasks" tags-todo 
"+jobmarket+application+TODO={NEXT}" 
((org-use-property-inheritance t)) 
 (org-agenda-sorting-strategy '(deadline-up 


as well as

   ("jn" "Job application NEXT tasks" tags-todo 
"+jobmarket+application+TODO={NEXT}" 
((org-use-property-inheritance '("DEADLINE")) 
 (org-agenda-sorting-strategy '(deadline-up 

in org-agenda-custom-commands.  But neither seems to work.  Can 
anyone provide any insights here?  Is it possible for sub-tasks 
within a project to inherit their deadline? 


Many thanks as always!

Best,
Richard



Re: [O] Auto-filling in org-mode without affecting the headlines

2017-09-10 Thread Richard Lawrence

Hi Narendra,

Narendra Joshi  writes:

I would like to have auto-fill enabled in org-mode but would 
like to prevent headings from being auto-filled. Is something 
available in org-mode that would allow me to do this? 


I have used auto-fill-mode in Org buffers for quite a while now 
without any problems, and a quick test on my end shows that long 
headings don't get auto-filled.  Are you having problems with 
this?  If so, what version of Org and Emacs are you using?   -- 
Best,

Richard



Re: [O] [RFC] Remove Org Struct mode

2017-08-21 Thread Richard Lawrence

Nicolas Goaziou  writes:


I would like to remove Org Struct minor mode from Org code base.


Like others, I only use Org Struct mode to get some Org-like list
handling and such when I'm writing email, and as Nicolas observes, it
barely works.  So I'm OK with this, especially if it makes Org easier to
maintain!

--
Best,
Richard



Re: [O] LaTeX Export

2017-08-10 Thread Richard Lawrence

Hi Scott,

Scott Randby  writes:

I tried this in the properties associated with a subtree:  

:export_latex_header: 
\hypersetup{pdfauthor={foo},pdftitle={foo},pdfsubject={foo},pdfkeywords={foo},pdfproducer={foo},pdfcreator={foo}} 

Unfortunately, all the other LaTeX headers are unrecognized when 
I do this.  


If you put the subtree-specific hypersetup in each subtree's 
EXPORT_LATEX_HEADER_EXTRA property, and the common stuff in 
#+LATEX_HEADER lines, I think it should do what you want.  Org 
should combine these properties rather than using the subtree 
property to override the global ones. 


Hope that helps!

--
Best,
Richard



Re: [O] How to paginate Babel Lisp output?

2017-07-09 Thread Richard Lawrence

Hi Jean,

Jean Louis  writes:

I have a very simple lisp output within Org Mode file, and it 
delivers list of lists with 4-5 elements, and it displays nicely 
as a text. 


(Do you mean: as a table?) 

#+BEGIN_SRC lisp :exports results  (load 
#P"/home/data1/protected//lisp/load-some-files.lisp" :verbose 
nil :print nil) (sku-list-values-by-location :location 1369) 
#+END_SRC 

But when exporting LaTeX I get the result that the table is 
basically too long, even being printed over the page number and 
below, and does not even continue on the next page. 

Is there a way or customization to make it break pages 
automatically when there is long output from source blocks? 


If you are already getting your results as an Org table, try 
adding 


#+ATTR_LATEX: :environment longtable

to that table.  This will use the LaTeX longtable package and
environment to format the table, which should take care of breaking it
across pages.

Best,
Richard



Re: [O] Price/ list comparison

2017-02-01 Thread Richard Lawrence
Hi Eduardo,

"Eduardo V."  writes:

> I constantly buy diodes, resistors, etc. for projects and i mostly save the
> receipts.
>
> I want to know if there is an org-mode way (or any other software) that
> let's me input all the data form the receipts and tell me which components
> are cheaper in X store.
>
> A plus if it has a way of telling me how many components i have left.

You might want to check out Ledger: http://ledger-cli.org/

It is a system for doing accounting with plain text, and has lots of features 
that I think you would find useful, including tracking prices and commodities.  
You should definitely be able to get it to tell you the things you're looking 
for, once you enter your receipts.  Also, it has good integration with Emacs, 
and (to some extent) with Org. 
 
Best,
Richard



Re: [O] Getting "Chapter" before a heading number

2016-12-29 Thread Richard Lawrence
Hi Peter,

Peter Davis  writes:

> I'm preparing a proposed table of contents for a book, and I'd like the HTML 
> output to appear like:
>
> Chapter 1 Blah
>   1.1 blah
>   1.2 blah blah
>   1.3 blah blah blah
>
> Chapter 2 Blah Blah
>   2.1 blah
>   2.2 blah blah
>
> etc.
>
> Is there a way to get HTML output to do this? I'd like the "*" at the start 
> of a line to denote a new chapter, and nested headings
> to go inside. I haven't been able to find anything like this.
>

This might not be exactly what you want, but it's pretty trivial to do
this with CSS.  In your stylesheet, you want something like (untested):

span.section-number-2:before { content: "Chapter" }

Basically, this tells the browser to put the word "Chapter" before the
number of a first-level headline (which, in a default setup, is exported
as a span tag with class section-number-2).

If you'd rather that "Chapter" appears in the actual HTML, you probably
want to use a filter on headlines during export.

Hope that helps!
 
Best,
Richard



Re: [O] sending an email to an orgmode file?

2016-12-21 Thread Richard Lawrence
Hi Xebar,

Xebar Saram  writes:

> im looking for a simple solution that will allow me to send an email from
> my mobile phone and habr that email either be appended to a txt(org) file
> or perhaps another solution to get my mobile on the go notes onto my laptop
> orgmode file. i tried orgzly and mobile org but i would prefer a simple
> solution as to send an email
>
> anyone have any thoughts on this?

I'd say a lot depends on the mail setup you use on the machine where
you use Org.  Emacs-based mail clients will be easier to integrate with
Org.  But really you're just limited by how much effort you want to sink
into this, at least if you store your mail in some format that is easy
to read with various tools.

If you use Gnus, one solution (I think?) is Gnorb:

https://elpa.gnu.org/packages/gnorb.html

I don't actually use this or know much about it but other people on this
list can tell you more about it.

In my own setup, I've settled on just making it easy to linking to an
email from a new TODO entry.  This is a very general mechanism that
works well for me, even though it is not completely automatic.  I
describe my setup below.

I use the Emacs interface for notmuch as my mail client.  I integrate it
with Org via org-notmuch and Org capturing.  Basically, I have a key that
automates capturing an email as a TODO item, containing a link to the
relevant mail:

#+BEGIN_SRC elisp
;; in my Org setup:

(require 'org-notmuch)

(setq org-capture-templates
; ...
  `(("t" "Task, Note, Project, etc.")
("tm" "Mail" entry
 (file ,(concat org-directory "/refile.org"))
 (file ,(concat org-template-directory "/mail-with-link.txt")

;; in my notmuch setup:
  (define-key notmuch-search-mode-map "P"
(lambda ()
  "postpone message (remove inbox tag, create task to reply)"
  (interactive)
  (notmuch-search-tag '("-inbox" "-unread"))
  (notmuch-search-show-thread)
  (org-capture nil "tm")))
#+END_SRC

The mail-with-link.txt template is simple:

#+BEGIN_SRC org
* %a:MAIL:
  :PROPERTIES:
  :Entered: %U
  :END:
  %? 
#+END_SRC

I mostly use this to record emails that I need to *reply* to as TODO
items, so a link to the original mail is enough for me.  

This probably isn't quite what you're looking for, but with more effort
on the notmuch side, a setup like this could be used to extract the
content of the email into the Org capture.  And you could also automate
this by looping over the messages that match a certain notmuch query,
etc.  
 
HTH,
Richard



Re: [O] Citation syntax: now supported in Pandoc

2016-12-01 Thread Richard Lawrence
Hi Nicolas,

Nicolas Goaziou  writes:

> I rebased wip-cite onto master. I didn't test the branch but "make test"
> reports no problem.

Great, thanks!

I guess this does not include the preliminary work on the org-cite
library (from Aaron Ecay's wip-cite-awe branch), or the work I was doing
on interfacing with a CSL processor via JSON.  Both of those pieces of
work are in the branch in [1].

> I'd rather have some testing on the Org side. This is not a Pandoc-devel
> mailing list. Besides, we really need to move forward on this branch.

Sure, of course.  In order for there to be testing on the Org side,
though, we need the support for citation syntax that the wip-cite branch
provides to get hooked up to the various exporters.  This is what we
made a start on in the org-cite library, before some sticky design and
dependency issues interrupted the momentum.  Will anyone have time to
work on those soon?

Best,
Richard

[1] https://github.com/wyleyr/org-mode/commits/wip-cite-rebase



Re: [O] Citation syntax: now supported in Pandoc

2016-12-01 Thread Richard Lawrence
Hi Matt,

Matt Price  writes:

> Richard, is wip-cite up to date with recent git versions of org-mode?

Nope.  At least, I have not done any work on this, and I don't know of
anyone else who has either.  I suppose I should rebase it onto master
again, but it's hard to justify the effort unless someone is going to
actively work on it, because otherwise it will quickly get out of date.
(Unfortunately, I can't put much time into this, at least not in the
short term.)

> I'd like ot try it out but would rather not go back to a pre-9.0
> version or org, having done a certain amount of work to update to the
> new framework...

Well, I'd still recommend trying it with Pandoc (instead of trying to
get wip-cite to work).  If there are issues with Pandoc's Org reader
post-9.0, those should be filed as bugs against Pandoc.

Best,
Richard



Re: [O] latex export: can I add an optional argument to an item?

2016-12-01 Thread Richard Lawrence
Hi Alan,

Alan Schmitt  writes:

> I would like to generate the following latex export:
>
> \begin{itemize}
> \item[\Checkmark] foo
> \end{itemize}

It's not exactly pretty, but you can use a description list while
resetting it to the itemize environment, like:

#+ATTR_LATEX: :environment itemize  
  - \Checkmark :: foo

That produces:

\begin{itemize}
\item[{\Checkmark}] foo
\end{itemize}

At least, that works for me (I have not yet upgraded to Org 9.0;
so I don't know if this will continue to work, though I don't know of
any reason why it shouldn't). 

Hope that helps!

Best,
Richard



[O] Citation syntax: now supported in Pandoc

2016-11-20 Thread Richard Lawrence
Hi everyone,

With Org 9.0 out, I thought it might be a good time to revisit the issue
of citation syntax.  Not much has happened with this recently, but I do
have one bit of progress to report:

About six months ago, Albert Krewinkel, who maintains the support for Org
syntax in Pandoc [1], wrote to me about supporting the citation syntax
we discussed previously, and that is partially implemented in
the wip-cite branch.  You can see examples of the syntax at [2].

Albert dove in, and was able to quickly add support for the syntax to
Pandoc's Org reader.  This support has been merged into released
versions of Pandoc.  Albert actually announced this a few months ago on
Reddit [3], but it looks like he did not email this list.

I finally got a chance to check out his work yesterday.  I converted one
citation-heavy chapter of my dissertation to use the new syntax, and
tested out Pandoc's support, with a couple of different CSL stylesheets,
including a note-based style.  It works great.  Thanks, Albert, for your
hard work!

There are some minor issues that probably need more discussion after
real-world use (like whether/when a conjunction should be inserted
automatically before the last reference in a multi-cite citation).  But
as far as I can tell, Albert has correctly implemented everything we
actually discussed and agreed on.

In short, Pandoc now has support for Org citation syntax, and that
support is fully integrated with Pandoc's citation processing
capabilities.  So there is now at least one "real" (non-prototype)
solution for processing Org citations via CSL, with output to HTML,
LaTeX, ODT, .docx, and the other formats that Pandoc supports.

This makes it a lot easier for people to test out the syntax in the real
world, so I encourage you to do so, and report back on what you find!

Best,
Richard

[1] http://pandoc.org/
[2] https://github.com/wyleyr/org-citeproc/blob/master/tests/testdoc.org
[3] 
https://www.reddit.com/r/orgmode/comments/4tes4x/ann_pandoc_v1172_has_been_released_it_now/



[O] Agenda filtering

2016-10-24 Thread Richard Lawrence
Hi everyone,

I have an Org file with a lot of projects that look like this:

* TODO Project A
  DEADLINE: <2016-10-28 Fri>
** NEXT Subtask 1
** TODO Subtask 2

* TODO Project B
  DEADLINE: <2016-10-31 Mon>
** DONE Subtask 1
** NEXT Subtask 2

Some of these projects have deadlines far in the future, some near.  I'd
like to focus on just the NEXT tasks for projects with upcoming
deadlines.

Is there a simple way for me to say to the agenda, "Show me all (and
only) the NEXT tasks that are part of a project whose deadline is before
(say) 2016-11-01"?  It seems like this should be possible with the
built-in agenda but I can't quite figure it out.  The tricky thing is
filtering by the deadline of the parent project, which might be several
levels up.

Many thanks for your insights!

Best,
Richard



Re: [O] How to change agenda sorting order temporarily?

2016-05-30 Thread Richard Lawrence
Hi Martin,

Martin Beck  writes:

> I'm working with a lot of pre-defined agendas, but I sometimes would
> like to change the sorting order of one of those to different criteria
> (by time, alphanumerically, etc.).
> I did not find a way to do that in org-mode directly - did I overlook
> something, or do I really have to change the configuration of the
> whole agenda and rebuild it?

Did you find the variable `org-agenda-sorting-strategy'?  That offers a
lot of possibilities.

> Related problem:
> If I change the configuration of the agenda by changing the
> customizing options, those changes do only have an effect, if I kill
> the agenda buffer and launch the agenda again.
> It would be nice, if I just could rebuild the existing agenda with "r"
> or "g", but that does not seem to be sufficient?

I'm afraid I don't know the answer to this, but would also be interested
to hear what it is.
 
Best,
Richard



Re: [O] Annotating org exporters

2016-05-11 Thread Richard Lawrence
Hi Sebastian and all,

Sebastian Fischmeister  writes:

> I'm still undecided between a regex replace and org-entities. Is there
> a straightforward way to define own directives for orgmode to then
> support something like the following?
>
> #+LaTeX_EXPORT: "=>":"$\rightarrow$"
>
> or more generic
>
> #+EXPORT_EXPAND: latex:"=>":"$\rightarrow$"
>
> Then I could just make these expansions part of the standard header in
> my org files.

Well, there are macros, which do something very similar.  As far as I
know, they are the only way to achieve this kind of thing without some
Elisp.

You could do something like:

#+MACRO: => @@latex:$\rightarrow$@@

but that actually doesn't seem to work as is, I suspect because "=>" is
not an allowable macro name.  (It works fine with an alphabetic name
like "ARR".)  But even if you got it to work, you'd then have to type

Some text {{{=>}}} other text after the arrow

in your document, which sort of defeats the point of the nice, simple
"=>".  If it were me, I'd opt for the regex replacement, and avoid
typing all the braces.
 
Best,
Richard



Re: [O] Annotating org exporters

2016-05-10 Thread Richard Lawrence
Hi Sebasitan,

Sebastian Fischmeister  writes:

> Is there a simple way to build regex-based extensions to the exporters?
> For example, I would like to convert this string "=>" to $\rightarrow$
> when converting the document to latex.

There's an example of how to do something like this in the "Advanced
Configuration" section of the Export section in the manual.  Maybe you
could adapt it like this?

 (defun my-latex-filter-rightarrow (text backend info)
   (when (org-export-derived-backend-p backend 'latex)
 (replace-regexp-in-string "=>" "$\rightarrow$" text)))

 (add-to-list 'org-export-filter-plain-text-functions
  'my-latex-filter-rightarrow)


Hope that helps!

Best,
Richard



Re: [O] pandoc-style citations

2016-04-28 Thread Richard Lawrence
Hi Alex,

Alex Fenton  writes:

>> Export is where efforts stalled last year.
>
> That's understandable, given that, as you say, it's a complex problem 
> given the range of citation styles and output formats. It's still a 
> shame given the work that you (pl.) have put into integrating citations 
> into the org parser & element tree so they are first class objects.
>
> I don't know whether it's conceivable that the data structures and 
> parsing could be integrated into org, with the (presumably) relatively 
> easy latex output, which I suspect is the commonest use case, and then 
> with some kind of "adequate" output for other targets (html, text, odt) 
> - perhaps an output that would require further post-processing by a 
> third-party tool such as citeproc or pandoc.
>
> Latex-outputters would be better off and other targets no worse off than 
> present, and it might act as a spur to solve the other target formats 
> one by one way. But I can see that this has been to some extent 
> considered and can also see the arguments against.

Unfortunately, even the LaTeX case is not so easy unless you assume
users are managing the bibliography database, and the mapping of
citation keys into that database, themselves.  It's easy to map the Org
syntax to the LaTeX syntax if you can just pass the keys through, but
that isn't a great assumption to make.  And things are a lot more
complicated once you have to worry about looking up the keys somewhere,
generating an appropriate .bib file, etc. We will have to do this stuff
for other backends anyway, so it seems sensible to do it for LaTeX too,
but that's what makes the project so big (especially if we can't just
pick one database format to support).

Maybe you're right that a good first step would just be to generate
LaTeX syntax, and just let users manage the bibliography database in
whatever way works for them, without trying to interface with it.  This
could be done quickly -- indeed, I think it is already most of the way
done.  And as you say, that would go a long way toward meeting many
people's needs.

I hope I will have some time next week to look at this.  I'll report
back!

Best,
Richard



Re: [O] pandoc-style citations

2016-04-27 Thread Richard Lawrence
Hi Alex,

Alex Fenton  writes:

> I see that there were several extensive and fruitful discussions on this 
> list last year on citation syntax. There seemed to be a reasonable 
> degree of consensus that pandoc-style citation syntax was at the least a 
> good model.
>
> I'd like to know if there are any implementations out there of elisp to 
> parse pandoc citation syntax and turn it into latex \cites. My question 
> is not so much "when/if this will be in org mode" but rather whether 
> there's something I can drop in now (likely as a link type).

As far as Elisp implementations go, I know of no specific parser for
Pandoc citation syntax.  But there is support for a Pandoc-like syntax
(discussed in the threads you read) in the wip-cite branch of Org's
repository.  This provides support for multi-cite citations in Org
syntax, but it isn't hooked up to the export framework at all.

Export is where efforts stalled last year.  Aaron Ecay, Vaidheeswaran C,
and I all worked on different proof-of-concept implementations to hook
up the citation parser to a citation processor and the various
exporters.  This is a non-trivial problem, and it seems that not many
people have a lot of time to work on it (including me), so if you want
to help, that would be great!  Aaron's work is in the Org repository
(see the wip-cite-awe branch).  Vaidheeswaran's is elsewhere; I don't
know where exactly, but you can search the list for a link.  My own is
here, in the wip-cite-rebase branch:

https://github.com/wyleyr/org-mode/.

There is also Pandoc itself, which can read (some) Org syntax.
Depending on what your document looks like, you might have good luck
just using Pandoc to convert it to LaTeX.

> I have a lot of longish citations with multiple references each with 
> their own pre- and post- ("'blah blah blah @ref1, p.23, also @ref2, for 
> a contrary view see @ref3 pp148-152") that end up as \cites. However my 
> home-brew link solution, stuffing the multiple pre- and posts- with 
> separator into the link description is unwieldy - difficult to write, 
> hard to read and easy to get wrong or breaking output.

You may also want to look at John Kitchin's org-ref, which I believe
works similar to your homebrew link solution, but has a lot of features
and may provide a better interface for what you're trying to do:

https://github.com/jkitchin/org-ref
 
Best,
Richard



Re: [O] [OT] A new web browser‽

2016-04-10 Thread Richard Lawrence
Eric Abrahamsen  writes:

> I installed the eww-lnum package right away, as that provides the main
> functionality I liked from Conkeror: hit a key, and pick a link to do
> something with. Rather strangely, the KeySnail plugin for Firefox seems
> to do everything *but* this, which I thought was weird since, if you're
> going to control your browser through the keyboard, following links is
> generally the main thing you want to do. I've only had it for a few
> hours, though, so maybe I'm missing something.

IIUC, Firefox actually has this behavior out of the box: press ' (single
quote) and start typing the link text.  I was surprised and pleased to
learn about that when I first found it.  That may be why it isn't
included in KeySnail.
 
Best,
Richard



Re: [O] Export Subtree to LaTeX: no EXPORT_LATEX_CLASS_OPTIONS ?

2016-02-21 Thread Richard Lawrence
Hi Axel,

Axel Kielhorn  writes:

> I want to export a subtree with a columnview dynamic block in
> landscape orientation.  I don’t want just a rotated table, the whole
> page should be in landscape mode.  I may have to tweak the page layout
> using geometry as well, thus I probably need EXPORT_LATEX_HEADER.

> * Projektboard :noexport:
> :PROPERTIES:
> :EXPORT_FILE_NAME: Projektboard
> :END:
> #+EXPORT_LATEX_CLASS_OPTIONS: [landscape]
> #+EXPORT_TITLE: Projektboard
> ...

I think what you want to do is put the EXPORT_LATEX_CLASS_OPTIONS (and
the TITLE) as a property in the property drawer, like this:

* Projektboard :noexport:
:PROPERTIES:
:EXPORT_LATEX_CLASS_OPTIONS: [landscape]
:EXPORT_TITLE: Projektboard
:EXPORT_FILE_NAME: Projektboard
:END:

That has the intended effect when I test it here, and my setup is
similar to yours.  In general, when you just want to affect the export
options of a subtree, you use properties, rather than keywords.

Best,
Richard



Re: [O] Attach a Word doc in org mode file

2016-01-29 Thread Richard Lawrence
Hi Eliya,

(I'm sending my reply to the list; please use reply-all or similar so
your responses go to the list, too.)

Eliya Voldman  writes:

> I'm using Windows OS.
> I'm using Emacs package from Cygwin hence I open any file types e.g. .doc
> using %cygstart.exe file.docx

I don't know anything about Cygwin (or Windows, really), but this looks
strange to me.  What does cygstart.exe do?  I would think you would want
a command like

word.exe file.docx

Maybe Cygwin has another way of starting Word?

> I never used to open file from Emacs buffer and I don't know how to open it
> from Emacs itself.
> It looks that text file Org opens with no problem but other type of file
> should be opened differently ( I think so )

Yes, presumably.

> I checked C-h v org-file-apps but it does not look straightforward for me
> how to set it up to open Word type file.

If it's not working automatically, then you probably need to add a
clause like

("\\.docx\\'" . "word.exe %s")

But I don't know what the second part (the command) should be; I'm not
even sure that it's possible to tell Word to open a file from the
command line.  I don't use Word, but maybe someone else will know.

By the way, I'm not sure what your reasons for running Emacs from Cygwin
are, but you *might* have better luck if you just run Emacs for Windows.
If Emacs is not running inside Cygwin, the system "open" command might
automatically work on Word files.  This is just a guess.

> When I tried to use 'link' ... it worked and followed the link but opened
> not the Word file but rather directory with a bunch of .xml files hence I
> was not able to open that file.

That's because, if I understand correctly, a .docx file is really just a
.zip archive with a bunch of .xml files inside.  Emacs knows how to open
a .zip archive, so it's showing you the internal .xml files in the
.docx.  (Which you *could* theoretically edit by hand, using Emacs...)

But that's not what you want.  You want to launch Word, right?  Or do
you have another program that you want to use to read Word files?

> Well I understood from your reply that *only* plain text file could be
> opened in Org major mode ( and it worked for me)
> I tried to use Emacs to open Word file by using C-c C-o but it again opened
> me a list of .xml file and not the Word file.

Yes, same explanation: Emacs sees the .docx as a .zip archive containing
those .xml files.

> Hence the question is: how to open Word file from emacs or Org mode?

The next thing you need to figure out is how to launch Word from the
command line in your environment, I think.

Best,
Richard



Re: [O] Attach a Word doc in org mode file

2016-01-28 Thread Richard Lawrence
Hi Eliya,

Eliya Voldman  writes:

> How could I attach a Word document in my .org file? I could do it for plain
> text file like C-c C-a but not .doc type.
>
> In other words: attach Word doc is not a problem but the problem is to open
> it.

What OS are you on?  Windows?  OS X?

I don't know how exactly Org decides how to open files on your system,
but if you're having trouble, I guess you should start by looking up the
value of the org-file-apps variable.  Try:

C-h v org-file-apps

> Are there other options to point to Word file in my system other the
> attach?

You can use a link; see this section of the manual:

http://orgmode.org/manual/Hyperlinks.html#Hyperlinks

> BTW: is it possible to attach and then open in org mode files other that
> plain text?

It depends what you mean by "open in org mode".  The only files it makes
sense to open as Emacs buffers with Org as the major mode are plain text
files in Org mode format.  But you can open other types of files in
Emacs in other modes.  And from within an Org mode buffer, you should be
able to launch programs other than Emacs (such as Word) that can read
other kinds of files, for example by following a link with C-c C-o.

Hope that helps!

Best,
Richard



[O] Bug: [PATCH] Tiny timezone bug in ox-icalendar [8.3.3 (release_8.3.3-2-g6bc48c @ /home/rwl/src/org-mode/lisp/)]

2016-01-21 Thread Richard Lawrence
Hi all,

I finally got annoyed enough to debug a problem with the timezone field
in the .ics export of my agenda.

Turns out the problem was simple: org-icalendar--vcalendar was being
called with the wrong argument order in
org-icalendar-export-current-agenda, so the timezone was going into the
description field rather than the timezone field.

Here's a patch that fixes that.  Note that it should be applied to
maint.  I did not generate a commit against master, but I can if need
be.

Best,
Richard

>From 410ba46791e2f4cf470f8727298c3faa5b717e56 Mon Sep 17 00:00:00 2001
From: Richard Lawrence <richard.lawre...@berkeley.edu>
Date: Thu, 21 Jan 2016 17:28:58 -0800
Subject: [PATCH] ox-icalendar:  fix timezone export bug

* ox-icalendar.el (org-icalendar-export-current-agenda): Correct
  argument order in call to org-icalendar--vcalendar (timezone should be
  third, description fourth).

TINYCHANGE
---
 lisp/ox-icalendar.el | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/lisp/ox-icalendar.el b/lisp/ox-icalendar.el
index be2f3e6..a7cb4fb 100644
--- a/lisp/ox-icalendar.el
+++ b/lisp/ox-icalendar.el
@@ -909,8 +909,8 @@ This function assumes major mode for current buffer is
(org-icalendar--vcalendar
 	org-icalendar-combined-name
 	user-full-name
-	org-icalendar-combined-description
 	(or (org-string-nw-p org-icalendar-timezone) (cadr (current-time-zone)))
+	org-icalendar-combined-description
 	contents)))
 (run-hook-with-args 'org-icalendar-after-save-hook file)))
 
-- 
2.1.4



Re: [O] Can weekly agenda show children nodes?

2016-01-04 Thread Richard Lawrence
Hi Tomasz,

Tomasz Piotrowski  writes:

> I usually plan my activities for a single day without specifying time of
> day it should be accomplished, e.g.,
>
> * <2016-01-04 Mon>
> ** Item 1
> ** Item 2
>
> In agenda, children nodes of
>
> * <2016-01-04 Mon>
>
> are not shown, i.e., only * <2016-01-04 Mon> is displayed and is empty.
>
> Can I tweak agenda to show
>
> ** Item 1
> ** Item 2

I am not sure whether it is possible to do exactly what you asked; maybe
someone else will know.  But I wanted to suggest an alternative: rather
than using the headline hierarchy to represent your schedule, use
timestamps within the individual activities, like:

** Item 1
   <2016-01-04>
** Item 2
   <2016-01-04>

This will cause them to appear in the agenda on that date, as you
wanted.  It also has the advantage that rescheduling the individual
activities for a different day is as easy as changing the timestamp --
you don't have to move the headline around in your file, and can just
let the agenda take care of assembling your schedule for the day.

You may also find the SCHEDULED and DEADLINE keywords helpful; see the
"Deadlines and scheduling" section of the manual.

Hope that helps!

Best,
Richard



Re: [O] Org list source block

2016-01-03 Thread Richard Lawrence
Hi Jarmo,

Jarmo Hurri  writes:

> I am trying to define a checklist and then use the same list in my
> exported document in multiple locations. ...
> What is the correct solution to this?

Would an #+INCLUDE file work here?  See

http://orgmode.org/manual/Include-files.html#Include-files

Best,
Richard



Re: [O] LaTeX export with section number, name and page in internal links

2015-12-09 Thread Richard Lawrence
John Kitchin  writes:

>> I do something like this with custom link types.
> Aha! I am not the only one ;)
>
>>
>> First of all, have a look at the variable
>>
>> org-latex-prefer-user-labels
>
> Is this a new 8.3 variable? It doesn't seem to be in my 8.2.10 MELPA
> version.

Yes, I believe it is new in 8.3, although it originated in a patch I
wrote quite a while ago (February 2014?) when I needed this behavior.

> I like your idea. I think you could simplify it to just:
>
> (org-add-link-type
>  "sec"
>  (lambda (path)
>(org-open-link-from-string (format "[[#%s]]" path)))
>  ...

Ah, nice, that is way simpler.  Thanks!

Best,
Richard



Re: [O] LaTeX export with section number, name and page in internal links

2015-12-08 Thread Richard Lawrence
Hi Ilya,

Ilya  writes:

> I export my Org-Mode notes with internal links to LaTeX and I want it to
> look like this ''Section 1.1 [Section name], page 99'' (Like in the Org
> Manual). I use this construction:
>
> #+BEGIN_EXAMPLE
>  * Chapter 1
>  ** Section 1.1
> :PROPERTIES:
> :CUSTOM_ID: section-1
> :END:
>  * Chapter 2
>  ** Section 2.1
> I want reference to Section 1.1 from here (See #section-1)
> #+END_EXAMPLE
>
> But as a result I get only the number of the section ''1.1'', not the
> ''Section 1.1 [Section name], page 99''.
> What options I need to use?

I do something like this with custom link types.

First of all, have a look at the variable

org-latex-prefer-user-labels

if you haven't already.  Setting it will cause Org to use CUSTOM_ID
properties to generate labels, so you don't need to manually insert your
own.

I use the following bit of Elisp to define some link types for referring
to sections this way.  You could modify this to insert the LaTeX command
you're interested in (as opposed to just \ref{}).  With your example
above, you'd write something like

#+BEGIN_EXAMPLE
I want reference to Section 1.1 from here (See [[sec:section-1]]).
#+END_EXAMPLE

Here's the code:
#+BEGIN_SRC elisp
;; Link types for targeting sections, tables, etc.
;; These assume that headlines with CUSTOM_ID defined will export using
;; that value as their \label keys.
(defun org-find-headline-by-custom-id (prefix path)
  "Find a headline in the current buffer by CUSTOM_ID value PREFIX:PATH."
  (save-excursion
(goto-char (point-min))
 (and
  ; borrowed from org.el; there doesn't seem to be a function that searches
  ; for a headline with a specific property value
  (re-search-forward
   (concat "^[ \t]*:CUSTOM_ID:[ \t]+" prefix ":" path "[ \t]*$") nil t)
  (setq pos (match-beginning 0
   (if pos
   (progn
 (goto-char pos)
 (org-back-to-heading t))
 (message (format "Headline with CUSTOM_ID %s:%s not found." prefix path

(defun org-export-dissertation-link (prefix path desc format)
  "Export a link to a dissertation section, etc.

In LaTeX, the exported link will look like:
  DESC~\\ref{PREFIX:PATH}
"
(when (member format '(latex linguistics))
  (format "%s~\\ref{%s:%s}" desc prefix path)))

; Sections:
(org-add-link-type
 "sec"
 (lambda (path)
   (org-find-headline-by-custom-id "sec" path))
 (lambda (path desc format)
   (org-export-dissertation-link "sec" path (or desc "Section") format)))

; etc. etc.
#+END_SRC elisp

Best,
Richard

OpenPGP Key ID: CF6FA646
Fingerprint: 9969 43E1 CF6F A646

(See http://www.ocf.berkeley.edu/~rwl/encryption.html for more information.)



Re: [O] More questions about CSL and org-mode

2015-12-07 Thread Richard Lawrence
Hi John,

John Kitchin  writes:

> Thanks.
>
> Its an interesting jam. You want to have multiple outputs as a
> possibility, but there isn't a robust markup that readily works across
> all backends.

Yes, indeed.  

> On export the in-text citations are transformed to unique text blobs,
> e.g. uuids, and the document exported. The only important features of
> these blobs is that they do not get changed on export, and they are
> unique because we replace them later.
>
> The strings in the bibliography entry are "exported" to convert the
> org-markup to the output format. The in-text citations, expanded
> bibliography and style are sent to the citation processor, which outputs
> replacements and a formatted bibliography in the desired output format.
>
> Finally, you replace each uuid with the appropriate replacement, and
> insert the bibliography where it belongs. That should be the final
> document.

IIUC, the problem with this approach is that it will not work well when
the citation style is note-based rather than inline.  The main
motivation for going "back to Org" is that note-based styles require the
document structure to change as a result of citation processing: new
footnotes have to be inserted, and existing ones have to be renumbered.
That is relatively hard to do if the rest of the document is already in
the target format (except with LaTeX).  By doing citation processing
early in the export process and converting the results to Org, we can
rely on Org's footnote processing to handle this later in the export
process.

As far as I can see, if it weren't for note-based styles, this approach
would work fine.  (Indeed, it is pretty much what the existing org-cite
code does, except that the mapping between citations and their
replacements is done with Lisp data structures rather than via string
replacement in the output buffer.  I stopped work on that right about
the time I realized the existing approach wouldn't work very well with
note-based styles.)

But given the problem about nested formatting, going back to Org at the
level of text replacements doesn't work.  In other words: both of the
simple-minded approaches (process citations directly to text in the
target format, or process them to Org text, then let Org convert them to
the target format) face problems.

I think probably what we'll have to do to accommodate both note-based
styles and the possibility of nested formatting is to get the results of
citation processing in some unambiguous format like HTML or JSON, then
parse it, and then use the result to directly modify the parse tree for
the Org document before continuing the export process.  I can't see an
easier way...can anyone else?

Best,
Richard



Re: [O] More questions about CSL and org-mode

2015-12-06 Thread Richard Lawrence
Richard Lawrence <richard.lawre...@berkeley.edu> writes:

>> IIUC, the current aim is to get a citeproc that will do the following on
>> export:
>> 1. replace in-text citation syntax with org-formatted replacements
>> 2. Insert an org-formatted bibliography somewhere in the document
>> 3. proceed with org-to-something export, with built-in
>> exporters.
>
> That's basically my understanding too.  There is one snag with the
> "org-formatted replacement" plan, though, which I saw in a Zotero dev
> discussion yesterday.  

Here's the reference for that discussion, by the way:

https://groups.google.com/d/msg/zotero-dev/Bz_IenruxX4/24QWuyEIp_IJ

Best,
Richard

P.S.  John, thanks for your continued research on this.  I see that our
procrastination habits are on the same schedule. :)



Re: [O] More questions about CSL and org-mode

2015-12-06 Thread Richard Lawrence
Hi John,

John Kitchin  writes:

> Hi all,
>
> This is mostly for the people working on citations in org-mode.
>
> I have been reading about CSL more this weekend. IIRC, one of the
> reasons to develop the new citation syntax was to get the ability to
> have pre/post text in citations more conveniently than what is currently
> possible.

Yes, that is my understanding, too.

> I have not seen any possibility for this with CSL, however. Is my
> understanding correct? Is this a problem, or something partially handled
> by org-export and partially by a citeproc?

The CSL processors I've looked at support prefix and suffix text for
individual references within a citation.  See, for example, the
citeproc-js documentation:

http://gsl-nagoya-u.net/http/pub/citeproc-doc.html#citation-data-object

prefix, suffix, and some other fields are supported.  pandoc-citeproc
supports the same set of fields.

However, my understanding is that neither citeproc-js nor
pandoc-citeproc support a BibLaTeX-style "common" prefix/suffix that
belongs to the citation as a whole, rather than the individual
references within it, as is available in the multi-cite commands.  We
currently have support for such common prefixes/suffixes in Org syntax. 

My solution to this in my org-citeproc wrapper for pandoc-citeproc is to
prepend the common prefix to the prefix for the first reference in a
citation, and append the common suffix to the last reference.  This is
not a great solution, because it is not really defined what kind of
punctuation (if any) should separate the common prefix from the first
item's prefix, and so on.  But I figured that was not an important issue
to address until we actually have people making use of common prefix and
suffix syntax who are not exporting to LaTeX...

> IIUC, the current aim is to get a citeproc that will do the following on
> export:
> 1. replace in-text citation syntax with org-formatted replacements
> 2. Insert an org-formatted bibliography somewhere in the document
> 3. proceed with org-to-something export, with built-in
> exporters.

That's basically my understanding too.  There is one snag with the
"org-formatted replacement" plan, though, which I saw in a Zotero dev
discussion yesterday.  CSL processing might result in multiple levels of
formatting, e.g. nested italics like

Something with an internal Title

and that won't translate very well back to Org syntax in general:

/Something with an internal /Title//

The suggestion was to just use HTML output, and then parse the HTML to
get a data structure that could be directly rendered into HTML, LaTeX,
etc., which support nested italics just fine.  I think we could do this,
though maybe there's a better solution.  That is, we can take HTML from
the citation processor and go directly to org-element objects, without
producing and re-parsing citations in Org format.

> The current contenders for a citeproc are Zotero and Pandoc.
>
> Has anyone looked at https://pypi.python.org/pypi/citeproc-py/
> or https://github.com/inukshuk/citeproc-ruby
>
> The ruby one looks pretty advanced.

I haven't looked at them closely.  My impression was that the Python
version was quite incomplete; and unfortunately, I don't know Ruby, so I
would be the wrong person to evaluate it (or write code for it).

Best,
Richard



Re: [O] Citation processing via Zotero + zotxt

2015-12-03 Thread Richard Lawrence
Hi Matt and all,

Matt Lundin  writes:

> But for bibtex users, wouldn't there presumably have to be another
> zotero plugin that would allow for live, automated importing of bibtex
> into zotero? (If anyone knows whether such a plugin exists, please do
> let me know.)

Well, my hope is that this could be added to zotxt without much effort,
so we could still just depend on Zotero and zotxt.  The translation
capability already exists in Zotero; it's just a matter of exposing it
as an API, and I imagine that Erik would happily accept a patch to zotxt
that does so.

> A thought experiment... Do we need a fork of pandoc-citeproc?

(The name "org-citeproc" might be a bit misleading: it's not a fork of
pandoc-citeproc, but a small wrapper script that takes care of
marshalling data into and out of Pandoc, in a format that's easy to
generate and read from Emacs lisp.  But because it's written in Haskell,
building it still requires a full Haskell build chain, and distributing
it is non-trivial.)

> Or could we rather write an emacs-lisp wrapper that would feed
> citation data and a bibliography to pandoc and receive a string
> containing citations formatted in org syntax (one of pandoc's
> outputs). This could be done via an export filter, with all the
> necessary manipulation being done on the emacs/org side of things.

Yes, you're basically describing the approach that I eventually realized
org-citeproc should take: use the full capabilities of Pandoc to render
citations and bibliography in Org format, then re-parse these on the Org
side.  I did start to work on this, though I didn't finish and I'm not
sure if I pushed it to the public repo.

If we want to use pandoc-citeproc directly, instead of wrapping it with
something like org-citeproc, what we'd need to do is be able to
translate an Org document (or at least the citations within it) both to
and from pandoc-compatible JSON, since pandoc-citeproc reads and writes
in that format.  This can certainly be done in Emacs Lisp, and maybe it
would be worth doing, because it would mean that we'd gain a nice
serialization format for Org documents.

I am not opposed to this idea -- indeed, I kind of like it, which is why
I started work on org-citeproc in the first place.  Still, it would be a
non-trivial amount of work to develop this solution even to the point
that it can do what Zotero and zotxt can do right now.

> Javascript interpreters/engines are widely available for all platforms
> if we create a wrapper script around citeproc-js. Node itself is also
> easily available for most platforms. But we wouldn't need to set it up
> as a node server à la citeproc-node.

My concern here is with the wrapper script.  Yes, it's pretty easy to
install a javascript interpreter; but getting from there to the point
where you have a fully-working toolchain for processing citations from
Org mode is the problem.  What I think we should avoid is a process that
looks like:

1) Install node (or whatever interpreter)
2) Install citeproc-js and the wrapper script
3) Make sure the wrapper script is available as an executable that can
be called from Emacs
4) ...

Steps 2 and 3 are a bit much to ask people to do manually, and we don't
have the option of doing them through the system package manager.  They
could of course be automated by packaging up citeproc-js and a wrapper
script into a node package.  But again, that requires some work that no
one so far has volunteered to do; and anyway, it doesn't get away from
the worry about having to install packages/plugins from heterogeneous
sources.  (Also, we'd have to develop the wrapper script, and our own
way of translating items stored in BibTeX format into a format
compatible with citeproc-js.  This means duplicating efforts that have
already been put into both pandoc-citeproc and Zotero.)

It's a question of where to focus the limited resources we've got.  My
impression is that going with the combination of Zotero and zotxt will
represent the least amount of effort to get citations working on
non-LaTeX backends, for both Org developers and users.  I understand
that it won't be ideal for everyone, and there is time enough later on
to develop an alternative toolchain if that would better suit the needs
of people who don't want to depend on a GUI, etc.  I fully support that.
But until more people have time to work on this, it seems to me that
Zotero and zotxt represent the most practical path forward.
 
Best,
Richard



Re: [O] Fwd: Citation processing via Zotero + zotxt

2015-12-02 Thread Richard Lawrence
Hi Matt and all,

Matt Lundin  writes:

> One question (based on complete ignorance of either zotero or zotxt) is
> whether those of us who maintain bibtex databases solely in emacs would
> have to interact with the zotero GUI.

My goal would be that you wouldn't need to interact with it at all.
(You would need to have Firefox running in the background, but that's it
-- no need to actually interact with the window as a user.)

> Let's say I have a bib file and an org document containing
> citations. Would everything place transparently in the background
> without ever having to interact with zotero? Can zotero query a bib
> file without requiring the user manually to import/export the
> database?

I do not know of a way to do this at the moment.  But it may already
exist somewhere, and if it doesn't, I don't think it will be too hard to
build.  (If anyone knows better, please let me know!)
 
Best,
Richard



Re: [O] Citation processing via Zotero + zotxt

2015-12-02 Thread Richard Lawrence
Hi Rasmus and all,

Rasmus  writes:

> Also, last I checked Zotero also existed as a standalone manager.  If this
> also works re your points below, using Zotero is a non-issue.  There
> exists a plugin to keep a bibtex file up to date.  I don’t know if it’s
> two way, though.

I haven't looked into whether things work the same way when Zotero is
installed as a standalone program, but I don't imagine they'd be
significantly different.  (What do you mean by "using Zotero is a
non-issue"?  Do you think installing as a standalone program would be
better, or did I misunderstand?)


>> Previously, I thought that it would be a similar amount of work to
>> communicate with Zotero from Emacs as any of the other CSL
>> implementations out there.  However, after looking at zotxt a bit more
>> closely, I discovered that it has an (undocumented) API endpoint [3]
>
> This sounds amazing, but also dangerous.  Do you know whether stabilizing
> the API has been discussed upstream?

As Matt mentioned, "upstream" in this case is zotxt, not Zotero.  I have
heard from Erik that he's open to changing and stabilizing this API.

>> that pretty much does exactly what we need: it accepts a list of
>> citation objects, and returns a list of formatted citations and a
>> formatted bibliography, which can be inserted into the exported
>> document.
>
> Could you give an example of the sort of input you give?  I.e. is it based
> on keys?  From my bibtex-centric world view I imagine something like:
>
>I send key or pointer @K to a DB entry as well as a CSL-file pointer C,
>and maybe a desired output format F.  I get a string back that is the
>formatting of the data behind @K formatted according to the rules in C,
>adapted to F.

Yes, that's correct, except that ideally you send the data for all
citations at once (because context is important).  So for example, if
Erik accepts a patch for some small changes I wrote, one can query

/zotxt/bibliography?outputFormat=html=chicago-fullnote-bibliography

sending POST data like

[
{ "citationItems": [{"key": "0_ZOTKEY1"}],
  "properties": {"noteIndex": 0}},
{ "citationItems": [{"key": "0_ZOTKEY2"}]
  "properties": {"noteIndex": 0}},
...
]

and get back data that basically looks like

{
  "bibliography": [
{
  ...
  "bibstart": "\n",
  "bibend": ""
},
[
  "  Formatted entry for 0_ZOTKEY1\n",
  "  Formatted entry for 0_ZOTKEY2\n",
  ...
]
  ],
  "citationClusters": [
"Formatted citation for 0_ZOTKEY1",
"Formatted citation for 0_ZOTKEY2",
...
  ]
}

> Is that correct?  If so, does it support html, text and odt?

At the moment it supports html and text.  I suppose it could be made to
support ODT, though I'm not sure how difficult it is.  However, I think
a better solution would actually be to have it return *Org* markup, and
then replace citation objects in the document with that.  The main
problem this solves concerns note-based styles: we can insert Org
footnotes into the document at the beginning of the export process, and
then let Org figure out how to number the total set of footnotes in its
usual way.  I think it's a lot harder to deal with citation footnotes if
they come already-formatted in the output format.

>> Erik has also written a package for communicating with zotxt from Emacs,
>> zotxt-emacs [4], which is available on MELPA.  This package already
>> contains a lot of useful functions for querying the Zotero database and
>> inserting reference data into documents, including links in Org
>> documents.  I think it would be pretty straightforward to extend this
>> package to provide a nice UI for writers who are inserting citations
>> into Org documents, including search-based lookups of keys, etc.
>> Perhaps org-ref could also be taught to communicate with zotxt (with or
>> without zotxt-emacs) without too much work.
>
> I guess we’d need to convince Erik to move it to GELPA.  Fortunately (from
> that point of view), neither zotxt nor zotxt-emacs seems to have too many
> contributors.

Is the concern here that Org should not officially depend on anything
that isn't in GELPA?  Or just that it would make installing easier for
users?

>> I know that many people (perhaps especially the `power users' who have
>> been active in the citations discussion so far) prefer to maintain their
>> reference database without the aid of a GUI reference manager like
>> Zotero.
>
> I think this solves part of that issue.  Again it could be a dangerous
> solution.  Alternatively, perhaps it’s possible to feed-in a bibtex file
> to Zotero via an API.  This would also limit the damage.

I am hoping it will prove simple to feed a .bib file to Zotero via an
existing API, or to add such an API to zotxt.  But at the moment that's
just a hope.  (If anyone knows more about how to access a BibTeX
translator in a Zotero plugin, please get in touch!)

> IMO we can leverage zotero as a tool, but we cannot enforce it 

Re: [O] Citation processing via Zotero + zotxt

2015-12-02 Thread Richard Lawrence
Hi Matt and all,

Matt Lundin  writes:

> Given these complexities, it seems that if we went the zotero route we
> could end up with a fairly large installation chain (firefox, zotero,
> zotxt, plugin for zotero). And this would require installing items from
> multiple, heterogeneous sources.

Well, I would guess that many people who are interested in this already
have Firefox installed, and after that, you just need to install two
Firefox plugins: Zotero and zotxt.  Open a couple of links, give your
permission, and that's it.

If you're skeptical, I encourage you to try it:

https://www.zotero.org/download/
https://addons.mozilla.org/en-US/firefox/addon/zotxt/

It's pretty easy.  And removing the plugins via about:addons is just as
easy.

> I wonder at this point whether pandoc-citeproc (packaged with pandoc)
> would actually be the simpler route. It can parse bibtex files directly
> and (as a filter within pandoc) can output formatted citations in org
> format.

We have discussed this before, and in fact, I already started work along
this route: see https://github.com/wyleyr/org-citeproc

I stopped because people objected that distributing a Haskell program is
too difficult.  Even if you can install pandoc-citeproc via your
system's package manager, to build org-citeproc against it you need a
complete Haskell build environment, which is (somewhat notoriously)
difficult to work with, and too much to expect for the average person
who just wants citation support in their Org documents.  Nor has anyone
volunteered to take care of building and distributing a binary for every
platform we'd want to support (including, I assume, Windows and OS
X...).

> As a GNU/Linux user, I would find installing zotero and all the add-ons
> messier and more cumbersome than installing pandoc and/or node-js (were
> we to use citeproc-js) from the command line.

I'm a Debian user, so I can appreciate your concern here.  But it's only
simpler to use the system package manager if all the dependencies are
already packaged for $YOUR_DISTRO, in a version that's up-to-date enough
for you to use.  Given the diversity of Org users, it seems likely that
we won't be able come up with a solution that goes via system package
managers that will work for everybody, at least not without a lot of
work.

The nice thing about Firefox (and these days, Emacs) is that it's a sort
of cross-platform package manager.  If the citation processing
dependencies are just Firefox plugins, they'll be much more accessible
to a much wider group of people without a lot of work on our part.  So,
that's why I'd prefer depending on Zotero to depending on something like
org-citeproc or citeproc-node.

Best,
Richard



Re: [O] Fwd: Citation processing via Zotero + zotxt

2015-12-01 Thread Richard Lawrence
Hi Eric and all,

Eric S Fraga  writes:

> On Tuesday,  1 Dec 2015 at 07:12, Matt Price wrote:
>> I love Zotxt. my only concern is for those people who run a remote Emacs.
>> If, say, you run emacs on a server that you access by ssh, it will not be
>> possible to process your citations with org.
>
> I'm not sure I understand this.
>
> Is the implication that one will need to be connected to the Internet to
> be able to export citations?  If so, this is not good.

No, you should not need to be connected to the Internet.  As I envision
things, Emacs will just speak to the Zotero installation on the local
machine.  That's what zotxt does now.  So, there should be no problem as
long as Firefox, Zotero and zotxt are running on the same machine that
is running Emacs.

I suppose the concern is that someone running a remote Emacs is likely
doing so on a machine that does not have a GUI...and hence may not have
Firefox installed, and consequently won't have Zotero or zotxt.  In that
case, citation processing would have to go another route.

Are there a lot of people in that situation?
 
Best,
Richard



Re: [O] Citation processing via Zotero + zotxt

2015-11-30 Thread Richard Lawrence
Hi John and all,

Thanks for your input!

John Kitchin  writes:

> This really means you are using bibtex as the reference database
> backend, and probably helm-bibtex to insert citations as org-mode links.
> Bibliography generation is probably managed by LaTeX/Bibtex. org-ref
> mostly provides the functionality of the links, and some utility
> functions to help build and maintain bibtex files, and to check all the
> cross-reference, label and citation links in the document.
>
> Ideally, something like zotxt would enable Zotero as the reference
> database backend with some selection mechanism (it could be helm, or
> ido, or some other interface) to insert citation syntax

This is pretty much what the zotxt-emacs package does.  It communicates
with the zotxt API, and it defines a minor mode for use with Org.  This
minor mode provides functions for querying the zotxt database (by
default, it searches by author, title and year) and for inserting and
updating Org links that point to items in the Zotero database.

It should be straightforward to extend these capabilities to insert
citation syntax instead of, or in addition to, links.

> , and at export time also leverage Zotero to generate the
> bibliography. I guess this would not too often be export to LaTeX,
> since that would require syncing with a Bibtex file. For quick and
> dirty work this would be fine, but my experience is converted bibtex
> files are never clean enough for production publication, and require a
> lot of manual cleaning of capitalization, journal abbreviations, math,
> etc... It is ok to do it once, but not every time you build a
> document!

Hmm, that is interesting and a bit concerning.  Zotero has the ability
to export sets of items to a BibTeX file.  Ideally this kind of cleanup
should, and in principle could, be dealt with `upstream' in the Zotero
database.  The CSL representation of items allows for pretty
fine-grained and explicit control over things like capitalization and
abbreviations.  But I do not know how much of that is exposed in Zotero,
and I don't know how well it is preserved upon export to BibTeX.

Certainly, I'd want export to LaTeX to be just as quick and painless as
export to other backends in general.  Some manual cleanup of the .bib
file seems acceptable to me as a final step before publication, but I
think it would count against Zotero if it was too annoying to

Zotero can also import BibTeX, so it would also be possible in principle
to go in the other direction: keep your database as .bib, but convert it
on the fly when exporting to other backends.

> A minimal, generalized interface that would probably work with org-ref
> would look something like (* is some prefix name like org-cite-):
>
> *-select-citations-from-db opens some selection interface with
>   candidates from the db-backend ((org-)bibtex, Zotero, Mendeley, RIS,
>   ...), allows user to select some citations with some filtering
>   capability then inserts citation syntax into org file. This function
>   should allow multiple selections, and allow appending to an existing
>   citation. org-ref supports reftex and helm-bibtex for this, and ebib
>   would also work.

As I mentioned above, zotxt-emacs already provides an API in this direction.

> *-export-citations This is the most difficult part, this code would have
>   to functionally do what LaTeX/BibTex accomplishes, and probably with
>   multiple passes to collect citations, get formatted bibliographic
>   entries, replace citations in the text (eg with numbers, au-year,
>   etc...), and insert the formatted bibliography in the right place. If
>   Zotero can give the formatted entry for each citation, this might be a
>   good start. It would be integrated into the export process. It might
>   even be backend-agnostic if the formatting was just plain or in org
>   markup text.

Yep, this is the hard one, but there is some more-or-less working code
to do this (via my org-citeproc script) in the in-progress org-cite.el.
I'm proposing reworking this code to communicate with Zotero, rather
than org-citeproc or another citeproc implementation.

> *-on-cite-click-func This is technically optional, but it is what makes
>   org-ref so great. Figures out what citation was clicked on, and runs
>   some action on that citation, e.g. open citation, find related
>   articles in google scholar, open browser to citation url, open notes
>   to the citation, etc... This is org-ref's most clever code, figuring
>   out what was clicked on, and providing multiple actions on that
>   citation key.

This needs more work, in org-element.  Right now, as far as I know,
there's no support for making citation-reference objects clickable.  On
the other hand, it doesn't seem like it should be that difficult.

> It is pretty easy to use zotero. I couldn't figure out zotxt though.

Hmm, where did you have trouble?  I was able to install it via this link


[O] Citation processing via Zotero + zotxt

2015-11-28 Thread Richard Lawrence
Hi everyone,

For the past few days, I've been looking more closely at using the
combination of Zotero [1] with Erik Hetzner's zotxt plugin [2] as a
means of processing citations when exporting to non-LaTeX backends.  I
am now thinking that this is probably our best option, but I'd like to
know what other people think before I sink a lot of work into it.

Here are the reasons I think this is the best option:

1) It is really easy for users.

For those unfamiliar, Zotero is a reference manager, and zotxt is a
Zotero plugin that makes it easier to work with Zotero from plain text
documents.  Both are Firefox plugins, which means they can be installed
by a non-technical user with a couple of clicks.  It also means that
users get updates automatically.  I think this is *really* important.

Pretty much all the other options we have talked about seem like they
will require multi-step, non-trivial installation procedures ("First
install {Node.js/Haskell/JVM ...}, then install
{citeproc-node/pandoc-citeproc/citeproc-java...}, then install our
wrapper script...").  Updating could require other manual operations of
similar complexity.  Avoiding that kind of procedure will make citations
a lot more usable from Org for everyone.

Also, unlike the other options, Zotero is a full-featured reference
manager, not just a batch processor.  So we as users get a useful piece
of software with a simple installation procedure; the other options
require a complex installation procedure for a less-useful program.

2) It is quite complete.

Previously, I thought that it would be a similar amount of work to
communicate with Zotero from Emacs as any of the other CSL
implementations out there.  However, after looking at zotxt a bit more
closely, I discovered that it has an (undocumented) API endpoint [3]
that pretty much does exactly what we need: it accepts a list of
citation objects, and returns a list of formatted citations and a
formatted bibliography, which can be inserted into the exported
document.

This endpoint still needs a little bit of work, to generalize it and
make it easier to get the data in the format we need.  (That is probably
why it is undocumented in the README.) But it requires much less work
than I thought it would, and much less work than it would be to get a
full-featured setup with something like citeproc-node.

Erik has also written a package for communicating with zotxt from Emacs,
zotxt-emacs [4], which is available on MELPA.  This package already
contains a lot of useful functions for querying the Zotero database and
inserting reference data into documents, including links in Org
documents.  I think it would be pretty straightforward to extend this
package to provide a nice UI for writers who are inserting citations
into Org documents, including search-based lookups of keys, etc.
Perhaps org-ref could also be taught to communicate with zotxt (with or
without zotxt-emacs) without too much work.

3) It uses citeproc-js.

In previous discussions, I think we agreed that it would be best for us
to use citeproc-js as a CSL processor, since it is the `canonical' CSL
implementation, as opposed to pandoc-citeproc or citeproc-java.  Zotero
just uses citeproc-js internally to process citations, so it meets this
requirement. 


I know that many people (perhaps especially the `power users' who have
been active in the citations discussion so far) prefer to maintain their
reference database without the aid of a GUI reference manager like
Zotero.  I still think Zotero + zotxt is the best option for non-LaTeX
citation processing, even for these folks.  The ease of installation
(and removal) of the required programs alone makes it worth it, even if
you never actually populate a Zotero database.  So given what I know at
the moment, I think our efforts would best be directed at making the
in-progress org-cite library communicate with Zotero via zotxt.  What do
you think?

Best,
Richard

[1] https://www.zotero.org/
[2] https://gitlab.com/egh/zotxt/
[3] See the bibliographyEndpoint function in:
https://gitlab.com/egh/zotxt/blob/master/extension/bootstrap.js
[4] https://gitlab.com/egh/zotxt-emacs



Re: [O] contributing to work on citations

2015-11-22 Thread Richard Lawrence
Hi Matt and all,

Matt Price  writes:
>> OK, I tried rebasing on current master, is online here & pull request has
>> been sent ot richard:
>
> https://github.com/titaniumbones/org-mode

Sorry it took me so long to get around to this.  Because there have been
commits to master in the meantime, and because I wanted to clean up the
(unnecessarily complicated) history of my branch, I have performed
another rebase of my branch onto master and put the result in a new
branch, here:

https://github.com/wyleyr/org-mode/tree/wip-cite-rebase

This branch linearizes the history of the various changes that went into
org-element (parsing citations) and org-cite (handling citations), and
cleans up my commit messages, which were produced by an old magit and
pretty ugly to deal with.

Hopefully that will make it easier for someone to continue this work!

Best,
Richard




Re: [O] Some projects

2015-10-27 Thread Richard Lawrence
Hi Rasmus and all,

Rasmus  writes:

> I would feel more comfortable relying on a JS library.  Perhaps it’s also
> easier to find people who are willing to work on/knows JS over the long
> haul...

Yes, I agree with both you and Aaron here.  JS is easier to distribute
and probably easier to find people to maintain.

>> OTOH, pandoc-citeproc includes a bibtex parser; we’d need to write a JS
>> one and wire it up to citeproc-js.  When I looked (quite some time ago),
>> there did not seem to be any good bibtex parsing libraries in JS (and
>> several third-rate ones).
>
> Bibtex support is essential, of course.
>
> Can someone remind me why citeproc-java isn’t good?  AFAIR, it has a
> bibtex parser.  But probably it lacks in some other dimension...

My recollection is that its command-line tool is not all that flexible,
and thus getting it to produce the kind of output we'd need requires
writing and maintaining Java, which no one really wanted to do.  Also, I
think the thought of firing up the JVM to process citations in an Org
document seemed a little...expensive.

Best,
Richard




Re: [O] Some projects

2015-10-26 Thread Richard Lawrence
Hi Nicolas and all,

Nicolas Goaziou <m...@nicolasgoaziou.fr> writes:

> Richard Lawrence <richard.lawre...@berkeley.edu> writes:
>
>> Yes, this is my understanding, too.  In particular, there does not seem
>> to be an Elisp CSL library, and it would be a lot of work to write
>> one.
>
> Here is a genuine question: what is the difficult part, or if there is
> none, the longer parts of the work? AFAIU, CSL is XML and we can use
> libxml for the low-level parsing.

I can't speak to how difficult it would really be.  But I do think it
would be a lot of work, and tedious, unglamorous work at that.
Essentially it means implementing the CSL specification:

http://docs.citationstyles.org/en/stable/specification.html

The specification describes the XML format for CSL styles, which IIUC
are something like XSLT programs: they describe how to transform an
input document tree into an output document tree.  So, implementing CSL
is something like implementing a very special-purpose declarative
programming language, which has to handle declarations about every
detail of citation formatting (where to put periods, how to capitalize
names like "von Neumann", whether to use "&" or a translation of "and"
for the current locale, etc.).  Lisp is a good language to do this kind
of thing in, but parsing the XML is surely the easy part.

> Could you give some examples of the UI offered by the citeproc-hs
> library?

The library I ended up wrapping is actually pandoc-citeproc, which is a
fork of citeproc-hs (which seems to be unmaintained).

pandoc-citeproc essentially exports one main function, processCites.
This function transforms a Pandoc data structure containing Citation
nodes into another Pandoc where the citations have been rendered into
the text and the bibliography is appended at the end of the document.

processCites also requires a list of references (read from a
bibliography database, e.g. .bib files, via the readBiblioFile function)
and a CSL style (read from a file via the readCSLFile function).  So
basically, the pandoc-citeproc API consists of these three functions:
processCites, readBiblioFile, and readCSLFile.

Best,
Richard




Re: [O] Some projects

2015-10-26 Thread Richard Lawrence
Hi all,

Aaron Ecay <aarone...@gmail.com> writes:

> Hi Nicolas,
>
> Thanks for writing this up.  It is important to think about, and
> ultimately solve, all the issues you raise.

Yes, thanks for this, Nicolas!  

> 2015ko urriak 25an, Nicolas Goaziou-ek idatzi zuen:
>> 
>> ** Citations
>> 
>> Development apparently stopped for some reason. We have a citation
>> syntax for Org in wip-cite and some work done in wip-cite-awe and
>> probably elsewhere.
>> 
>> I think we could at least provide features defined in Org Ref using the
>> new syntax (minus hydra/helm related functions).
>> 
>> We don't need a silver bullet. Just something with a non-empty user
>> base, and extensible. In any case, the work done so far shouldn't be
>> wasted.
>
> I was working on this rather intensively at one time, but I had to stop
> because other aspects of life intruded.  I have just been coming back
> towards a situation where I can imagine myself having some (still small,
> but non-zero) chunks of time to devote to working on org.  So I hope I
> will be able to pick this back up, but (regrettably) I’m not able to
> make any promises.
>
> Based on my recollection, here’s what the problems were when I stopped:
>
> - The only “off the shelf”-capable citation processing library that we
>   found last time is in Haskell, which introduced some difficulties for
>   distributing the resulting tool.  I know some projects
>   (e.g. git-annex) are written in Haskell and distributed as static
>   binaries for windows/mac/linux/etc.  We’d need to figure out how to do
>   this, or find another citation processing library in an
>   easier-to-distribute language.

Yes, this is my understanding, too.  In particular, there does not seem
to be an Elisp CSL library, and it would be a lot of work to write one.

The other CSL library that looks complete and usable is citeproc-js; but
like the Haskell library (pandoc-citeproc) it would need to be wrapped
somehow so that it can talk with Org.

It should be relatively straightforward for someone who knows Javascript
to write such a wrapper, if anyone wants to work on that.  But this does
not really solve the problem with distribution.  Either of the
off-the-shelf CSL libraries will require both a wrapper and a platform
for building/installing/running the wrapper and library as a complete
external tool.

> (I should say, all the work on the external tool was done by Richard
> Lawrence; I worked on the exporter for the citation syntax including
> the interface with an external tool.)

The tool I was working on is here:
https://github.com/wyleyr/org-citeproc

The branch of Org that it needs is here:
https://github.com/wyleyr/org-mode

At the moment, it supports single- and multiple-work citations in inline
styles (e.g. Chicago/Harvard type citations, and I think also styles
that use numbered references to the bibliography).  It doesn't presently
work with note-based styles, and making it work will require some
modifications on the Org side.  Specifically, the Org side will have to
get a bit smarter about how it inserts the formatted citations into the
document (Org needs to understand them as footnotes so that they get
correctly numbered, etc. amongst other non-citation footnotes).

> - There is a difference between citations as done by latex/bibtex/etc.,
>   and those done in every other format (handled through CSL).  Assuming
>   latex users want to keep their native processing rather than
>   delegating to CSL, we need to solve the myriad small inconsistencies
>   between these two tools.  I think this is an area where it’s important
>   to get things right: users of citations generally have exacting
>   requirements.  “Approximately Chicago-style” or “almost MLA” aren’t
>   worth anything.

I guess I would just add that it is not clear how much we need to solve
here, at least in the short term.  I can't remember whether we found any
concrete examples of needs people have that BibLaTeX can handle but
CSL cannot, or vice versa.  Anyway, there is a core set of citation
features that both types of backends handle readily, and I think it
would be a big win to have these accessible via a common syntax in Org.
No silver bullets is indeed the maxim to keep in mind.

> (I should also say, if someone else is interested in working on this
> please don’t hesitate to jump right in.  I will help you however I can!)

I also want to echo this.  I don't really have much time to work on this
myself right now (trying to get the ol' dissertation finished this year)
but I will help out however I can.

Best,
Richard




Re: [O] Showing Property in headline; generating table from properties; exporting table to CSV

2015-08-06 Thread Richard Lawrence
Hi Matt and all,

Matt Price mopto...@gmail.com writes:

 On Thu, Aug 6, 2015 at 9:13 AM, John Kitchin jkitc...@andrew.cmu.edu
 wrote:

 How do you enter your grade? I use a function, bound to a convenient key
 like s-s g, which sets the grade property. You could have that function
 change the heading TODO state to DONE so you know it is done, and maybe
 add a tag with the grade, or just append the grade on the end of the
 headline.

 Adding the grade as a tag doesn't seem quite right, as I often change
 grades after a rewrite.  I'd need to get rid of the original tag.

Here's another idea for getting the grade into the headline: set the
grade as a priority, like

** [#A] John Doe

Pros: it's a simple hack that doesn't require you to do any additional
parsing; it's easy to change up or down either interactively or
programmatically (cf. org-priority* functions); and it's easy to sort
headlines by priority in a custom agenda view, to give you a quick
overall picture of your grade distribution.

Cons: this would only work for A/B/C/D/F grades, and even then, it
doesn't represent +/- variants.  So it's not very flexible or granular,
but it might be enough.  

Best,
Richard

P.S. Like others, I too have written my own grading system:

https://github.com/wyleyr/schoolutils

It stores grades and student data in a SQLite database, and allows you
to write end-of-term grade calculations in Python, instead of messing
with spreadsheets. (Overkill? Maybe so... :)

It currently only has a command line interface, but I have been thinking
about adding an Org interface that would do basic CRUD operations on
grades via Babel.  If anyone is interested in such a thing, let me
know...




Re: [O] How to get list item depth within the exporter framework?

2015-07-27 Thread Richard Lawrence
Hi Marcin,

Marcin Borkowski mb...@mbork.pl writes:

 as I mentioned, I'm writing a simple exporter.  However, I stumbled on
 something apparently simple.  How to get the current list level within
 org-whatever-item transcoder?

I ran into this problem once; the solution I found was to just walk up
the AST via org-export-get-parent until you run out of parents.

Something like this should work (untested, and probably needs to be
tweaked, as my Elisp is a little rusty):

(defun find-depth (element)
  Find the depth of a (list) element during export.
  (let ((parent (org-export-get-parent element)))
 (case (org-element-type parent)
   ('item (1+ (find-depth parent)))
   ('plain-list (find-depth (org-export-get-parent parent)))
   (t 1

Hope that helps!

Best,
Richard




Re: [O] How to make a non-GPL Org-mode exporter?

2015-07-27 Thread Richard Lawrence

Hi Marcin and all,

Marcin Borkowski mb...@mbork.pl writes:

 after a short discussion in a recent thread, I have a serious technical
 question.

 Assume that (for some reason) I want to write an Org-mode exporter which
 won't be GPL'd.  (Use-case: having written a few custom exporters, I'm
 writing a tutorial on them, and I consider publishing a *tutorial* with
 GPL'd code a Bad Thing™.  (The idea of a programming tutorial is that
 other people can or even should reuse the code in the tutorial, right?
 And I see no reason to impose GPL on them.))

I am not a lawyer, this is not legal advice, etc., but for what it's
worth, here's my opinion:

It sounds to me like you are not creating a program derived from GPL'ed
code, but rather *quoting* GPL'ed code (and introducing some
modifications to those quotes) in a document whose purpose is to educate
its readers.

I don't know about your jurisdiction, but in the U.S., that sounds to me
like it would fall well within the bounds of fair use:

https://en.wikipedia.org/wiki/Fair_use

(There is also a brief section there about fair use in Poland; maybe
that will help you?)

Only a court can say, but if you are only publishing this source code as
part of a tutorial, so that it
  - has an educational purpose
  - serves to discuss ideas that can be gleaned from reading the original
  - does not substantially copy the original
  - does not devalue the original 
then I would guess the GPL does not apply to the code, as it is quoted/derived
in your tutorial, due to a fair use limitation.

So my advice would be: (1) treat the code in your tutorial along
whatever lines are appropriate for quotation of copyrighted works in
your jurisdiction; and (2) maybe include a notice in your tutorial to
your readers, that if they create *and distribute* a working exporter
based on the code in your tutorial and/or code in the Org distribution,
the GPL may apply.

If you are actually distributing the complete exporter as a working
program, that seems a little murkier to me; the issue there, as you are
aware, is whether that working program counts as a derived work.  I have
no idea how to answer that question.

Hope that helps!

Best,
Richard




Re: [O] Shifting only past timestamps

2015-07-25 Thread Richard Lawrence
Hi Edward,

Edward Guyatt edwardguy...@gmx.co.uk writes:

 I like to be reminded to check my post on Mondays and Thursdays, every
 week. When I've done it on Monday, I mark it 'DONE' so that I can stop
 org-mode reminding me until Thursday. Here's how I used to achieve
 this:

 *** URGENT [#A] Check post  :important:
 DEADLINE: 2015-07-27 Mon +1w -0d
 DEADLINE: 2015-07-30 Thu +1w -0d

This is, I believe, technically invalid by the current standards of Org
syntax.  See http://orgmode.org/worg/dev/org-syntax.html under Clock,
Diary Sexp, and Planning; planning elements have to be on the same
line, and I am pretty sure you can't have two deadlines.  I am surprised
Org operates on the second one at all.

I solve this problem by just having two different headlines, each with
their own deadline, like:

*** URGENT [#A] Check post  :important:
DEADLINE: 2015-07-27 Mon +1w -0d

*** URGENT [#A] Check post  :important:
DEADLINE: 2015-07-30 Thu +1w -0d

I know it may not seem terribly elegant, but it's simple, and it's the
only way I can think of to have the deadlines recur on fixed days of the
week.  (If that's not important to you, you could use habits instead of
two headlines.)

Best,
Richard




Re: [O] org-cite and org-citeproc

2015-06-18 Thread Richard Lawrence
Hi Matt,

Matt Price mopto...@gmail.com writes:

 I'm wondering what kind of work is required to make use of org-cite and
 org-citeproc at present. In particular, I'm wondering what kinds of changes
 I'll need to make to my current setup, and whether it's worthwhile to use
 my ultra-slow coding skills to create whatever glue is still necessary.

At the moment, org-cite/org-citeproc has no way of talking to Zotero.
So you'd need to manually export citation data from Zotero to a format
that pandoc-citeproc understands, like BibTex.  It sounds like zotxt and
zotxt-emacs provide a lot of what's needed to glue org-cite together
with Zotero; so one thing that would be helpful, if you're up for it, is
hacking org-cite to pull bibliography data from Zotero in a format that
can be passed to org-citeproc.  I don't use Zotero myself, so this is
something I'm unlikely to do anytime soon without some help.  (Borrowing
code from zotxt-emacs and putting it in org-cite is probably the way to
go here, as I doubt that we want to make zotxt-emacs a dependency of
org-cite.)

 All of this is fine for my current purposes, but I would like to figure out
 a more flexible and enduring solution, so I'd like to try out org-cite and
 org-citeproc.  But I'm not quite sure what's required, and whether there's
 support currently for odt and html export.

`Flexible and enduring' does not describe org-citeproc at the moment. :)
I'd be very happy to have you test out org-citeproc and give feedback
that will help improve it, but I can't recommend that you rely on it or
switch to it for serious work any time soon.  It is a working
proof-of-concept, but only that.

Still, there is support in org-cite/org-citeproc for both HTML and ODT
export, and it handles quite a few of the common cases.  So let me know
if you're interested in trying it out.  There are brief installation
instructions in the README (https://github.com/wyleyr/org-citeproc); let
me know if you need more than that.
 
Best,
Richard




Re: [O] Conditional .gitignore for org-mode files

2015-04-27 Thread Richard Lawrence
Hi David,

David Dynerman da...@block-party.net writes:

 Sorry in advance, this might be more of a git question than an org-mode
 question, but I thought someone on this list might know the answer.

 Is it possible to conditionally gitignore certain files based on files
 that are being tracked?

 What I'd like is something like the following gitignore logic:

 if filename.org is tracked by git:
ignore filename.tex, filename.html

 If this isn't possible, does anyone have any nice setups for ignoring
 exported versions of org-mode files?

Unfortunately, I don't know how to do what you asked, but what I tend to
do in this situation is to run my exports in a build/ subdirectory, and
then add

build*

to my .gitignore.  Maybe that would work for you?

Best,
Richard




Re: [O] org-cite and org-citeproc

2015-04-06 Thread Richard Lawrence
Hi Aaron and all,

Richard Lawrence richard.lawre...@berkeley.edu writes:

 Alright, I'll try to move to json.el, and possibly change to having
 org-citeproc generate Org markup in the meantime.

Just a heads up: I've pushed some changes to my branch of Org to make
org-cite use json.el, and to add a basic Org format writer to
org-citeproc.

I have not made any other changes to org-cite to use the Org formatted
output from org-citeproc, though, as I believe doing this properly will
involve parsing the output and inserting it into Org's exporter's parse
tree (to accommodate the bibliography and note-based styles).  I won't
have time to work on that this week, but I'll come back to it.

Best,
Richard



Re: [O] org-exported pdf files?

2015-04-04 Thread Richard Lawrence
Hi Jude,

Jude DaShiell jdash...@panix.com writes:

 Do those files by default conform to screen reader accessibility
 standards or can such files be made to conform to screen reader
 accessibility standards? 

I don't actually know anything about this, but the quick answer is: Org
uses LaTeX to generate PDFs, so PDFs produced by Org will conform to
those standards to the extent that PDFs produced by LaTeX do.  

For any particular document, I would guess that a more detailed answer
depends on both the content of the document and the particular LaTeX
processing pipeline which produces the PDF (pdflatex? latex, then
dvipdf? etc.).

Hope that helps a bit, however tiny a bit it may be.

Best,
Richard




Re: [O] org-cite and org-citeproc

2015-04-02 Thread Richard Lawrence
Rasmus ras...@gmx.us writes:

 Richard: do your FSF papers in order.  Or do you plan to get them in
 order?

I haven't done them yet (never had a reason to!) but I have no problems
with it and I'll get started on it.  

Best,
Richard




Re: [O] org-cite and org-citeproc

2015-04-02 Thread Richard Lawrence
Hi Tom and all,

t...@tsdye.com (Thomas S. Dye) writes:

 OK, I see, that makes things clearer.  Would it make sense to have two
 keywords, say LATEX_CITE_STYLE and CSL_FILE or similar, so that the
 style can vary independently when exporting to LaTeX vs. non-LaTeX?  I'm
 thinking it will be tricky to come up with a single set of values for a
 CITATION_STYLE keyword that can be correctly mapped to both kinds of
 backend.  Or maybe CITATION_STYLE should have sub-keywords, like

 #+CITATION_STYLE: biblatex:authoryear csl:chicago-author-date.csl

 Won't the backends sort this out without the additional mapping?

Surely they could, but I guess I'm unclear on how that should happen.

Org could keep a variable mapping citation styles to default values for
the respective backends, like:

(defcustom org-cite-citation-styles
  '((author-year (biblatex-pkg-args 
citestyle=authoryear,bibstyle=authoryear)
   (csl-file /path/to/chicago-author-date.csl))
   ...))

so in a document, you could just write

#+CITATION_STYLE: author-year

or similar.  Is this what you have in mind?

That seems like a good way to provide reasonable defaults using a
high-level label.  But I think using a high-level label like this will
underdetermine the actual style to use (on both sides, I assume); and
the problem is that if we make the labels more fine-grained, there's no
longer any guarantee that, for a given style label, a suitable style
file will be available on both LaTeX and non-LaTeX backends.

There obviously needs to be some mechanism so authors can specify their
citation style quite precisely for the backends they are interested in.
(Maybe just customizing this variable would do the trick.)  But what
should the fallback mechanism look like when a particular style does not
specify the required information for a given export backend?  E.g., if
CITATION_STYLE is X and we're exporting to HTML, but the entry in
org-cite-citation-styles does not specify a CSL file for style X?  Would
it be enough to have a single 'default clause or similar in
org-cite-citation-styles to use in that kind of case?

Best,
Richard




Re: [O] org-cite and org-citeproc

2015-04-02 Thread Richard Lawrence
Hi Rasmus,

Thanks, this is helpful.  I will try to fix these things soon.

Rasmus ras...@gmx.us writes:

 Hmm.  But the citations are all just represented as text:p
 nodes...surely that doesn't have to be defined elsewhere?

 You are right.  Also, oolatex inserts citations as plain text as well.  As
 I recall, it can be done semantically and section 6.3 of the odt
 standard suggest that this may be true, but it's not immediately obvious
 how to do it.

 I am now guessing that the problem is that you can't have one text:p
 inside another.  Each paragraph is wrapped in a text:p, but so are the
 citations within it...maybe that is not correct and so LibreOffice
 doesn't like it.

 I don't think text:p can be nested cf.

 http://docs.oasis-open.org/office/v1.2/os/OpenDocument-v1.2-os-part1.html#__RefHeading__1415138_253892949

OK, good to know.  Looks like text:reference-mark or text:note might
be the way to go. 

 Also, the bibliography is not correct in the sense that if it was setup
 in the right semantic way, it would be gray in LO, like the TOC.

 Do you know what other markup is required in this case?  It looks like
 maybe the TOC is gray because it is marked with a text:protected
 attribute, or maybe because it has an associated OrgIndexSection
 style?

 It has to be formatted as a bibliography.

 http://docs.oasis-open.org/office/v1.2/os/OpenDocument-v1.2-os-part1.html#element-text_bibliography


Also useful.  This might take a while for me to figure out, as Pandoc
does not seem to generate this markup when formatting a
bibliography...maybe I'll see if they are willing to work on this
upstream.

Best,
Richard




Re: [O] org-cite and org-citeproc

2015-04-02 Thread Richard Lawrence
Hi Eric and all,

Eric S Fraga e.fr...@ucl.ac.uk writes:

 On Wednesday,  1 Apr 2015 at 08:49, Andreas Leha wrote:

 [...]

 I am a happy biblatex user for all my 'own' documents.  But (as was
 mentioned previously) scientific journals that accept latex submissions
 will require bibtex and won't support biblatex.  So, I'd say that one of
 the other methods (preferably bibtex) is still necessary.

 Ahhh, yes, I'd forgotten that journals expect bibtex.  This is a key
 requirement for me as well therefore.

Can someone suggest how a parenthetical citation with common prefix and
suffix data, like

  [(cite): For more on this topic, see:; @Work1 for a review; @Work2;
  and references therein.] 

should map to plain BibTeX?  Maybe there is no general answer to this
question, but what would a reasonable default be?  Maybe this?

  (For more on this topic, see: \cite{Work1} for a review, \cite{Work2},
  and references therein.)

That is, just place the prefix and suffix data in the surrounding text,
inserting commas after the part for each individual work, and wrapping
the whole thing in parentheses?

Thanks!
 
Best,
Richard




Re: [O] org-cite and org-citeproc

2015-04-02 Thread Richard Lawrence
Hi Aaron,

Thanks for your comments, and for looking over my code!

Aaron Ecay aarone...@gmail.com writes:

 I’ve been (barely) following this discussion, but have been too busy to
 do any actual coding.  I sat down today to try to integrate Richard’s
 branch with my work, but didn’t get very far.  I think it would be a
 waste of effort to try to support more than one citation processor
 (citeproc-java vs. org-citeproc).

In the long run, I agree, though I still think it might be good in the
short term to see how a few different tools behave.

 I went round and round with myself about this, and concluded that we
 ought to keep on working on the org-citeproc approach for now (drop
 citeproc-java).  

(I take that as a compliment -- thanks!  I'm curious as to what persuaded
you.)

 But I do think someone eventually ought to reimplement org-citeproc
 based on citeproc-js, to yield something that can be distributed via
 npm.  This will be less fool-proof than java, but better than the
 Haskell experience for many users (such as Rasmus and me – far from
 non-technical people!).

That is fine with me, as I think citeproc-js is the canonical
implementation, and probably the best supported.  I am not the person to
do this, though, as I don't really know JavaScript, and personally I
have found it rather frustrating to work with whenever I have had to do
so.

(I am a little puzzled, though, as to why you and Rasmus find the
Haskell experience unpalatable.  On my machine, I installed the Haskell
Platform, and after that, building and installing pandoc,
pandoc-citeproc, and other programs via cabal Just Works.  Has that not
been your experience?  Why would instructions like First install Node.js,
then do: $ npm install org-citeproc be better than First install the
Haskell platform, then do: $ cabal install org-citeproc?) 

 You mention zotero as a third option – it’s possible, but I think we’d
 be better served by a tool that focuses solely on processing and is
 not so closely tied with database management.

Yes, I agree, at least for those of us who don't use a citation manager.
On the other hand, other people already have Zotero, but would find it
difficult to install and configure a command-line tool and supporting
platform.

If there were a common API so that most of org-cite could operate
independently of external tools, but call out to whichever tool was
installed at critical moments, I think that would be beneficial from a
user's perspective.  (That was part of my motivation for representing
citations to org-citeproc as citeproc-js-compatible JSON.)  I know that
would be more work, but it shouldn't lead to too much duplicated effort
if we are careful to separate the tool-independent code from the
tool-dependent code, and to minimize the latter.  It would also make it
easier to support new and better tools in the future as they become
available.

 There are a couple of other differences in our approaches.

 The first is whether the processor generates the in-text citations (you)
 or whether it’s done in elisp (me).  It’s not obvious which is superior.
 The real test will come when more diverse citation types are implemented
 (e.g. full citations in footnotes or numbers which reference a numbered
 bibliography at the end of the document).

 The second is whether the processor generates markup in the target
 language directly (you) or whether the processor’s output is converted
 to org markup, then passed through org’s exporters (me).  I think my
 approach here is preferable, since it generalizes to every backend.  Do
 you think something like this could work for org-citeproc?  It would
 probably require some additional special code in ox-odt.  (But we might
 need that in any case, see below.)

Yes, I think that could work, and shouldn't be too hard to achieve, as
Pandoc already has an Org writer.  And I agree it is probably preferable
to go this way.  I realized this when I tested org-citeproc with a CSL
file for a note-based style; things start to get complicated when you
have to parse the output format to look for e.g. a footnote reference
and definition.

 It would be good from an intelligibility/maintainability perspective if
 you could use the json.el library rather than generating json strings by
 hand.  This is part of the emacs core since version 23.something, so
 compatibility should not be a big issue.

Ah, ok, I didn't know about json.el.  Yes, I agree.

 You wrote (mixing replies to various emails in this thread):

 This complicates things enough that probably custom citation modes
 [in Latex – AE] should be defined as Lisp functions, rather than via
 format strings...what do you think?

 I’d rather avoid it, since I think org-latex is going to be an important
 usecase for many people.  I see us eventually supporting two flavors of
 latex output.  The first should aim to generate a full set of biblatex
 commands but with little user customizability.  The second will rely on
 just 2 citation commands 

Re: [O] org-cite and org-citeproc

2015-04-01 Thread Richard Lawrence
Rasmus ras...@gmx.us writes:

 Richard Lawrence richard.lawre...@berkeley.edu writes:

 I don't really know anything about the ODT format, though.  My code
 more-or-less blindly pastes Pandoc-generated XML into the document
 during Org ODT export.  Can someone who knows more about the format take
 a look at the file and see if there is some subtle problem I'm not
 noticing?

 I can't test your code 'cause I, for personal/silly reasons, refuse to
 spend any more dealing with compiling Haskell.

Fair enough. :)

 That being said, my gut feeling is that you have to define the data
 elsewhere.

 For example, to add a (sub)title to a odt document the field/keyword is
 defined in a file different from contents.xml and will just not be printed
 if used in contents.xml only.

Hmm.  But the citations are all just represented as text:p
nodes...surely that doesn't have to be defined elsewhere?

I am now guessing that the problem is that you can't have one text:p
inside another.  Each paragraph is wrapped in a text:p, but so are the
citations within it...maybe that is not correct and so LibreOffice
doesn't like it.

 Also, the bibliography is not correct in the sense that if it was setup
 in the right semantic way, it would be gray in LO, like the TOC.

Do you know what other markup is required in this case?  It looks like
maybe the TOC is gray because it is marked with a text:protected
attribute, or maybe because it has an associated OrgIndexSection
style?
 
Thanks!

Best,
Richard




Re: [O] org-cite and org-citeproc

2015-04-01 Thread Richard Lawrence
Hi Tom and all,

Thanks for answering my questions!

t...@tsdye.com (Thomas S. Dye) writes:

 With natbib, it is possible to give a pre-note and a post-note to the
 citation as a whole, but not to individual citations within it.  In
 order to support your syntax fully, I think BibLaTeX is needed.

OK, good to know.

 (Also, do you think it is important to support plain BibTeX at all?  It
 seems like we should not bother with this problem unless it's important
 for a lot of people.  I personally would be fine with just targeting
 BibLaTeX, and it sounds like Eric would be too.)

 Well, one benefit of Aaron's function was to make this choice
 superfluous, both now and in the future.  It binds the two citation
 commands you've implemented to citation commands implemented in
 CITATION_STYLE.  As Aaron notes, it should be easy to modify this (to
 bind additional commands) when advanced citation support comes along.

I think I have to retract what I said earlier: I doubt this part of
Aaron's code still works in my branch, because I think Aaron was
assuming citation objects contain just one reference; in my branch, I've
merged in the parser support Nicolas later implemented for multi-cite
citations.  So a CITATION_MODE needs to know how to turn a list of
works, each with associated prefix and suffix data, into a complete
citation command.

This complicates things enough that probably custom citation modes
should be defined as Lisp functions, rather than via format
strings...what do you think?

I'm still having a hard time seeing what an analogous customization
would look like for non-LaTeX backends.  The LaTeX exporter is unique in
that Org produces output which must then be further processed by another
tool, so having customizable control over how a citation `looks' to that
tool makes sense.  But in other backends, the Org exporter itself
produces the final document; there's no intermediate representation
besides Org's own, plus whatever arguments are passed to a citation
processing tool like org-citeproc.  So, if that's right, the analogous
customization in a non-LaTeX backend would be something like a filter,
one that pre-processes citation objects before they are run through the
external tool, or that post-processes the strings that come back (or
both).  Does that make sense?  Certainly, both of those things are
possible.

 Typically, a bibliography style file defines several citation commands,
 which might belong to one or more modes.  ...
 I think you might be able to merge CSL_FILE and CITATION_STYLE, since
 they both point to a style file.

OK, I see, that makes things clearer.  Would it make sense to have two
keywords, say LATEX_CITE_STYLE and CSL_FILE or similar, so that the
style can vary independently when exporting to LaTeX vs. non-LaTeX?  I'm
thinking it will be tricky to come up with a single set of values for a
CITATION_STYLE keyword that can be correctly mapped to both kinds of
backend.  Or maybe CITATION_STYLE should have sub-keywords, like

#+CITATION_STYLE: biblatex:authoryear csl:chicago-author-date.csl

or something similar?

Best,
Richard




Re: [O] org-cite and org-citeproc

2015-03-31 Thread Richard Lawrence
Hi Eric and all,

Eric S Fraga e.fr...@ucl.ac.uk writes:

 On Saturday, 28 Mar 2015 at 10:53, Richard Lawrence wrote:

 I thought I should send an update to let you know that org-citeproc [1],
 the command-line citation processing tool I've been working on, now
 supports multi-cites.  I believe that means it is now capable of
 processing all citations in the basic citation syntax.  It can output
 plain text, HTML, and ODT (and a Pandoc native format, mostly useful for
 debugging).

 This looks really good!  Thanks.

Thanks!

 However, for some reason, libreoffice doesn't display the citations in
 the ODT document you have included.  I have had a look at the actual ODT
 file and it looks fine.  Can you suggest what may be wrong?  

Hmm, you're right.  I don't have LibreOffice on the machine where I am
working on org-citeproc, but I tested it on another machine (OS X,
LibreOffice version 4.2.8.2 I think), and the citation text is indeed
missing.

As you say, the actual file looks fine to me, and it displays correctly
on Google Drive (which is where I originally tested the output).

So LibreOffice might be where the problem is.  That doesn't mean there
isn't a problem with org-citeproc, or the ODT exporter, but given that
the file looks fine and another viewer handles it correctly, LibreOffice
would be my first suspect.

I don't really know anything about the ODT format, though.  My code
more-or-less blindly pastes Pandoc-generated XML into the document
during Org ODT export.  Can someone who knows more about the format take
a look at the file and see if there is some subtle problem I'm not
noticing?

 A second question: what will be required to use the new cite syntax with
 LaTeX/PDF which will remain my main target for export?

I think this needs more discussion, actually.

The citation syntax can basically be mapped directly to BibLaTeX syntax,
so generating LaTeX that will be processed with BibLaTeX is a simple and
straightforward modification to Org's LaTeX exporter, and compiling the
exported document should continue to require no external programs except
the LaTeX distribution itself.  That is, `C-c C-e l whatever' should
continue to be all that is needed from a user's perspective, plus or
minus some LATEX_HEADER setup.

However, there are a couple of other scenarios to think about:

1) Some people may still need to use plain BibTeX.  Generating LaTeX
that is intended to be processed with BibTeX, as opposed to BibLaTeX, is
a little trickier, because (IIUC) BibTeX does not support multi-cite
citations.  Also, I don't know how easy it would be to capture the other
features of citations (e.g., the in-text vs. parenthetical distinction)
without relying on a package like natbib.  If generating
BibTeX-compatible LaTeX is needed, is it OK to rely on such a package?

2) Some people might find it useful *not* to generate LaTeX citation
commands, and instead have a tool like org-citeproc process citations
instead, with the exporter inserting the rendered output into the
document.  This could be useful if e.g. you are preparing to submit to a
journal that provides a CSL file, but not a BibTeX or BibLaTeX style.

If either of these scenarios represents an important use case, it will
be more work to implement.  

I suggest that for now we just target BibLaTeX, but I'd like to hear
from other people about whether there's reason to do more than that.
 
Best,
Richard



Re: [O] org-cite and org-citeproc

2015-03-31 Thread Richard Lawrence
Hi Tom and all,

t...@tsdye.com (Thomas S. Dye) writes:

 I know next to nothing about citations in general, so please bear with
 me: if multi-cite support means being able to condense citations (e.g.
 [1-3, 5, 9]), then bibtex can do at least some of that
 (e.g. http://texblog.org/2007/05/28/mulitple-reference-citation/).

 Multiple citations with natbib are limited in the sense that individual
 citations within them don't carry pre- and post-notes.  BibLaTeX does
 away with this restriction.

Just to clarify: our syntax allows both pre- and post-notes for each
individual reference within a citation, plus common pre- and post-notes
for the citation as a whole.  Is it only the latter which BibTeX does
not support?  Or is it also that it can't handle pre- and post-notes for
individual references when there is more than one work cited?  I don't
think I've ever tried to do something that complicated with plain BibTeX.

(Also, do you think it is important to support plain BibTeX at all?  It
seems like we should not bother with this problem unless it's important
for a lot of people.  I personally would be fine with just targeting
BibLaTeX, and it sounds like Eric would be too.)

 The org-export-cite-add-citation-mode-latex function that Aaron Ecay
 wrote allows the author to choose (implicitly) which package to use.  I
 like this design because it can accommodate new packages without changes
 to the Org mode code.

 ,-
 | Your comment inspired me to implement   
 | org-export-cite-add-citation-mode-latex in the experimental citation
 | support I just pushed.  So you can do:  
 | 
 | (org-export-cite-add-citation-mode-latex tsd  
 | \\mycitecommand[%s][%s]{%s} \\myparencitecommand[%s][%s]{%s})   
 | 
 | Add to your document:   
 | 
 | #+CITATION_MODE: tsd
 | 
 | And citations should just work with your chosen commands.  I’m sure when
 | advanced citation support comes along (whether from subtypes or plists),
 | a similarly simple wrapper can be implemented.  
 `-

 I haven't found time to experiment with the citation developments or to
 read through the developing code base.  Was this feature of Aaron's
 experimental support subsequently dropped?

No, I haven't touched that part of the code, and as far as I know that
should still work.  However, I also haven't been able to make sense of
how the CITATION_MODE and CITATION_STYLE keywords should work in
non-LaTeX backends, so my code doesn't make any use of them, either.  

If I understand correctly, the mode and the style are two pieces of
information that jointly determine how citations should be formatted.
The reason they are separated is that you can have multiple styles
associated with a mode (or is it the other way around?).  Is it right to
say that in LaTeX, choosing a mode and a style determines both
which citation commands are available and what their behavior is?

In the CSL world, both of these pieces of information seem to be
determined by the choice of CSL stylesheet.  Thus, I went with a single
keyword, CSL_FILE, to specify this information.  I guess one could also
go the other way, and try to select a stylesheet based on CITATION_MODE
and CITATION_STYLE, but I wasn't sure how to do that.  I also don't
really know if it would be useful to provide a notion of `custom modes'
outside of LaTeX; as far as I can see, all customization would just come
down to selecting a different stylesheet.  But maybe I'm missing
something important?

Best,
Richard




[O] org-cite and org-citeproc

2015-03-28 Thread Richard Lawrence
], and 
sometimes remain unpublished [(cite): @Ross1985; @Faraci1970].

*** Parenthetical, with prefix and suffix data for individual works
Some great ideas occur in books, articles, or collections
[(cite): see @Brandom1994 chapter 7; also @Hofweber2007; @Russell1919 is the locus classicus].  
Some occur in conference proceedings or dissertations
[(cite): @Rogers1996; for an overview, see @RogersKepser2007 and references therein].

*** Parenthetical, with common prefix and suffix data
Some great ideas occur in books, articles, or collections
[(cite): For more on this topic, see ; @Brandom1994; @Hofweber2007; @Russell1919; and references therein].  

*** All in-text, keys only
Some great ideas occur in books, articles, or collections
such as [cite: @Brandom1994; @Hofweber2007; @Russell1919].  

Some occur in conference proceedings or dissertations like
[cite: @Rogers1996; @RogersKepser2007; @Caponigro2003], and 
sometimes remain unpublished, like [cite: @Ross1985; @Faraci1970].

*** All in-text, with common prefix and suffix
Some great ideas occur in books, articles, or collections.
[cite: See: ; @Brandom1994; @Hofweber2007; @Russell1919; and references therein.]  

Some occur in conference proceedings or dissertations. 
[cite: For more on this topic, see ; @Rogers1996; @RogersKepser2007; @Caponigro2003].

* References
#+BIBLIOGRAPHY: here
@book{Brandom1994,
  author={Robert Brandom},
  title={Making it Explicit},
  publisher={Harvard University Press},
  year={1994}
}
@article{Hofweber2007,
  author={Hofweber, Thomas},
  title={Innocent Statements and their Metaphysically Loaded Counterparts},
  journal={Philosophers' Imprint},
  year={2007},
  volume={7},
  number={1},
  month={February}
}
@incollection{Russell1919,
  author={Bertrand Russell},
  title={Descriptions},
  booktitle={The Philosophy of Language},
  publisher={Oxford University Press},
  year={2001},
  editor={A. P. Martinich},
  chapter={15},
  pages={221--227},
  edition={Fourth}
}
@inproceedings{Rogers1996,
  author={James Rogers},
  title={A model-theoretic framework for theories of syntax},
  booktitle={Proceedings of the 34th annual meeting on Association for Computational Linguistics},
  year={1996},
  pages={10--16},
  address={Santa Cruz, CA, USA},
  organization={Association for Computational Linguistics}
}
@proceedings{RogersKepser2007,
  title={Model-theoretic syntax at 10},
  year={2007},
  editor={James Rogers and Stephan Kepser},
  organization={Association for Logic, Language and Information}
}
@phdthesis{Caponigro2003,
  author={Ivano Caponigro},
  title={Free not to ask: On the semantics of free relatives and Wh-words cross-linguistically},
  school={University of California, Los Angeles},
  year={2003},
  note={Cited text available at http://idiom.ucsd.edu/~ivano/Papers/2003_dissertation_revised_7-13-05.pdf}
}
@misc{Ross1985,
  author={Ross, John R.},
  title={The source of pseudocleft sentences},
  howpublished={Handout of a talk given at New York University},
  month={November},
  year={1985}
}
@unpublished{Faraci1970,
  author={Faraci, R.},
  title={On the deep question of pseudo-clefts},
  year={1970}
}
@book{BelnapSteel1976,
  author={Belnap, Jr., Nuel D. and Steel, Jr., Thomas B.},
  title={The logic of questions and answers},
  publisher={Yale University Press},
  year={1976}
}
@book{Vaanaanen2011,
  author={Jouko V\{a}\{a}n\{a}\{a}nen},
  title={Models and Games},
  publisher={Cambridge University Press},
  year={2011},
  volume={132},
  series={Cambridge studies in advanced mathematics}
}
@article{denDikkenMeinungerWilder2000,
  author={{den Dikken}, Marcel and Andr\'{e} Meinunger and Chris Wilder},
  title={Pseudoclefts and ellipsis},
  journal={Studia Linguistica},
  year={2000},
  volume={54},
  pages={41--89}
}
@article{Belnap1970,
  author={Belnap, Nuel},
  title={Conditional assertion and restricted quantification},
  journal={No\^{u}s},
  year={1970},
  volume={4},
  number={1},
  pages={1--12}
}
  
   ORG-CITEPROC TESTS


Richard Lawrence
  


Table of Contents
─

1 Org markup
.. 1.1 Simple citations
. 1.1.1 Parenthetical
. 1.1.2 In-text
. 1.1.3 With prefix and suffix data
. 1.1.4 Citations to works with tricky field data
.. 1.2 Multi-cite citations
. 1.2.1 Parenthetical, keys only
. 1.2.2 Parenthetical, with prefix and suffix data for individual works
. 1.2.3 Parenthetical, with common prefix and suffix data
. 1.2.4 All in-text, keys only
. 1.2.5 All in-text, with common prefix and suffix
2 References





1 Org markup


1.1 Simple citations


1.1.1 Parenthetical
╌╌╌

  Some great ideas occur in books (Brandom 1994). Others in articles
  (Hofweber 2007). Still others are in collections of previously
  published work (Russell 2001), or in conference proceedings (Rogers
  1996

  1   2   3   4   >