Re: [RFC PATCH] Changes to pop-up source buffers

2020-01-20 Thread Kyle Meyer
Jack Kamm  writes:

> My main motivation was to use my own display-buffer configuration to
> show the source buffer. So I've rewritten the patch to be smaller and
> more conservative, just adding a "plain" option to org-src-window-setup,
> and not changing the implementation of any existing options.  I think
> this is less likely to disrupt existing workflows or introduce
> accidental bugs.
>
> What do you think of using this smaller patch instead?

The more restricted patch is fine by me.

I suppose that to some degree [*] the main benefit of this patch is that
it offers an option that calls quit-restore-window.  For example,
without this patch, you can already configure things with
display-buffer-alist when org-src-window-setup is set to
`current-window'.  Say with

  (add-to-list 'display-buffer-alist
   '("^\\*Org Src" . (display-buffer-at-bottom)))

On exit, though, the window that was popped up remains.

And that makes me think that the current options that go through a
simple display-buffer-based call (current-window and other-window) would
benefit from calling quit-restore-window like your `plain` option does.
If you agree, perhaps it's worth adding another patch on top that does
that.

[*] "to some degree" because the option added by your patch has the
advantage that it'd work with display-buffer-base-action too.  Plus,
I think it's good to have a dedicated option that points to
display-buffer-alist/display-buffer-base-action.

> As an aside, in case we do decide to re-implement some of the display
> options, now or in future, I did have a slight discrepancy from the
> behavior you describe for split-window-right:
>
>> Quickly testing, this has a slight change in behavior.  If there is
>> already a window below the current Org buffer window, the new source
>> window will be popped up below the _other_ window rather than the Org
>> buffer.  I think this could be fixed (and the code in general
>> simplified) by using display-buffer-below-selected.
>
> On my own system, the window pops up below the existing Org buffer, even
> if I have several existing horizontal splits. I'm not sure why.

Hmm, weird.  I tried again (Emacs 26.3, vanilla config) and still see
the behavior I reported.  Oh well.

> Subject: [PATCH] org-src: Add option 'plain to org-src-window-setup
>
> * lisp/org-src.el (org-src-window-setup): Add option 'plain for
> org-src-window-setup, that uses vanilla display-buffer to show the
> source window.

My only minor nitpick is that, in the places you write “'plain”, it be
more common to drop the leading quote, as `plain' and ~plain~ already
suggest a symbol.  (No need to reroll for that if no one else requests
changes; I'll touch it up when applying.)

I'll wait another day or so for others to comment before applying.

Thanks.



Re: ob-scheme doesn't support :stdin?

2020-01-20 Thread Vladimir Nikishkin
I just thought that practically speaking, I no interpreter should
really need stdin, right?
Since everything that is prepended with a shebang: #! interpreter
is at the same time a valid shell script, isn't it?

сб, 18 янв. 2020 г. в 13:02, Kyle Meyer :
>
> Vladimir Nikishkin  writes:
>
> > Is it true that ob-scheme doesn't support :stdin ?
> >
> > I just tried, and doesn't seem to work, although it works with ob-shell.
> >
> > (This is not a complaint, I just would like to confirm that I
> > understand things correctly.)
>
> Yes, that's true as far as I can tell:
>
>   $ git grep :stdin
>   lisp/ob-awk.el:;; - :stdin takes an Org data or code block reference, the 
> value of
>   lisp/ob-awk.el:  (stdin (let ((stdin (cdr (assq :stdin params
>   lisp/ob-sed.el:  (stdin (let ((stdin (cdr (assq :stdin params
>   lisp/ob-shell.el:(stdin (let ((stdin (cdr (assq :stdin params
>   testing/examples/ob-awk-test.org:#+begin_src awk  :stdin genseq :results 
> silent
>   testing/examples/ob-sed-test.org:  #+BEGIN_SRC sed :stdin ex1
>
> So it seems support for :stdin is limited to awk, sed, and shell.



-- 
Yours sincerely, Vladimir Nikishkin



Re: `org-next-link' skips links inside PROPERTIES drawer

2020-01-20 Thread Kyle Meyer
Samuel Wales  writes:

> is isearch intended to be also affected?  or just org-next-link?  i
> suppose isearch is a generic emacs thing, just as lgrep would be
> generic, and most folk would want it to be generic and would get too
> confused if it were not, but just want to make sure.

No, isearch isn't affected.

> i have noticed a similar issue.  i wonder if it is related to the next
> link issue, or completely different and related to text properties or
> overlays or something?
>
> namely, isearch will not find google in the following link:
>
> [[http://google.com][go]]
>
> is there a hack to make isearch find links?  even though they are
> hidden?

Here are two options:

  * Set search-invisible to t.  (The default value of `open' won't allow
you to match links.)

  * Make the links visible with org-toggle-link-display before
searching.



[PATCH] Fix several issues with python session value blocks

2020-01-20 Thread Jack Kamm
This patch fixes several related issues with python blocks with
parameters ":session :results value", including:

- Broken if-else and try-except statements.
- Correctly parsing blank lines in indented blocks.
- Returning the correct value when the underscore "_" variable
  has been assigned.

It works by using the built-in ast python module to parse the source
block, then executes it, evaluating the last line separately to store
the result.

Note this patch also creates a slight change in behavior: the final
result must be a top-level expression, otherwise we return "None" as the
result.

There is some useful background and discussion of the issues here:

https://lists.gnu.org/archive/html/emacs-orgmode/2017-11/msg00274.html

In that thread from 2017, we solved similar problems for the ":results
output" case. With this patch, I'm now circling back to try and fix the
":results value" case.

I've checked this patch works for both Python2 and Python3, as well as
for IPython.

Here are some examples that this patch fixes. I've also added them as
unit tests:

This block should return 20, but previously it raised an
IndentationError and then returned 10:

#+begin_src python :session :results value
  foo = 0
  for i in range(10):
  foo += 1

  foo += 1

  foo
#+end_src

This block should return "success", but previously it raised a
SyntaxError and then returned "failure":

#+begin_src python :session :results value
  value = "failure"
  if False:
  pass
  else:
  value = "success"
  value
#+end_src

This block should return "success", but previously it returned "failure":

#+begin_src python :session :results value
  _ = "failure"
  "success"
#+end_src

>From 0d8f35d8a34c6297ade50f321f79b07a3cf8d5aa Mon Sep 17 00:00:00 2001
From: Jack Kamm 
Date: Mon, 20 Jan 2020 17:40:22 -0800
Subject: [PATCH] ob-python: Fix several issues with :session :results value

* lisp/ob-python.el (org-babel-python-evaluate-session): Fix a few
related issues with :session :results value blocks, including broken
if-else statements, indented blocks with blank lines, and returning
the wrong value when underscore has been used.  Uses the built-in ast
python module to parse a source block, execute it, and evaluate the
last line separately to return as a result.  Introduces a slight
change in behavior, requiring that the last line must be a top-level
statement if it's result is to be saved (otherwise, the result is
None).
---
 lisp/ob-python.el  | 64 ++
 testing/lisp/test-ob-python.el | 35 +++
 2 files changed, 69 insertions(+), 30 deletions(-)

diff --git a/lisp/ob-python.el b/lisp/ob-python.el
index 823f6e63d..a610c471f 100644
--- a/lisp/ob-python.el
+++ b/lisp/ob-python.el
@@ -247,6 +247,20 @@ open('%s', 'w').write( pprint.pformat(main()) )")
")); "
"__org_babel_python_fh.close()"))
 
+(defconst org-babel-python--eval-tmpfile "import ast
+with open('%s') as f:
+__org_babel_python_ast = ast.parse(f.read())
+
+__org_babel_python_final = __org_babel_python_ast.body[-1]
+if type(__org_babel_python_final) == ast.Expr:
+__org_babel_python_ast.body = __org_babel_python_ast.body[:-1]
+exec(compile(__org_babel_python_ast, '', 'exec'))
+__org_babel_python_final = eval(compile(ast.Expression(
+__org_babel_python_final.value), '', 'eval'))
+else:
+exec(compile(__org_babel_python_ast, '', 'exec'))
+__org_babel_python_final = None")
+
 (defun org-babel-python-evaluate
   (session body  result-type result-params preamble)
   "Evaluate BODY as Python code."
@@ -294,34 +308,10 @@ If RESULT-TYPE equals `output' then return standard output as a
 string.  If RESULT-TYPE equals `value' then return the value of the
 last statement in BODY, as elisp."
   (let* ((send-wait (lambda () (comint-send-input nil t) (sleep-for 0 5)))
-	 (dump-last-value
-	  (lambda
-	(tmp-file pp)
-	(mapc
-	 (lambda (statement) (insert statement) (funcall send-wait))
-	 (if pp
-		 (list
-		  "import pprint"
-		  (format "open('%s', 'w').write(pprint.pformat(_))"
-			  (org-babel-process-file-name tmp-file 'noquote)))
-	   (list (format "open('%s', 'w').write(str(_))"
-			 (org-babel-process-file-name tmp-file
-  'noquote)))
 	 (last-indent 0)
 	 (input-body (lambda (body)
-		   (dolist (line (split-string body "[\r\n]"))
-			 ;; Insert a blank line to end an indent
-			 ;; block.
-			 (let ((curr-indent (string-match "\\S-" line)))
-			   (if curr-indent
-			   (progn
- (when (< curr-indent last-indent)
-   (insert "")
-   (funcall send-wait))
- (setq last-indent curr-indent))
-			 (setq last-indent 0)))
-			 (insert line)
-			 (funcall send-wait))
+		   (mapc (lambda (line) (insert line) (funcall send-wait))
+			 (split-string body "[\r\n]"))
 		   (funcall send-wait)))
  (results
   (pcase result-type
@@ -344,17 +334,31 @@ last 

Re: `org-next-link' skips links inside PROPERTIES drawer

2020-01-20 Thread Samuel Wales
thanks from here too!  interesting.


is isearch intended to be also affected?  or just org-next-link?  i
suppose isearch is a generic emacs thing, just as lgrep would be
generic, and most folk would want it to be generic and would get too
confused if it were not, but just want to make sure.

i have noticed a similar issue.  i wonder if it is related to the next
link issue, or completely different and related to text properties or
overlays or something?

namely, isearch will not find google in the following link:

[[http://google.com][go]]

is there a hack to make isearch find links?  even though they are
hidden?  and find everythign else that isearch does not find, if there
is anythign else?

in 9.1.14, a link like  will be found by
org-next-link, so perhaps it is more recent than that.

[now i wodner if whatever kludge i put someplace that makes links in
comments fontified or... clickable... or something... is not needed in
more recent org.]


On 1/20/20, Pedro R.M. Júnior  wrote:
> Thanks for the
> clarification and quoted suggestion.



Re: `org-next-link' skips links inside PROPERTIES drawer

2020-01-20 Thread Pedro R . M . Júnior
Hello Kyle,

> This behavior came up in a previous thread.  Quoting Nicolas:
>
> ,[ https://lists.gnu.org/archive/html/emacs-orgmode/2019-04/msg00075.html 
> ]
> | This is expected.
> |
> | The value in a properties drawer cannot be a link, even though
> | `org-open-at-point' would normally open it, as is done for comments.
> |
> | In addition to no being syntactically a valid link, I think it is
> | undesirable for an interactive command to display hidden internal data.
> |
> | If you want to reach it, just write a function calling
> |
> |(re-search-forward org-any-link-re nil t)
> `

In fact, this have passed unobserved to me.  Thanks for the
clarification and quoted suggestion.

Bests,
Pedro



org-mode functional programming library

2020-01-20 Thread Dwarshuis, Nathan J
Hello,

I recently authored an package called "om.el" which is a functional org-mode 
API akin to dash.el primarily using org-element. Briefly, it provides a library 
of (mostly) pure functions that manipulate the parse tree generated by 
org-element.el, and uses this to either edit or query the buffer with all the 
advantages of functional programming (eg lack of side effects, referential 
transparency, easier testing, etc). The github repo for om.el is here: 
https://github.com/ndwarshuis/om.el.

I'm posting to the mailing list a) for general feedback on this package and b) 
because I am wondering if this would be a good package to include with org-mode 
itself rather than in another repository such as MELPA. The code for om.el is 
tightly integrated with org-element.el and it might make sense for development 
between these to be closely intertwined.

There is also an open submission for this to MELPA and the discussion is here: 
https://github.com/melpa/melpa/pull/6623.

Thank you,
Nathan Dwarshuis






Re: `org-next-link' skips links inside PROPERTIES drawer

2020-01-20 Thread Kyle Meyer
Hi Pedro,

Pedro R.M. Júnior  writes:

> I am testing Emacs 28 (Org 9.3) from the Git repository and I have
> noticed that, compared to version 26.3 (Org 9.1.9), `org-next-link'
> function is skipping links located inside the PROPERTIES drawer.
> [...]
> Let me know if this is an intentional behavior for newer versions of Org
> mode.  If so, I think it could receive a customization option for this
> behavior.

This behavior came up in a previous thread.  Quoting Nicolas:

,[ https://lists.gnu.org/archive/html/emacs-orgmode/2019-04/msg00075.html ]
| This is expected.
| 
| The value in a properties drawer cannot be a link, even though
| `org-open-at-point' would normally open it, as is done for comments.
| 
| In addition to no being syntactically a valid link, I think it is
| undesirable for an interactive command to display hidden internal data.
| 
| If you want to reach it, just write a function calling
| 
|(re-search-forward org-any-link-re nil t)
`






`org-next-link' skips links inside PROPERTIES drawer

2020-01-20 Thread Pedro R . M . Júnior
Hello,

I am testing Emacs 28 (Org 9.3) from the Git repository and I have
noticed that, compared to version 26.3 (Org 9.1.9), `org-next-link'
function is skipping links located inside the PROPERTIES drawer.  (I
have not tested version 27.)  The same does not happen for the links
outsize drawers or links inside a drawer other than PROPERTIES one,
e.g., LOGBOOK drawer.

As an example, consider the Org file between "=" lines
below.  Set the cursor at the beginning of the file.  Then call
`org-next-link'.

=
* Heading
  :PROPERTIES:
  :MYLINK:   https://orgmode.org/
  :END:
  Another link: https://duckduckgo.com/
=

I would expect `org-next-link' to find the link inside the drawer (i.e.,
https://orgmode.org/) as in version 26.3.  Instead, it goes directly to
the second link (i.e., https://duckduckgo.com/).

The same happens with `org-previous-link'.

Let me know if this is an intentional behavior for newer versions of Org
mode.  If so, I think it could receive a customization option for this
behavior.

Bests,
Pedro R.M. Júnior



Re: Src block fontification when scrolled off window

2020-01-20 Thread Aaron Jensen
On Tue, Jan 14, 2020 at 8:20 AM Fraga, Eric  wrote:
>
> On Tuesday, 14 Jan 2020 at 07:21, Aaron Jensen wrote:
> > In one of my org files, I will occasionally see src blocks lose their
> > formatting. [...]
> >
> > Is this expected behavior? Is there a way to increase the lookback amount?
>
> I don't see this behaviour.  You might want to look at
> jit-lock-chunk-size but that's me grasping at straws...

I'm trying this out. I've also started opening src edit blocks in a
different window. I haven't seen the behavior since making these
changes, so I'm happy. Thanks!



Re: export and split orgmode headers into separate md files?

2020-01-20 Thread Diego Zamboni
Hi Z,

(replying to the list)

I have put together a simplified function based on the code from
https://medium.com/@lakshminp/publishing-a-book-using-org-mode-9e817a56d144,
which simply exports each top-level header to a plain Markdown file. You
can find it here:
https://gist.github.com/zzamboni/2e6ac3c4f577249d98efb224d9d34488

I have added some commentary in the code about what it does. After
evaluating the code, you just need to call M-x org-multi-file-md-export in
the buffer you want to export.

I hope this helps in getting you started!

Cheers,
--Diego

On Wed, Jan 15, 2020 at 2:48 PM Xebar Saram  wrote:

> Thx Deigo for the detailed answer! Really appreciate it!
>
> i think though it’s a bit over my head as i have limited lisp knowledge :)
>
> Does this export to standard markdown files or leanpub format? Honestly at
> this stage im willing to export to any flavor of markdown but basic or MMD
> would work best. Any tips on how i can adjust the code for that?
> if not can i just run your code to get a different markdown format? What
> the main function you run after evaluating the code in emacs?
> i apologize in advance for the silly questions :)
>
> thx again
>
> Z
>
> On Sun, Jan 12, 2020 at 7:20 AM Diego Zamboni  wrote:
>
>> Hi Z,
>>
>> I do something similar in my ox-leanpub-book module [1], which exports
>> each top-level heading to a different file. The general idea is to use
>> =org-map-entries= to loop over the entire buffer [2]. The function you call
>> can then check whether the current entry is a header at the level you want
>> [3] and then export it to the corresponding file. The title can be used to
>> deduct the filename [4].
>>
>> I found that I had to mark the entire subtree before calling the export
>> function [5], otherwise the headline was not getting included in the export.
>>
>> I based my code originally on this blog post, which might be a simpler
>> starting point:
>> https://medium.com/@lakshminp/publishing-a-book-using-org-mode-9e817a56d144
>> - this code does not select the entire subtree before exporting, which
>> means only the contents of the section is exported, but not the headline
>> itself.
>>
>> Hope this helps!
>> --Diego
>>
>> [1] https://github.com/zzamboni/ox-leanpub/tree/book-and-markua
>> [2]
>> https://github.com/zzamboni/ox-leanpub/blob/book-and-markua/ox-leanpub-book.el#L185-L186
>> [3]
>> https://github.com/zzamboni/ox-leanpub/blob/book-and-markua/ox-leanpub-book.el#L125
>> [4]
>> https://github.com/zzamboni/ox-leanpub/blob/book-and-markua/ox-leanpub-book.el#L131-L135
>> [5]
>> https://github.com/zzamboni/ox-leanpub/blob/book-and-markua/ox-leanpub-book.el#L170-L174
>>
>>
>> On Sun, Jan 12, 2020 at 4:54 AM Xebar Saram  wrote:
>>
>>> Hi all
>>>
>>> For work specific needs at uni i have a need to take a comprehensive org
>>> file with hundreds of headers and split each header into separate .md files
>>> (with the header name as file name//first header in the md file).
>>> Has anyone done anything remotely similar? Or if not can someone point
>>> me in the right direction on how to even start dealing with this?
>>>
>>> thx a lot in advance any tips would be very much appreciated
>>>
>>> kind regards
>>>
>>> Z
>>>
>>


Re: breakage: Using self-defined Macro in macro definition

2020-01-20 Thread Berry, Charles


> On Jan 20, 2020, at 2:27 AM, Robert Klein  wrote:
> 
> 
> Hi,
> 
> when I use a self-defined macro in a macro definition, subsequent
> macros in the same macro definition don't get expanded (tested with
> org version 9.2.1 and tip of maint):


The expansions in your example follow the rules. 

The problem as explained below is that you have created a macro whose expansion 
cannot be further expanded.

> 
> --- snip example ---
> #+Macro: newline (eval "\n")

That will add a newline character.

> #+Macro: new $1 {{{newline}}}#+Index: $1 {{{newline}}}

This is a bit tricky. It adds a newline character before the hash `#'.

Then you have this line in the current buffer (see the 
`org-export-with-buffer-copy' docstring):

#+Index: $1 {{{newline}}} 

and org needs to do something with it, but as the manual says

Org recognizes macro references in following Org markup areas:
paragraphs, headlines, verse blocks, tables cells and lists.  Org also
recognizes macro references in keywords, such as ‘CAPTION’, ‘TITLE’,
‘AUTHOR’, ‘DATE’, and for some back-end specific export options.

and INDEX was not one of the keywords, so the reference is not recognized. 

So it is not expanded. This has nought to do with it being `self-defined'.

Of course, you see it on export if the exporter deals with INDEX (it looks like 
you are using 'latex).

> 
> Use the {{{new(format)}}}
> command to format a string according to the
> /format-string/ argument.
> --- snip example ---
> 
> 
> the output of which is:
> 
> --- snip resulting output ---
> Use the format a 
> \index{format {{{newline
> command to format a string according to the
> \emph{format-string} argument.
> --- snip resulting output ---
> 
> 

As expected.

> The expected output would be:
> 
> --- snip expected output ---
> Use the format a 
> \index{format} 
> command to format a string according to the
> \emph{format-string} argument.
> --- snip expected output ---
> 
> 
> PS: leaving the second {{{newline}}} out is not a solution, as
> paragraph reformatting will put the macro in the middle of the line.
> 
> 
> 
> The issue doesn't crop up, when using a predefined macro, e.g. ` date'
> or `author'.
> 
> 
> It also doesn't show up, when the first macro in the macro is e.g. the
> predefined macro `date'.  That is the following example 2 works ok:
> 
> --- snip example 2 ---
> #+Date: <2020-01-20 Mon>
> #+Macro: old $1 {{{date}}} {{{newline}}} alpha {{{newline}}} beta
> 
> {{{old}}}
> --- snip example 2 ---
> 
> 
> Thanks for any hints/help.
> 

None of those examples involve a macro that creates output in which further 
macros cannot be recognized.

Build an `eval' macro that does all the expansions within its own code without 
referring to other macros. The property API may help with this.

Or maybe try an idiom that does not use a macro, such as a custom link.

HTH,

Chuck

breakage: Using self-defined Macro in macro definition

2020-01-20 Thread Robert Klein


Hi,

when I use a self-defined macro in a macro definition, subsequent
macros in the same macro definition don't get expanded (tested with
org version 9.2.1 and tip of maint):

--- snip example ---
#+Macro: newline (eval "\n")
#+Macro: new $1 {{{newline}}}#+Index: $1 {{{newline}}}

Use the {{{new(format)}}}
command to format a string according to the
/format-string/ argument.
--- snip example ---


the output of which is:

--- snip resulting output ---
Use the format a 
\index{format {{{newline
command to format a string according to the
\emph{format-string} argument.
--- snip resulting output ---


The expected output would be:

--- snip expected output ---
Use the format a 
\index{format} 
command to format a string according to the
\emph{format-string} argument.
--- snip expected output ---


PS: leaving the second {{{newline}}} out is not a solution, as
paragraph reformatting will put the macro in the middle of the line.



The issue doesn't crop up, when using a predefined macro, e.g. ` date'
or `author'.


It also doesn't show up, when the first macro in the macro is e.g. the
predefined macro `date'.  That is the following example 2 works ok:

--- snip example 2 ---
#+Date: <2020-01-20 Mon>
#+Macro: old $1 {{{date}}} {{{newline}}} alpha {{{newline}}} beta

{{{old}}}
--- snip example 2 ---


Thanks for any hints/help.

Best regards
Robert