Re: [O] ob-clojure with tangling current broken

[Frederick Giasson]

Hi,

That's why I added a patch recently. Now Nicolas already applied my
> patch. You should already seen it now.
>

Yeah I just received that email right after I sent mine. Didn't know it was
in the pipeline, I just re-subscribed to the mailing list.


> >
> > It turns out that the culprit is the following line:
> > https://code.orgmode.org/bzg/org-mode/src/master/lisp/ob-clojure.el#L108
> >
> > I am wondering why is there such code injection in
> > "org-babel-expand-body:clojure". It looks like to be related to the
> Clojure code
> > block execution, but it also appears to be called from the tangling
> function.
> > Clearly those two different use cases needs to be properly handled at
> the level
> > of "ob-clojure.el"
> >
>
> Expanding clojure code is necessary to tangling, because when user have
> noweb reference etc in source block, expanding here is necessary.
>

Yes, that is right. What I was referring to I guess is to handle that case
(tangling vs. code block execution within Emacs), which is what this patch
is about, thanks!


>
> > I didn't follow all the changes to the "ob-clojure" mode in the last
> year or so,
> > so please pardon my ignorance if the module is taking a different
> direction than
> > I am used to. Is there a rational behind this behaviour or is there a
> new feature
> > that I am not aware of which properly handle those two use cases?
> >
>
> Which part you don't understand, I might can answer your question,
> because I did most of those changes in last year.
>

The only thing I meant here is that the last time I looked into this code,
it was quite different, you guys appears to have worked a lot on it.

Another thing I discovered is the =ob-clojure-literate=, but even after
reading its [sparse] doc and its code, I am not sure what it concretely
adds to =ob-clojure=

Thanks for this work!

Take care,

Fred

[O] ob-clojure with tangling current broken

[Frederick Giasson]

Hi,

I updated org-mode to latest dev version recently and I was wondering why I
was seeing namespaces appended to each [clojure] code block I was tangling
(which generates unusable tangled Clojure source files)

It turns out that the culprit is the following line:
https://code.orgmode.org/bzg/org-mode/src/master/lisp/ob-clojure.el#L108

I am wondering why is there such code injection in
"org-babel-expand-body:clojure". It looks like to be related to the Clojure
code block execution, but it also appears to be called from the tangling
function. Clearly those two different use cases needs to be properly
handled at the level of "ob-clojure.el"

I didn't follow all the changes to the "ob-clojure" mode in the last year
or so, so please pardon my ignorance if the module is taking a different
direction than I am used to. Is there a rational behind this behaviour or
is there a new feature that I am not aware of which properly handle those
two use cases?

Thanks,

Take care,

Fred

Re: [O] org-detangle seems broken

[Frederick Giasson]

Hi


>
> > block to its tangled source is created, then we could use it with that
> option in `org-babel-detangle` to switch back to the source if it is the
> wish of the user.
>
> It sounds like org-babel-tangle-jump-to-org is what you want.
>

In fact it is its opposite that I need, code block to tangled source file.

Fred

>

Re: [O] org-detangle seems broken

[Frederick Giasson]

Hi Grant,


I think it is expected. The code looks like this:
>
> (when (setq new-body (org-babel-tangle-jump-to-org))
>   (org-babel-update-block-body new-body))
>

Yeah not sure I fully understand that code. Basically it does this for each
code block that needs to be detangled right? I am just not sure why the
focus doesn't return back to the source at the end of the process.


> The first time I detangled I was surprised, too.
>
> But then it hit me: if you are detangling then you probably might want to
> 1. See the updated code
> 2. Commit the updated code
>

Yes, I agree with you. However on a team that work on a LP project,
different people have different preferences and workflows. Personally I
only code in the Org-file, and toggle a code block to its major mode if I
need something provided by the major mode (coding in Clojure). However,
some people prefer see the full namespace (one namespace per Org file)
directly in Cider (the major mode) and detangle from there (this is why I
got interested in detangling).

In their case, having the buffer switching back to the Org file is a bit
irritating, particularly since there is no (yet) any function that let you
switch to the tangle source file of a code block.

If this is really the behavior, then I think that a new option should be
added to it such that we can choose one behavior or the other. If a
function that jump from a code block to its tangled source is created, then
we could use it with that option in `org-babel-detangle` to switch back to
the source if it is the wish of the user.

Thoughts?


Thanks,

Fred

Re: [O] org-detangle seems broken

[Frederick Giasson]

Hi Nicolas,

> Here it is. If you =detangle= the =broken-detangle--working.el= file, it
> > will work as expected.
> >
> > However, if you try to detangle the =broken-detangle--NOT-working.el=
> file,
> > you should get a "Not in a source block" error in *message*.
>
> This is fixed in "master".
>

That is great! Working perfectly no, thanks!



> > Another thing I noted while creating the ECM is if there is only *one*
> code
> > block in the =.org= file, then it throws a =End of Buffer= error.
>
> I couldn't reproduce this. If the fix below doesn't solve it, could you
> provide a dedicated ECM?
>

Yes, that fixed it as well.



There is one last thing related to =org-babel-detangle= that seems "broken"
is that as far as I can see, the top =save-excursion= should save the
position of the cursor of the source file and keep the buffer at that
position when we perform a detangling on it, no?

Right now, the behavior is that when we do a detangle from a source file,
the org file get opened in the buffer of the source file.

However, I think that the right behavior is that when we detangle that the
remain at the same position in the source file otherwise this gets
confusing in my opinion.

Is that a bug or the expected behavior?

Thanks!

Take care

Fred

Re: [O] org-detangle seems broken

[Frederick Giasson]

Hi Nicolas,

Here it is. If you =detangle= the =broken-detangle--working.el= file, it
will work as expected.

However, if you try to detangle the =broken-detangle--NOT-working.el= file,
you should get a "Not in a source block" error in *message*.

Another thing I noted while creating the ECM is if there is only *one* code
block in the =.org= file, then it throws a =End of Buffer= error.



On Sat, Jun 2, 2018 at 6:17 AM, Nicolas Goaziou 
wrote:

> Hello,
>
> Frederick Giasson  writes:
>
> > It appears that org-detangle is broken. I am using spacemacs, but even
> with
> > a vanilla Emacs (emacs -Q) it appears to be broken.
> >
> > When I tangle a file with =:comments link= every things works as expected
> > and the comments are added in the tangled file.
> >
> > When I just detangle that tangled file, everything get detangled as
> > expected.
> >
> > If I change one or two lines, it normally works as expected.
> >
> > If I change more than 4 lines, I always get the error message "Not in a
> > block file".
> >
> > Some debugging shows that the function =org-element-at-point= sometimes
> tag
> > the element as =paragraph= instead of =src-block=, like if it doesn't
> know
> > where it is... somehow.
> >
> > Does anybody use that function without experiencing this (apparent major)
> > issue?
>
> Could you provide an ECM demonstrating the issue?
>
> Thank you.
>
> Regards,
>
> --
> Nicolas Goaziou
>


broken-detangle.org
Description: Binary data
;; [[file:~/Documents/git/broken-detangle.org::*Experience%20broken%20detangle][Experience broken detangle:1]]
(defun test
  (message "broken detangle")
  (message "test")
  (message "test"))
;; Experience broken detangle:1 ends here

;; [[file:~/Documents/git/broken-detangle.org::*Experience%20broken%20detangle][Experience broken detangle:2]]
(defun test
(message "broken detangle")
  (message "test"))
;; Experience broken detangle:2 ends here

;; [[file:~/Documents/git/broken-detangle.org::*Experience%20broken%20detangle][Experience broken detangle:3]]
(defun test
(message "broken detangle")
(message "test")
(message "test"))
;; Experience broken detangle:3 ends here
;; [[file:~/Documents/git/broken-detangle.org::*Experience%20broken%20detangle][Experience broken detangle:1]]
(defun test
(message "broken detangle"))
;; Experience broken detangle:1 ends here

;; [[file:~/Documents/git/broken-detangle.org::*Experience%20broken%20detangle][Experience broken detangle:2]]
(defun test
(message "broken detangle")
  (message "test"))
;; Experience broken detangle:2 ends here

;; [[file:~/Documents/git/broken-detangle.org::*Experience%20broken%20detangle][Experience broken detangle:3]]
(defun test
(message "broken detangle")
(message "test")
(message "test"))
;; Experience broken detangle:3 ends here

[O] org-detangle seems broken

[Frederick Giasson]

Hi Everybody,

It appears that org-detangle is broken. I am using spacemacs, but even with
a vanilla Emacs (emacs -Q) it appears to be broken.

When I tangle a file with =:comments link= every things works as expected
and the comments are added in the tangled file.

When I just detangle that tangled file, everything get detangled as
expected.

If I change one or two lines, it normally works as expected.

If I change more than 4 lines, I always get the error message "Not in a
block file".

Some debugging shows that the function =org-element-at-point= sometimes tag
the element as =paragraph= instead of =src-block=, like if it doesn't know
where it is... somehow.

Does anybody use that function without experiencing this (apparent major)
issue?

Thanks,

Take care,

Fred

[O] Issues with org-babel-detangle and :comments noweb

[Frederick Giasson]

Hi Everybody,

I am trying to have noweb references working with org-babel-detangle. Let's
use this example:

```
#+NAME: a
#+BEGIN_SRC clojure

(def a "a")

#+END_SRC

#+NAME: b
#+BEGIN_SRC clojure

(def a "b")

#+END_SRC

#+NAME: c
#+BEGIN_SRC clojure

(def a "c")

#+END_SRC

#+NAME: a
#+BEGIN_SRC clojure

(def a "a")

#+END_SRC


#+NAME: test
#+BEGIN_SRC clojure :tangle test.clj :mkdirp yes :noweb yes :padline yes
:results silent :comments noweb

<>
<>
<>

#+END_SRC

```


Once tangled, the test.clj file looks like:

```
;; [[file:test.org::test][test]]
;; [[file:test.org::a][a]]

(def a "a")

;; a ends here
;; [[file:test.org::a][a]]

(def a "a")

;; a ends here
;; [[file:test.org::b][b]]

(def a "b")

;; b ends here
;; [[file:test.org::c][c]]

(def a "c")

;; c ends here
;; test ends here
```

If I perform org-babel-tangle-jump-to-org  everything works fine, I jump to
the block codes.

However if I perform org-babel-detangle, when I get the message "Not in
tangle code". The problem appears to be that the references are "embedded"
in the tangled block code. If I move the "test ends here" above, then it
works, but then when the tangled block code get detangled, I loose the
noweb references.

Any idea if there is a work around for that? I would like to use
org-babel-tangle-jump-to-org and org-babel-detangle while being about to
change the order of the code within the block codes in the tangled file.

Thanks!

Take care,

Fred

Re: [O] [PATCH] New header parameter :show-process for Org-babel-clojure

[Frederick Giasson]

Hi Nicolas,


Could you recreate the commit so it is self-contained?


That should be it.

Thanks,

Fred

>From e19c358f9d4cab9b22c3e4e3bf6b8f6298e61b0e Mon Sep 17 00:00:00 2001
From: Frederick Giasson <f...@fgiasson.com>
Date: Fri, 18 Nov 2016 12:01:02 -0500
Subject: [PATCH] Multiple improvements of ob-clojure have been performed in
 this commit.

  (1) better error handling. Errors and exceptions are output in the results 
section of a org-mode document. They were silenced with the previous version.
  (2) A new header parameter has been created: ":show-process" what will output 
the underlying NREPL processing in a new window + buffer
  (3) new setting "org-babel-clojure-sync-nrepl-timeout" to specify the Cider 
nrepl timeout. 'nil' means that it will never timeout

Note that the behavior of ob-clojure *did not change* with these modications.

Only the function (org-babel-execute:clojure) has been modified.
---
 etc/ORG-NEWS   |  42 ++-
 lisp/ob-clojure.el | 117 ++---
 2 files changed, 134 insertions(+), 25 deletions(-)

diff --git a/etc/ORG-NEWS b/etc/ORG-NEWS
index a360e35..5a5be53 100644
--- a/etc/ORG-NEWS
+++ b/etc/ORG-NEWS
@@ -23,8 +23,48 @@ into
 
 : (file (lambda () (sexp)))
 
-
 ** New features
+*** Babel
+ Org-babel-clojure, new setting 'org-babel-clojure-sync-nrepl-timeout'
+
+Creation of a new setting to specify the Cider timeout. By setting the 
=org-babel-clojure-sync-nrepl-timeout=
+setting option. The value is in seconds and if set to =nil= then no timeout 
will occur.
+
+#+BEGIN_SRC emacs-lisp
+  ; Disable Cider's timeout
+  (setq org-babel-clojure-sync-nrepl-timeout nil)
+
+  ; Set the timeout to 20 seconds
+  (setq org-babel-clojure-sync-nrepl-timeout 20)
+#+END_SRC
+
+
+ Org-babel-clojure :show-process
+A new block code header has been created for Org-babel-clojure that enables 
developers to output the process of an ongoing process into a new window/buffer.
+
+You can tell Org-babel-clojure to output the process of a running code block.
+
+To show that output you only have to specify the =:show-process= option in the 
code block's header like this:
+
+#+begin_example
+#+BEGIN_SRC clojure :results output :show-process
+  (dotimes [n 10]
+(println n ".")
+(Thread/sleep 500))
+#+END_SRC
+#+end_example
+
+If =:show-process= is specified that way, then when you will run the
+code using =C-c C-c= a new window will open in Emacs. Everything that is
+output by the REPL will immediately be added to that new window.
+
+When the processing of the code is finished, then the window and its buffer 
will be closed and the results will be reported in the
+=#+RESULTS= section. 
+
+Note that the =:results= parameter's behavior is *not* changed. If =silent= is 
specified, then no result will be displayed. If =output= is
+specified then all the output from the window will appears in the
+results section. If =value= is specified, then only the last returned
+value of the code will be displayed in the results section.
 
 *** Horizontal rules are no longer ignored in LaTeX table math mode
 
diff --git a/lisp/ob-clojure.el b/lisp/ob-clojure.el
index 72ea77d..070e0ca 100644
--- a/lisp/ob-clojure.el
+++ b/lisp/ob-clojure.el
@@ -2,7 +2,7 @@
 
 ;; Copyright (C) 2009-2016 Free Software Foundation, Inc.
 
-;; Author: Joel Boehland, Eric Schulte, Oleh Krehel
+;; Author: Joel Boehland, Eric Schulte, Oleh Krehel, Frederick Giasson
 ;;
 ;; Keywords: literate programming, reproducible research
 ;; Homepage: http://orgmode.org
@@ -45,6 +45,9 @@
 (declare-function cider-current-connection "ext:cider-client" ( type))
 (declare-function cider-current-session "ext:cider-client" ())
 (declare-function nrepl-dict-get "ext:nrepl-client" (dict key))
+(declare-function nrepl-dict-put "ext:nrepl-client" (dict key value))
+(declare-function nrepl--merge "ext:nrepl-client" (dict1 dict2))
+(declare-function nrepl-request:eval "ext:nrepl-client" (input callback 
connection))
 (declare-function nrepl-sync-request:eval "ext:nrepl-client"
  (input connection session  ns))
 (declare-function org-trim "org" (s  keep-lead))
@@ -56,6 +59,15 @@
 (defvar org-babel-default-header-args:clojure '())
 (defvar org-babel-header-args:clojure '((package . :any)))
 
+(defcustom org-babel-clojure-sync-nrepl-timeout 10
+  "Timeout value, in seconds, of a Clojure sync call. 
+   If the value is nil, timeout is disabled."
+  :type 'integer
+  :version "25.1"
+  :package-version '(Org . "9.0")  
+  :safe #'wholenump  
+  :group 'org-babel)
+
 (defcustom org-babel-clojure-backend
   (cond ((featurep 'cider) 'cider)
(t 'slime))
@@ -84,34 +96,91 @@
   body)))
 
 (defun org-babel-execute:clojure (body params)
-  "Execute a block of Clojure code with Babel."
+  "Execute a block of C

Re: [O] [PATCH] New header parameter :show-process for Org-babel-clojure

[Frederick Giasson]

Hi Guys,


It looks like this warning results from a defcustom for this variable
not being included in the patch (and AFAICT it should be).


Sorry but I was confused. The documentation for this setting was 
documented in Worg, but it was not yet part of the package.


Here is the commit related to that.

Thanks,

Fred
>From e68c5611f00eb37783205352d68efb7feccae8ab Mon Sep 17 00:00:00 2001
From: Frederick Giasson <f...@fgiasson.com>
Date: Fri, 18 Nov 2016 10:29:16 -0500
Subject: [PATCH] Adding the defcustom for the
 "org-babel-clojure-sync-nrepl-timeout" variable used to configure the nrepl
 execution timeout.

---
 lisp/ob-clojure.el | 9 +
 1 file changed, 9 insertions(+)

diff --git a/lisp/ob-clojure.el b/lisp/ob-clojure.el
index f78dd69..6dc9d90 100644
--- a/lisp/ob-clojure.el
+++ b/lisp/ob-clojure.el
@@ -56,6 +56,15 @@
 (defvar org-babel-default-header-args:clojure '())
 (defvar org-babel-header-args:clojure '((package . :any)))
 
+(defcustom org-babel-clojure-sync-nrepl-timeout 10
+  "Timeout value, in seconds, of a Clojure sync call. 
+   If the value is nil, timeout is disabled."
+  :type 'integer
+  :version "25.1"
+  :package-version '(Org . "9.0")  
+  :safe #'wholenump  
+  :group 'org-babel)
+
 (defcustom org-babel-clojure-backend
   (cond ((featurep 'cider) 'cider)
(t 'slime))
-- 
1.9.5.msysgit.0

Re: [O] [PATCH] New header parameter :show-process for Org-babel-clojure

[Frederick Giasson]

Hi Nicolas,

   In end of data:
   ob-clojure.el:180:1:Warning: the following functions are not known to be
   defined: nrepl-request:eval, nrepl--merge, nrepl-dict-put

Could you look into it? In particular, "nrepl--merge" looks like an
internal function from nrepl. Is it safe to use it?


Yes, these functions are part of Cider's nrepl. Yes, they should be 
safe. Also, they are only used in a context where the code block uses 
Cider as the backend, so I guess it is right to use them in that 
context. Thoughts?


Thanks,

Fred

Re: [O] [PATCH] New header parameter :show-process for Org-babel-clojure

[Frederick Giasson]

Hi,


It looks like this warning results from a defcustom for this variable
not being included in the patch (and AFAICT it should be).


Humm, I thought that it was already part of Org-mode 8.x (something I 
submitted a way back). Let me check that.


If I have to add it again, should I recreate the commit or simply submit 
another patch?


Thanks,

Fred

Re: [O] [PATCH] New header parameter :show-process for Org-babel-clojure

[Frederick Giasson]

Hi Nicolas,


Great! Please let me know once the process is complete, so that I can
apply your patch in master.


I am still waiting for FSF's signature. Should come in soon I guess.

Thanks,

Fred

Re: [O] [PATCH] New header parameter :show-process for Org-babel-clojure

[Frederick Giasson]

Hi Nicolas,


Thank you. Some nit-picks follow.


I made all the changes requested and merged in a single commit. See the 
attachment.



BTW, did you sign FSF papers already? I cannot find any reference in
org-contribute.org?


I did as of today. Waiting to get a reply from them on that.

Thanks,

Fred
>From 3eb282c554f8099627b748e7c4eb07d0e810a2ae Mon Sep 17 00:00:00 2001
From: Frederick Giasson <f...@fgiasson.com>
Date: Mon, 14 Nov 2016 15:43:16 -0500
Subject: [PATCH] Multiple improvements of ob-clojure have been performed in
 this commit.

  (1) better error handling. Errors and exceptions are output in the results 
section of a org-mode document. They were silenced with the previous version.
  (2) A new header parameter has been created: ":show-process" what will output 
the underlying NREPL processing in a new window + buffer

Note that the behavior of ob-clojure *did not change* with these modications.

Only the function (org-babel-execute:clojure) has been modified.
---
 etc/ORG-NEWS   |  32 
 lisp/ob-clojure.el | 105 +
 2 files changed, 113 insertions(+), 24 deletions(-)

diff --git a/etc/ORG-NEWS b/etc/ORG-NEWS
index a360e35..95c97a7 100644
--- a/etc/ORG-NEWS
+++ b/etc/ORG-NEWS
@@ -27,6 +27,38 @@ into
 ** New features
 
 *** Horizontal rules are no longer ignored in LaTeX table math mode
+*** Org-babel-clojure :show-process
+
+A new block code header has been created for Org-babel-clojure that enables
+developers to output the process of an ongoing process into a new 
window/buffer.
+
+You can tell Org-babel-clojure to output the process of a running code block.
+
+To show that output you only have to specify the =:show-process= option
+in the code block's header like this:
+
+#+begin_example
+#+BEGIN_SRC clojure :results output :show-process
+  (dotimes [n 10]
+(println n ".")
+(Thread/sleep 500))
+#+END_SRC
+#+end_example
+
+If =:show-process= is specified that way, then when you will run the code using
+=C-c C-c= a new window will open in Emacs. Everything that is output
+by the REPL will immediately be added to that new window.
+
+When the processing of the code is finished, then the window and its
+buffer will be closed and the results will be reported in the
+=#+RESULTS= section. 
+
+Note that the =:results= parameter's behavior is *not* changed. If
+=silent= is specified, then no result will be displayed. If =output= is
+specified then all the output from the window will appears in the results
+section. If =value= is specified, then only the last returned value of
+the code will be displayed in the results section.
+
 
 * Version 9.0
 
diff --git a/lisp/ob-clojure.el b/lisp/ob-clojure.el
index 72ea77d..f78dd69 100644
--- a/lisp/ob-clojure.el
+++ b/lisp/ob-clojure.el
@@ -2,7 +2,7 @@
 
 ;; Copyright (C) 2009-2016 Free Software Foundation, Inc.
 
-;; Author: Joel Boehland, Eric Schulte, Oleh Krehel
+;; Author: Joel Boehland, Eric Schulte, Oleh Krehel, Frederick Giasson
 ;;
 ;; Keywords: literate programming, reproducible research
 ;; Homepage: http://orgmode.org
@@ -84,34 +84,91 @@
   body)))
 
 (defun org-babel-execute:clojure (body params)
-  "Execute a block of Clojure code with Babel."
+  "Execute a block of Clojure code with Babel. 
+   The underlying process performed by the code block can be output 
+   using the :show-process parameter."
   (let ((expanded (org-babel-expand-body:clojure body params))
-   result)
+(sbuffer "*Clojure Show Process Sub Buffer*")
+   (show (assq :show-process params))
+   (response (list 'dict))
+status
+result)
 (cl-case org-babel-clojure-backend
   (cider
(require 'cider)
(let ((result-params (cdr (assq :result-params params
-(setq result
-  (nrepl-dict-get
-   (nrepl-sync-request:eval
-expanded (cider-current-connection) (cider-current-session))
-   (if (or (member "output" result-params)
-   (member "pp" result-params))
-   "out"
- "value")
-  (slime
-   (require 'slime)
-   (with-temp-buffer
-(insert expanded)
-(setq result
-  (slime-eval
-   `(swank:eval-and-grab-output
- ,(buffer-substring-no-properties (point-min) (point-max)))
-   (cdr (assq :package params)))
-(org-babel-result-cond (cdr (assq :result-params params))
-  result
-  (condition-case nil (org-babel-script-escape result)
-   (error result)
+ ;; Check if the user want show the process in an output buffer/window.
+ (when show
+   ;; Create a new window with the show output buffer.
+   (switch-to-buffer-other-window sbuffer)
+
+   ;; Run the Clojure code in nREPL.
+   (nrepl-reque

Re: [O] HTML export with ":export" parameter with Orgmode 9.0

[Frederick Giasson]

Hi Charles,

Setting `org-export-babel-evaluate' to nil will ensure that NONE of 
your header arguments are followed.


Like the docstring says

 "Users who wish to avoid evaluating code on export should use the 
header argument ‘:eval never-export’."


It is tempting to `make-obsolete-variable' this variable and change 
its name to something that more completely describes what does not 
happen when set to nil.


Great, thanks for the clarification!

Take care,

Fred

Re: [O] HTML export with ":export" parameter with Orgmode 9.0

[Frederick Giasson]

Hi Chuck,


even with a minimal init file.

If you ever set the value of `org-export-babel-evaluate' to nil this 
might produce the behavior you describe. 


Maybe, but as stated in the manual: "Setting the 
org-export-babel-evaluate variable to nil will ensure that no code 
blocks are evaluated as part of the export process."


From here: http://orgmode.org/manual/Exporting-code-blocks.html

The thing is that I indeed want to make sure that the code blocks are 
evaluated, but to me evaluation of code block is different than what to 
export from the Org file. It is often the case that I don't want export 
to evaluate anything, but just to export what is in the Org file's 
markup (while following what the header parameters say).


Thanks,

Fred

Re: [O] HTML export with ":export" parameter with Orgmode 9.0

[Frederick Giasson]

Hi Nicolas,


It seems other users experienced it, yet it is unexpected and I cannot
reproduce it even with a minimal init file.


Ok I think I found the issue.

In all the files I worked on since I upgraded to 9.0 I had the following 
setting at the top of my org files:


=
# -*- org-export-babel-evaluate: nil -*-
=

In these notebooks, I don't want org-export to evaluate the code blocks, 
I want to do it manually (since some of them takes quite a while to run 
and I don't want to wait minutes to get an export).


In 8.x this was working perfectly. However, it appears that this 
behavior changed in 9.0. If I have that in place, then the :export 
header parameter is basically ignored and defaulted to "both".


Did this change and is this to be expected? If so, what should I use to 
get the same behavior?


Thanks!

Fred




Regards,

Re: [O] [BUG] org-create-formula-image requires 'gs' when it doesn't look like to be necessary (causing issues on Windows)

[Frederick Giasson]

Hi Nicolas,


Do you mean that "gs" could be removed from all entries in
`org-preview-latex-process-alist'?

I'm Cc'ing Feng Shu, who implemented the new LaTeX preview mechanism. He
hopefully knows why this dependency was introduced in the first place.


I think so yes, at least on Windows (didn't try on Linux nor OSX)


Thanks,

Fred

Re: [O] HTML export with ":export" parameter with Orgmode 9.0

[Frederick Giasson]

Hi Nicolas,


It seems other users experienced it, yet it is unexpected and I cannot
reproduce it even with a minimal init file.


Ok good. I will try to do some debugging on my side. I am wondering if 
it may not be a Windows issue, or some older 8.x configurations that may 
be at cause here.


Will keep you updated.

Thanks,

Fred

Re: [O] [BUG] org-create-formula-image requires 'gs' when it doesn't look like to be necessary (causing issues on Windows)

[Frederick Giasson]

Hi again,

There is one thing I forgot to mention: PDF/LateX export was working fine.

Thanks,

Fred


Hi,


I am a Windows user and I am testing the Org 9.x branch. I just 
discovered that I couldn't display inline latex anymore using C-c C-x 
C-l. I was getting that error:



org-create-formula-image: Can’t find ‘gs’ (you need to install the 
programs: latex, dvipng and ghostscript.)



A look at orl.el line #4079: "(defcustom 
org-preview-latex-process-alist", for "dvipng" we have a programs 
list: "("latex" "dvipng" "gs")"


The problem here is that "gs" is not an executable that exists in 
Ghostscript for Windows nor I think it is necessary for that to work.


I did remove it from the list of required parameters, then I restarted 
Emacs and everything is working fine now.



Any thoughts on that? Should it be fixed? (I guess)


Thanks,

Fred

[O] [BUG] org-create-formula-image requires 'gs' when it doesn't look like to be necessary (causing issues on Windows)

[Frederick Giasson]

Hi,


I am a Windows user and I am testing the Org 9.x branch. I just 
discovered that I couldn't display inline latex anymore using C-c C-x 
C-l. I was getting that error:



org-create-formula-image: Can’t find ‘gs’ (you need to install the 
programs: latex, dvipng and ghostscript.)



A look at orl.el line #4079: "(defcustom 
org-preview-latex-process-alist", for "dvipng" we have a programs list: 
"("latex" "dvipng" "gs")"


The problem here is that "gs" is not an executable that exists in 
Ghostscript for Windows nor I think it is necessary for that to work.


I did remove it from the list of required parameters, then I restarted 
Emacs and everything is working fine now.



Any thoughts on that? Should it be fixed? (I guess)


Thanks,

Fred

Re: [O] [PATCH] New header parameter :show-process for Org-babel-clojure

[Frederick Giasson]

Hi Nicolas,


I noticed that you included some of my previous fixes in
Org-babel-clojure, so here are the latest work I did. I updated it to
work with the latest code on Git. I did two things with these fixes:


   (1) I better handle any possible error or exceptions that may be
raised by nREPL by displaying any errors in the RESULTS section

   (2) I added a new header option called ":show-process" which create
a new window/buffer to display anything that is outputted by the
Clojure code in the code block.


In the past, I did refer to this as ":async", but as we discussed at
that time, it really was not async. So I changed the naming of this
new feature to remove this ambiguity.

OK.


In any case, I did update the worg documentation accordingly too.

Finally I updated ORG-NEWS in the 9.1 section.

I think you can merge patch 2 and 3. ORG-NEWS updates usually do not
require their own


See the patch files attached to this email.

Thank you. Some nit-picks follow.


Subject: [PATCH 1/2] Better error handling with nREPL. Displaying any possible
  errors or exceptions in the result block. Adding a new :show-process header
  option such that the underlying process of a Clojure block code get output in
  a new buffer and a new window such that the developer can see what is going
  on.

Could you provide a proper commit message? In particular, each modified
function has to be specified there.

  (defun org-babel-execute:clojure (body params)
-  "Execute a block of Clojure code with Babel."
+  "Execute a block of Clojure code with Babel. The underlying process 
performed by the

"The underlying..." has to go on the line below.


+  code block can be output using the :show-process parameter"
(let ((expanded (org-babel-expand-body:clojure body params))
-   result)
+(sbuffer "*Clojure Show Process Sub Buffer*")
+(show (if (assoc :show-process params) t nil))

   (show (assq :show-process params))


+(response (cons 'dict nil))

   (response (list 'dict))


+status
+result)
+ ; Check if the user want show the process in an output buffer/window

Need ";;" instead of ";". Also, full stop missing at the end.


+ (when show
+   ; Create a new window with the show output buffer

   ;; Create output buffer.


+   (switch-to-buffer-other-window sbuffer)
+
+   ; Run the Clojure code in nREPL

Ditto.


+   (nrepl-request:eval
+expanded
+(lambda (resp)
+  (when (member "out" resp)
+; Print the output of the nREPL in the output buffer

Ditto.


+(princ (nrepl-dict-get resp "out") (get-buffer sbuffer)))
+  (when (member "ex" resp)
+; In case there is an exception, then add it to the output 
buffer as well

Ditto.


+(princ (nrepl-dict-get resp "ex") (get-buffer sbuffer))
+(princ (nrepl-dict-get resp "root-ex") (get-buffer sbuffer)))
+  (when (member "err" resp)
+; In case there is an error, then add it to the output buffer 
as well

Ditto.


+(princ (nrepl-dict-get resp "err") (get-buffer sbuffer)))
+  (nrepl--merge response resp)
+  ; Update the status of the nREPL output session

Ditto.

+  (setq status (nrepl-dict-get response "status")))
+(cider-current-connection)
+(cider-current-session))
+
+   ; Wait until the nREPL code finished to be processed

Ditto.


+   (while (not (member "done" status))
+ (nrepl-dict-put response "status" (remove "need-input" status))
+ (accept-process-output nil 0.01)
+ (redisplay))
+
+   ; Delete the show buffer & window when the processing is finalized

Ditto.


+   (let ((wins (get-buffer-window-list sbuffer nil t)))
+ (dolist (win wins)
+   (delete-window win))
+ (kill-buffer sbuffer))

   (mapc #'delete-window (get-buffer-window-list sbuffer nil t))
   (kill-buffer sbuffer)


+
+   ; Put the output or the value in the result section of the code 
block

See above.


+   (setq result (concat (nrepl-dict-get response
+(if (or (member "output" 
result-params)
+(member "pp" 
result-params))
+"out"
+  "value"))
+(nrepl-dict-get response "ex")
+(nrepl-dict-get response "root-ex")
+(nrepl-dict-get response "err"
+ ; Check if user want to run code without showing the process

Ditto.


+ (when (not show)

   (unless show ...)


Ok good, will fix these issues and resend the patch.


BTW, did you sign FSF papers already? I cannot find any 

[O] HTML export with ":export" parameter with Orgmode 9.0

[Frederick Giasson]

Hi,


I am not clear about the change in the Export blocks that occurred in 
Org-mode 9.0



Does this change impact the ":export [...]" code block header parameter? 
What I am often using is this:



http://orgmode.org/manual/Exporting-code-blocks.html


Which currently doesn't work but it is unclear from the release notes if 
it should or not and the code provided to fix the Export blocks doesn't 
appear to look for this syntax, so I guess it should still be working.



Anybody else experience this issue?


Thanks,


Fred

Re: [O] Bug: HTML exports issues: none not working, and issues with #+TITLE

[Frederick Giasson]

Hi Nicolas,

Thanks anyway, this answer led me to do some more research. What did 
work for me is this:


http://emacs.stackexchange.com/a/16616/13925

Only running C-u M-x org-reload was fixing the issue, but when I 
restarted Emacs, it was restarting. Doing org-reload, uninstall and 
re-install fixed the issue.



Thanks,

Fred

Hi Nicolas,

This seem to indicate you have a mixed installation. See, e.g.,
.


I am getting a 404 error for that page, would you have another link 
for that thread?


Thanks,

Fred

Re: [O] Bug: HTML exports issues: none not working, and issues with #+TITLE

[Frederick Giasson]

Hi Nicolas,

This seem to indicate you have a mixed installation. See, e.g.,
.


I am getting a 404 error for that page, would you have another link for 
that thread?


Thanks,

Fred

[O] Bug: HTML exports issues: none not working, and issues with #+TITLE

[Frederick Giasson]

Hi,

I upgraded to Emacs 25.1 and Org-mode 9.0 (latest from Elpa) and I am 
experiencing two issues with export:



  (1) The header parameter ":exports none" doesn't appear to be 
working. With the following code, the code block is exported to HTML:


=

#+BEGIN_SRC elisp :results silent :exports none
(setq org-html-htmlize-output-type 'inline-css)
#+END_SRC
=


  (2) As soon as I have a non-empty #+TITLE: defined in a org-mode 
file, I am getting this kind of error with exporting in HTML (or any 
other formats):


=

#+TITLE test

=

returns:

=

apply: Wrong type argument: listp, #("test" 0 4 (:parent (#0)))

=



Thanks,


Fred

[O] [PATCH] New header parameter :show-process for Org-babel-clojure

[Frederick Giasson]

Hi Nicolas,


The new Org-mode release 9.0 made me realize that I never finished some 
of my work with Org-babel-clojure.


I noticed that you included some of my previous fixes in 
Org-babel-clojure, so here are the latest work I did. I updated it to 
work with the latest code on Git. I did two things with these fixes:



  (1) I better handle any possible error or exceptions that may be 
raised by nREPL by displaying any errors in the RESULTS section


  (2) I added a new header option called ":show-process" which create a 
new window/buffer to display anything that is outputted by the Clojure 
code in the code block.



In the past, I did refer to this as ":async", but as we discussed at 
that time, it really was not async. So I changed the naming of this new 
feature to remove this ambiguity.


One thing I noticed in this process is that you actually did update worg 
with the documentation of ":async" but didn't add the actual 
Org-babel-clojure code modifications.


In any case, I did update the worg documentation accordingly too.

Finally I updated ORG-NEWS in the 9.1 section.

See the patch files attached to this email.


Thanks!


Take care,


Fred

From df8284b45c35e0c1dcbccf40ba7b76fd18270b90 Mon Sep 17 00:00:00 2001
From: Frederick Giasson <f...@fgiasson.com>
Date: Mon, 7 Nov 2016 13:48:50 -0500
Subject: [PATCH 1/2] Better error handling with nREPL. Displaying any possible
 errors or exceptions in the result block. Adding a new :show-process header
 option such that the underlying process of a Clojure block code get output in
 a new buffer and a new window such that the developer can see what is going
 on.

---
 lisp/ob-clojure.el | 106 +
 1 file changed, 82 insertions(+), 24 deletions(-)

diff --git a/lisp/ob-clojure.el b/lisp/ob-clojure.el
index 72ea77d..4800c9a 100644
--- a/lisp/ob-clojure.el
+++ b/lisp/ob-clojure.el
@@ -2,7 +2,7 @@
 
 ;; Copyright (C) 2009-2016 Free Software Foundation, Inc.
 
-;; Author: Joel Boehland, Eric Schulte, Oleh Krehel
+;; Author: Joel Boehland, Eric Schulte, Oleh Krehel, Frederick Giasson
 ;;
 ;; Keywords: literate programming, reproducible research
 ;; Homepage: http://orgmode.org
@@ -84,34 +84,92 @@
   body)))
 
 (defun org-babel-execute:clojure (body params)
-  "Execute a block of Clojure code with Babel."
+  "Execute a block of Clojure code with Babel. The underlying process 
performed by the
+  code block can be output using the :show-process parameter"
   (let ((expanded (org-babel-expand-body:clojure body params))
-   result)
+(sbuffer "*Clojure Show Process Sub Buffer*")
+(show (if (assoc :show-process params) t nil))
+(response (cons 'dict nil))
+status
+result)
 (cl-case org-babel-clojure-backend
   (cider
(require 'cider)
(let ((result-params (cdr (assq :result-params params
-(setq result
-  (nrepl-dict-get
-   (nrepl-sync-request:eval
-expanded (cider-current-connection) (cider-current-session))
-   (if (or (member "output" result-params)
-   (member "pp" result-params))
-   "out"
- "value")
-  (slime
-   (require 'slime)
-   (with-temp-buffer
-(insert expanded)
-(setq result
-  (slime-eval
-   `(swank:eval-and-grab-output
- ,(buffer-substring-no-properties (point-min) (point-max)))
-   (cdr (assq :package params)))
-(org-babel-result-cond (cdr (assq :result-params params))
-  result
-  (condition-case nil (org-babel-script-escape result)
-   (error result)
+ ; Check if the user want show the process in an output buffer/window
+ (when show
+   ; Create a new window with the show output buffer
+   (switch-to-buffer-other-window sbuffer)
+
+   ; Run the Clojure code in nREPL
+   (nrepl-request:eval
+expanded 
+(lambda (resp) 
+  (when (member "out" resp)
+; Print the output of the nREPL in the output buffer
+(princ (nrepl-dict-get resp "out") (get-buffer sbuffer)))
+  (when (member "ex" resp)
+; In case there is an exception, then add it to the output 
buffer as well
+(princ (nrepl-dict-get resp "ex") (get-buffer sbuffer))
+(princ (nrepl-dict-get resp "root-ex") (get-buffer sbuffer)))
+  (when (member "err" resp)
+; In case there is an error, then add it to the output buffer 
as well
+(princ (nrepl-dict-get resp "err") (get-buffer sbuffer)))
+  (nrepl--merge response resp)
+  ; Update the 

Re: [O] ob-clojure?

[Frederick Giasson]

Hi Lawrence,

Is there an ob-clojure in elpa? If not should I use this 
?


You may be interested in this version:

http://fgiasson.com/blog/index.php/2016/06/16/improving-org-babel-clojure/

I didn't find the time to push into the Org-mod update pipeline, but I 
will do so soon (hopefully). In mean time all the code is there.



Thanks,

Fred

[O] Table (used a "spreadsheet") org-sbe issues when the value is a string

[Frederick Giasson]

Hi Everybody,

I am experiencing a few issues trying to update a cell value using 
=org-sbe= when the value of a column is a string.


Here is an example that works when the value is a number:




#+NAME: average-dataset-size
#+BEGIN_SRC clojure :var f="turtle" :exports none :results value
(identity f)
#+END_SRC

#+RESULTS: average-dataset-size
: turtle


| Format | Sizes distribution in MB | Average size in MB |
|+--+|
|  7 |  |  7 |
#+TBLFM: $3='(org-sbe "average-dataset-size" (f $1))



Now, if I put a string in the format column, I al getting the following:



#+NAME: average-dataset-size
#+BEGIN_SRC clojure :var f="turtle" :exports none :results value
(identity f)
#+END_SRC

#+RESULTS: average-dataset-size
: turtle


| Format | Sizes distribution in MB | Average size in MB |
|+--+|
| turtle |  | #ERROR |
#+TBLFM: $3='(org-sbe "average-dataset-size" (f $1))



The debugging session I have is (which doesn't tell me with Result: is 
#ERROR):




Substitution history of formula
Orig:   ?
$xyz->  '(org-sbe "average-dataset-size" (f $1))
@r$c->  '(org-sbe "average-dataset-size" (f $1))
$1->'(org-sbe "average-dataset-size" (f "turtle"))
Result: #ERROR
Format: NONE
Final:  #ERROR




Note #1: (identity) is a function in Clojure that returns the same value 
that is in input.



I am wondering if this is a bug, or something that I am missing in how 
to use these features?



Thanks for helping,


Fred

Re: [O] [PATCH] new :async feature for org-babel-clojure

[Frederick Giasson]

Hi Nicolas,




Right now, async is really more about "feedback" than "asynchronous".
However it has never been clear is it was possible or not, and if so,
how :)

Any pointers on how this could be done in emacs?

You could start a new Emacs evaluating some code in a subprocess, with
`start-process' and collect return value. You may want to have a look at
`org-export-async-start' or "async.el" library in ELPA.

Unfortunately, it is not yet possible to re-use the latter in Org code,
since it is unavailable in vanilla Emacs.

The hard part, IMO, is to bring the right context in the subprocess
(e.g., the environment should be the same, other blocks may be needed,
etc.).


Yeah, will have to take some more time to check that, I probably have 
much to learn regarding Elisp itself first.


Rethinking about that, would it not be better to remove the confusion by 
changing the terminology I used: ":async"? What it really is is to have 
feedback of the underlying process, and not really an asynchronous process.


I thought about two possibilities:

  (1) creating two new results options: :output-process and 
:value-process. Which means that we output the underlying process of a 
:output or :value :results

  (2) renaming :async by :show-process or something similar.

Then we take the time to check how a real asynchronous system can be put 
in place.


Thoughts?

Thanks,

Fred





Regards,

Re: [O] [PATCH] new :async feature for org-babel-clojure

[Frederick Giasson]

Hi Nicolas,


As for 3 and 4, I think a more general mechanism for asynchrnous
eval'ing would be preferable. Besides, AFAIU, because of

; Wait until the nREPL code finished to be processed
(while (not (member "done" status))
  (nrepl-dict-put response "status" (remove "need-input" status))
  (accept-process-output nil 0.01)
  (redisplay))

`org-babel-execute:clojure' is still somewhat synchronous, isn't it?



Yes, and this is what I was mentioning initially (here, I think, and on 
my blogpost about it). I just don't know how all this processing could 
be handled in a different thread. That way it would free the current 
threads and we could continue to work with the current instance while 
the processing is happening because as I mentioned, everything is still 
stuck during the processing.


Right now, async is really more about "feedback" than "asynchronous". 
However it has never been clear is it was possible or not, and if so, how :)


Any pointers on how this could be done in emacs?

Thanks,

Fred




Regards,

Re: [O] [PATCH] new :async feature for org-babel-clojure

[Frederick Giasson]

Hi Nicolas,

Any news regarding these latest fixes to that patch?

Thanks,

Fred



Here is my proposal to create a new :async feature for 
Org-babel-clojure. This is discussed at length in this blog post:


http://fgiasson.com/blog/index.php/2016/04/05/using-clojure-in-org-mode-and-implementing-asynchronous-processing/ 



I added the commit of the changes, the commit for the ORG-NEWS file 
and the commit for the Worg documentation.


IMPORTANT NOTE: this includes the patches for the new :async feature 
*AND* the new org-babel-clojure-sync-nrepl-timeout setting (for 
consistency's sake)


Thanks,

Fred

[O] Org-list-todo not accessible via "C-c a t"?

[Frederick Giasson]

Hi,


I am using the latest code, and I am wondering where did the 
org-list-todo keybinding goes? Given the documentation, I should be able 
to get is via "C-c a t", but the "t" is not available in the options. 
But the latest Worg still have "C-c a t" as the binding.



Am I missing something?


Thanks,


Fred

[O] Does org-babel-tangle-jump-to-code exists?

[Frederick Giasson]

Hi Everybody,

I was wondering if something like "org-babel-tangle-jump-to-code" exists 
anywhere? I am looking for a way to jump to a tangled file from the Org 
file, and not the opposite (which is handled using 
"org-babel-tangle-jump-to-org").



Thanks,


Fred

Re: [O] [PATCH] expose nrepl's timeout setting in ob-clojure.el

[Frederick Giasson]

Hi Nicolas,


Some keywords are missing:

   :version "25.1"
   :package-version '(Org . "9.0")

and perhaps

   :safe #'wholenump


Ok good, added.



  (defcustom org-babel-clojure-backend
(cond ((featurep 'cider) 'cider)
(t 'slime))
@@ -94,8 +100,9 @@
 (let ((result-params (cdr (assoc :result-params params
 (setq result
   (nrepl-dict-get
-   (nrepl-sync-request:eval
-expanded (cider-current-connection) (cider-current-session))
+   (let ((nrepl-sync-request-timeout 
org-babel-clojure-sync-nrepl-timeout))
+ (nrepl-sync-request:eval
+  expanded (cider-current-connection) (cider-current-session)))

You forgot to

(defvar nrepl-sync-request-timeout)


This one is defined in the nREPL package. Maybe there is something that 
I don't understand, but do I have to re-defined it here?



I also think it makes sense to merge the 3 patches.


Ok will do.

Thanks,

Fred

[O] [PATCH] new :async feature for org-babel-clojure

[Frederick Giasson]

Hi,

Here is my proposal to create a new :async feature for 
Org-babel-clojure. This is discussed at length in this blog post:


http://fgiasson.com/blog/index.php/2016/04/05/using-clojure-in-org-mode-and-implementing-asynchronous-processing/

I added the commit of the changes, the commit for the ORG-NEWS file and 
the commit for the Worg documentation.


IMPORTANT NOTE: this includes the patches for the new :async feature 
*AND* the new org-babel-clojure-sync-nrepl-timeout setting (for 
consistency's sake)


Thanks,

Fred
>From 2fe515d1fb89ba1ea160711d10f2f68fe0287246 Mon Sep 17 00:00:00 2001
From: Frederick Giasson <f...@fgiasson.com>
Date: Wed, 6 Apr 2016 12:18:21 -0400
Subject: [PATCH 3/4] Adding information about the new org-babel-clojure :async
 feature.

---
 etc/ORG-NEWS | 9 +
 1 file changed, 9 insertions(+)

diff --git a/etc/ORG-NEWS b/etc/ORG-NEWS
index e684587..fad0679 100644
--- a/etc/ORG-NEWS
+++ b/etc/ORG-NEWS
@@ -110,6 +110,15 @@ becomes
 This new option tells the =org-babel-clojure= module the length of the timeout 
 of a synchronous call to the nREPL. This value is in seconds. If =nil= then
 no timeout will occur.
+*** New code block option =:async= for =org-babel-clojure=
+This new option gives the possibility to =org-babel-clojure= user to run
+a block code asynchronously using a new code block parameter =:async=.
+If =:async= is specified in the code block, then it will be processed
+asynchronously by Org-mode. This means that everything that will be 
+output to the REPL by the code will appear in a new window when it will
+be output. Once the processing is finalized, then the window will be closed
+and the output will be added to the results section (if =:results output= is 
+specified).
 *** New org-protocol key=value syntax
 
 Org-protocol can now handle query-style parameters such as:
-- 
1.9.5.msysgit.0

>From e218aaef38f6dc0f9affa516aafa18b77c2a3a1e Mon Sep 17 00:00:00 2001
From: Frederick Giasson <f...@fgiasson.com>
Date: Wed, 6 Apr 2016 12:19:11 -0400
Subject: [PATCH 4/4] Adding a new feature :async feature to org-babel-clojure.
 With this new feature, someone can evaluate Clojure code in Org-mode code
 blocks asynchronously.

---
 lisp/ob-clojure.el | 78 ++
 1 file changed, 61 insertions(+), 17 deletions(-)

diff --git a/lisp/ob-clojure.el b/lisp/ob-clojure.el
index 86f1cff..6059d7d 100644
--- a/lisp/ob-clojure.el
+++ b/lisp/ob-clojure.el
@@ -91,35 +91,79 @@
   body)))
 
 (defun org-babel-execute:clojure (body params)
-  "Execute a block of Clojure code with Babel."
+  "Execute a block of Clojure code with Babel. The block can be executed
+   synchenously by default or asynchronously with the :async parameter"
   (let ((expanded (org-babel-expand-body:clojure body params))
+   (sbuffer "*Clojure Sub Buffer*")
+   (async (if (assoc :async params) t nil))
+   (response (cons 'dict nil))
+   status
result)
 (case org-babel-clojure-backend
   (cider
(require 'cider)
(let ((result-params (cdr (assoc :result-params params
-(setq result
-  (nrepl-dict-get
-   (let ((nrepl-sync-request-timeout 
org-babel-clojure-sync-nrepl-timeout))
- (nrepl-sync-request:eval
-  expanded (cider-current-connection) (cider-current-session)))
-   (if (or (member "output" result-params)
-   (member "pp" result-params))
-   "out"
- "value")
+ ; Check if the user want to run code asynchronously
+ (when async
+   ; Create a new window with the async output buffer
+   (switch-to-buffer-other-window sbuffer)
+
+   ; Run the Clojure code asynchronously in nREPL
+   (nrepl-request:eval
+expanded 
+(lambda (resp) 
+  (when (member "out" resp)
+; Print the output of the nREPL in the asyn output buffer
+(princ (nrepl-dict-get resp "out") (get-buffer sbuffer)))
+  (nrepl--merge response resp)
+  ; Update the status of the nREPL output session
+  (setq status (nrepl-dict-get response "status")))
+(cider-current-connection) 
+(cider-current-session))
+   
+   ; Wait until the nREPL code finished to be processed
+   (while (not (member "done" status))
+ (nrepl-dict-put response "status" (remove "need-input" status))
+ (accept-process-output nil 0.01)
+ (redisplay))
+
+   ; Delete the async buffer & window when the processing is finalized
+   (let ((wins (get-buffer-window-list sbuffer nil t)))
+ (dolist (win wins)
+   (delete-window win))
+ 

Re: [O] [PATCH] expose nrepl's timeout setting in ob-clojure.el

[Frederick Giasson]

Hi Nicolas!


-;;; ob-clojure.el --- Babel Functions for Clojure-*- lexical-binding: t; 
-*-
+;;; ob-clojure.el --- org-babel functions for clojure evaluation

Your patch reverts a change introduced in development version. Could you
rebase it on top of latest Org first?


Yeah sorry, I did this from the latest published version, not master. 
Fixed now.





  ;; Copyright (C) 2009-2016 Free Software Foundation, Inc.
  
@@ -55,6 +55,7 @@
  
  (defvar org-babel-default-header-args:clojure '())

  (defvar org-babel-header-args:clojure '((package . :any)))
+(defvar org-babel-clojure-nrepl-timeout 10)

Would it make sense to turn it into a defcustom instead? If so, this
should be added in etc/ORG-NEWS file.


Patch attached.


Bonus points if Worg documentation
about this back-end


Patch attached.




  (defcustom org-babel-clojure-backend
(cond ((featurep 'cider) 'cider)
@@ -67,7 +68,7 @@
  
  (defun org-babel-expand-body:clojure (body params)

"Expand BODY according to PARAMS, return the expanded body."
-  (let* ((vars (org-babel--get-vars params))
+  (let* ((vars (mapcar #'cdr (org-babel-get-header params :var)))

You are also reverting a previous change here.


Fixed.




 (result-params (cdr (assoc :result-params params)))
 (print-level nil) (print-length nil)
 (body (org-babel-trim
@@ -94,8 +95,9 @@
 (let ((result-params (cdr (assoc :result-params params
 (setq result
   (nrepl-dict-get
-   (nrepl-sync-request:eval
-expanded (cider-current-connection) (cider-current-session))
+   (let ((nrepl-sync-request-timeout 
org-babel-clojure-nrepl-timeout))
+ (nrepl-sync-request:eval
+  expanded (cider-current-connection) (cider-current-session)))

It seems you need to define `nrepl-sync-request-timeout' as
a dynamically scoped variable:

   (defvar nrepl-sync-request-timeout)

Indeed "ob-clojure.el" uses lexical binding, even though you disabled it
in the first line of your patch ;)


Fixed :)


Eventually, could you add an appropriate commit message for this patch?
Something like

   ob-clojure: Make REPL timeout configurable

   * lisp/ob-clojure.el (org-babel-clojure-nrepl-timeout):  New variable.
   (org-babel-expand-body:clojure): Use new variable.

   ... explanations about the motivation of the change...


Patch attached.


Hope everything is good now!

Thanks,

Fred

>From c8e89d774590e9b39604252b3a344f95b56e3924 Mon Sep 17 00:00:00 2001
From: Frederick Giasson <f...@fgiasson.com>
Date: Wed, 6 Apr 2016 10:00:18 -0400
Subject: [PATCH] Document the new option ob-clojure option
 "org-babel-clojure-sync-nrepl-timeout".

---
 org-contrib/babel/languages/ob-doc-clojure.org | 11 +++
 1 file changed, 11 insertions(+)

diff --git a/org-contrib/babel/languages/ob-doc-clojure.org 
b/org-contrib/babel/languages/ob-doc-clojure.org
index 72942df..17c0f20 100644
--- a/org-contrib/babel/languages/ob-doc-clojure.org
+++ b/org-contrib/babel/languages/ob-doc-clojure.org
@@ -90,6 +90,17 @@ Add these lines to your .emacs file to configure CIDER:
   (require 'cider)
 #+END_SRC
 
+Optionally you can specify the Cider timeout by setting the 
=org-babel-clojure-sync-nrepl-timeout=
+setting option. The value is in seconds and if set to =nil= then no timeout 
will occur.
+
+#+BEGIN_SRC emacs-lisp
+  ; Disable Cider's timeout
+  (setq org-babel-clojure-sync-nrepl-timeout nil)
+
+  ; Set the timeout to 20 seconds
+  (setq org-babel-clojure-sync-nrepl-timeout 20)
+#+END_SRC
+
 ** Create a Clojure Project with Leiningen
 
 Create a Leiningen project directory tree:
-- 
1.9.5.msysgit.0

>From 9a81363719f3078d13de9b971440ee6fd456b4b2 Mon Sep 17 00:00:00 2001
From: Frederick Giasson <f...@fgiasson.com>
Date: Wed, 6 Apr 2016 10:24:54 -0400
Subject: [PATCH 1/2] Update of ORG-NEWS to mention the addition of the new
 setting "org-babel-clojure-sync-nrepl-timeout"

---
 etc/ORG-NEWS | 4 
 1 file changed, 4 insertions(+)

diff --git a/etc/ORG-NEWS b/etc/ORG-NEWS
index 82d5ad0..e684587 100644
--- a/etc/ORG-NEWS
+++ b/etc/ORG-NEWS
@@ -106,6 +106,10 @@ becomes
 : ("pdf" . (lambda (file link) (foo)))
 
 ** New features
+*** New option =org-babel-clojure-sync-nrepl-timeout= for =org-babel-clojure=
+This new option tells the =org-babel-clojure= module the length of the timeout 
+of a synchronous call to the nREPL. This value is in seconds. If =nil= then
+no timeout will occur.
 *** New org-protocol key=value syntax
 
 Org-protocol can now handle query-style parameters such as:
-- 
1.9.5.msysgit.0

>From dab6472228d1a0a41831dcae969335d0a7fb7531 Mon Sep 17 00:00:00 2001
From: Frederick Giasson <f...@fgiasson.com>
Date: Wed, 6 Apr 2016 10:46:35 -0400
Subject: [PATCH 2/2] Addition of a new customization variable called
 "org-babel-clojure-sync-nrepl-timeout" which expose the Cider timeout setting
 as a cust

Re: [O] ob-clojure: possible to display results as soon as something is written to STDOUT?

[Frederick Giasson]

Hi,


Yes, you are. There is a "go to the topic" action on the left panel.
Anyway, there is not much to see, just some pointers to start
a discussion.


Haa great thanks, see it now :)

Let me revise a few things, suggest another patch, and then will start 
to think about these things.


Thanks,

Fred

Re: [O] ob-clojure: possible to display results as soon as something is written to STDOUT?

[Frederick Giasson]

Hi Nicolas,


However, I am not sure neither how generic such a solution could be,
and what would still be required to implement in each OB plugin. My
(very little) experience so far is the Python prototype that John
wrote, and then my analysis of the Clojure OB plugin. As I was
explaining in this blog post, things are quite different between the
two. In Python, it appears that one has to run a sub-process, in
Clojure this is not necessary since we benefit the nREPL that handle
that.

The link I gave is a first approximation of the generic behaviour.
Basically, we could spawn a new Emacs process which could then
evaluation the block synchronously.


Ok good.


One difficulty is to grab the "closure" of the block. Another one is to
collect and insert the results back into the buffer without being
disruptive. The thread above gave hints about that second point.


Humm, from the page you provided, I don't see any threads, but just the 
initial message, am I looking at the right thing?


Thanks,

Fred

Re: [O] ob-clojure: possible to display results as soon as something is written to STDOUT?

[Frederick Giasson]

Hi Nicolas,


Here is a blog post that explains the changes I did. I still have one
improvement to do (but don't how it could be done or even if it can be
done) which is explained at the end of the post:

http://fgiasson.com/blog/index.php/2016/04/05/using-clojure-in-org-mode-and-implementing-asynchronous-processing/

Thanks for your help, it is now working the way I need it to.

This looks interesting. Thank you for sharing your result.

Org Babel would benefit from a generic asynchronous execution mechanism.
See for example .


Indeed, it would be interesting to generalize such a generic mechanism 
directly in Babel, and not in each OB plugins.


However, I am not sure neither how generic such a solution could be, and 
what would still be required to implement in each OB plugin. My (very 
little) experience so far is the Python prototype that John wrote, and 
then my analysis of the Clojure OB plugin. As I was explaining in this 
blog post, things are quite different between the two. In Python, it 
appears that one has to run a sub-process, in Clojure this is not 
necessary since we benefit the nREPL that handle that.


So far, the best idea about how to present async process I think is what 
John was proposing with the new window. Otherwise I still wonder about 
having a real background process for this thing to run such that one can 
continue to write things in the Org buffer while a code block is running 
in background. But as I explained, I will have to seriously look into 
Elisp in order to assess the full scope of such a task.



So, if anyone feels like discussing the details of an implementation of
such a thing, please chime in.


So far this is the only thing I have to say about that. In mean time I 
will fix my other patch, and then I will submit another one for this 
async feature (which will probably require much more review from your part).



Thanks,

Fred

Re: [O] ob-clojure: possible to display results as soon as something is written to STDOUT?

[Frederick Giasson]

Hi John,

Here is a blog post that explains the changes I did. I still have one 
improvement to do (but don't how it could be done or even if it can be 
done) which is explained at the end of the post:


http://fgiasson.com/blog/index.php/2016/04/05/using-clojure-in-org-mode-and-implementing-asynchronous-processing/

Thanks for your help, it is now working the way I need it to.


Take care,

Fred

Re: [O] ob-clojure: possible to display results as soon as something is written to STDOUT?

[Frederick Giasson]

Hi Jon,


You can see a commented out version for shell blocks here:
https://github.com/jkitchin/jmax/blob/master/jmax-org.el#L936

It looks like I just redefined the org-babel-execute:sh function after
it was loaded. That may not be recommended good practice, but it works
;)

I am not sure why it is commented out, maybe because I don't use it alot
and its not well tested.

You could always bind your own key sequence instead of C-c C-c.



Yes this is right, probably better/easier/cleaner


I am almost here with my code. In fact, it is quite different than yours 
since the nrepl does async communication. So far, I am really just using 
the Buffer in a new window idea from your code since I don't need to do 
anything with external files.


Will keep you updated!

Thanks,

Fred

Re: [O] ob-clojure: possible to display results as soon as something is written to STDOUT?

[Frederick Giasson]

Hi John,


you should rename it org-babel-async-execute:clojure, and adapt it to
run clojure.


Yes


I wrote the function in the org-file that is that post, and executed the
code block (C-c C-c) which "registers" the function for that instance of
emacs.


I am not sure I understand here. Once you execute the elisp block with 
that function, the function becomes defined for that instance. But when 
you execute your other block, the example python block using C-c C-c, 
how does org-mode knows to use "org-babel-async-execute:python" instead 
of "org-babel-execute:python"??


You put the cursor in the block, and then call it using M-x?


Later you could put it in an init file that is loaded when Emacs starts.

and use M-x to call it. Once I added an :async option to the header args
to make C-c C-c execute it, but until it works the way you want with M-x
that is just convenience ;)


Yes, this is that :async option that would be great and necessary once 
it works using M-x :)


How can this be done? (Is there an extension mechanism in org-mode for 
that, or it needs to hack the core code?)


Thanks,


Fred

Re: [O] ob-clojure: possible to display results as soon as something is written to STDOUT?

[Frederick Giasson]

Hi John!

Interesting approach :)

I am really new with org-mode, but I will try to adapt for ob-clojure.el.

One question: this function "org-babel-async-execute:python", where 
should I register it? I guess it should replace "org-babel-execute:python"?


Thanks for this precision will work on that once I know how to plug that 
into org-mode to start testing the code I will adapt.


Thanks,

Fred



This might not be totally true.

In this post I use a few different asynchronous approaches to running
Python that do something like what you want, perhaps especially the last
bit of the post.

http://kitchingroup.cheme.cmu.edu/blog/2015/11/20/Asynchronously-running-python-blocks-in-org-mode/

Maybe you could adapt it for clojure.

Frederick Giasson writes:


Hi Eric,


Note: I am not sure if this is something related to Org-mode, or
ob-clojure or Cider.

None of these directly.  It's Emacs, basically: it has very minimal
support for threading so the whole process (Emacs) is waiting for the
sub-process (Clojure) to finish before doing anything else.


Ok good, thanks for confirming!

Take care,

Fred


--
Professor John Kitchin
Doherty Hall A207F
Department of Chemical Engineering
Carnegie Mellon University
Pittsburgh, PA 15213
412-268-7803
@johnkitchin
http://kitchingroup.cheme.cmu.edu

Re: [O] ob-clojure: possible to display results as soon as something is written to STDOUT?

[Frederick Giasson]

Hi Eric,


Note: I am not sure if this is something related to Org-mode, or
ob-clojure or Cider.

None of these directly.  It's Emacs, basically: it has very minimal
support for threading so the whole process (Emacs) is waiting for the
sub-process (Clojure) to finish before doing anything else.



Ok good, thanks for confirming!

Take care,

Fred

[O] ob-clojure: possible to display results as soon as something is written to STDOUT?

[Frederick Giasson]

Hi everybody,

I was wondering if it was possible for Org-mode to write results in 
#+RESULTS as soon as something is written to STDOUT?


Here is my issue: I am writing Notebooks in Clojure using org-mode 
(clojure). I have some long-running procedures in the notebook that 
output the current status of the process (like, 1/4986000) to the 
REPL using printnl.


With now, I am using the header "clojure :session :results output.

I am getting the output, but *only once the function finished its 
processing*. What I would like to know if it is possible to have 
Org-mode to write things when they are written to STDOUT instead then 
just when it finished?


Note: I am not sure if this is something related to Org-mode, or 
ob-clojure or Cider.


Thanks,

Fred

[O] [PATCH] expose nrepl's timeout setting in ob-clojure.el

[Frederick Giasson]

Hi everybody,

I am starting to use org-mode to create literate applications in 
Clojure. So far so good, this is a terrific piece of software.


One thing why I wanted to use org-mode is to use it to create Clojure 
Notebooks. The problem I had is that I have many functions that can take 
some time (more than 10 seconds) to complete. This means that I was 
often receiving that error from nrepl:


===
nrepl-send-sync-request: Sync nREPL request timed out (op eval session 
57bdacff-b178-4952-8bf8-5e01ac9d745a code (def umbel 
(create-ontology-structure))

===

The problem is that there is no way to define the nrepl timeout from 
Org-mode. What I did is to expose that nrepl setting in ob-clojure.el. 
This is the ob-clojure.el.diff patch I am proposing here.


The name of the new setting is "org-babel-clojure-nrepl-timeout". If it 
is set to /nil/ then no timeout will occur, otherwise any integer will 
define a timeout value.


This variable can be set from the /.emacs/ global setting file.

It works fine from here, please modify as required.

Thanks,

Fred
diff --git a/lisp/ob-clojure.el b/lisp/ob-clojure.el
index 89a09a9..46031e1 100644
--- a/lisp/ob-clojure.el
+++ b/lisp/ob-clojure.el
@@ -1,4 +1,4 @@
-;;; ob-clojure.el --- Babel Functions for Clojure-*- lexical-binding: t; 
-*-
+;;; ob-clojure.el --- org-babel functions for clojure evaluation
 
 ;; Copyright (C) 2009-2016 Free Software Foundation, Inc.
 
@@ -55,6 +55,7 @@
 
 (defvar org-babel-default-header-args:clojure '())
 (defvar org-babel-header-args:clojure '((package . :any)))
+(defvar org-babel-clojure-nrepl-timeout 10)
 
 (defcustom org-babel-clojure-backend
   (cond ((featurep 'cider) 'cider)
@@ -67,7 +68,7 @@
 
 (defun org-babel-expand-body:clojure (body params)
   "Expand BODY according to PARAMS, return the expanded body."
-  (let* ((vars (org-babel--get-vars params))
+  (let* ((vars (mapcar #'cdr (org-babel-get-header params :var)))
 (result-params (cdr (assoc :result-params params)))
 (print-level nil) (print-length nil)
 (body (org-babel-trim
@@ -94,8 +95,9 @@
(let ((result-params (cdr (assoc :result-params params
 (setq result
   (nrepl-dict-get
-   (nrepl-sync-request:eval
-expanded (cider-current-connection) (cider-current-session))
+   (let ((nrepl-sync-request-timeout 
org-babel-clojure-nrepl-timeout))
+ (nrepl-sync-request:eval
+  expanded (cider-current-connection) (cider-current-session)))
(if (or (member "output" result-params)
(member "pp" result-params))
"out"

warning: LF will be replaced by CRLF in lisp/ob-clojure.el.
The file will have its original line endings in your working directory.