Re: Babel C-mode corrupts double-quoted strings in output
tbanelwebmin writes: > This fix in ob-C.el passes all unit tests, as well as Martin's example, and > some other examples. > > If someone get a regression, please tell me! > In a few days, I will commit. > > The fix: in ob-C.el line 185, change: > (org-babel-read results t) > to: > results Did you have a chance to push the commit? I am not seeing anything in the git log. -- Ihor Radchenko, Org mode contributor, Learn more about Org mode at https://orgmode.org/. Support Org development at https://liberapay.com/org-mode, or support my work at https://liberapay.com/yantar92
Re: Babel C-mode corrupts double-quoted strings in output
This fix in ob-C.el passes all unit tests, as well as Martin's example, and some other examples. If someone get a regression, please tell me! In a few days, I will commit. The fix: in ob-C.el line 185, change: (org-babel-read results t) to: results Have fun Le 02/09/2022 à 13:02, tbanelwebmin a écrit : This looks like a bug in ob-C.el Around line 196 we should replace (org-babel-read results t) with results In this way, ob-C.el will look more like ob-shell.el Let me see what are the consequences with such a fix. Thanks Martin for investigating deep in the sources! Regards Le 31/08/2022 à 18:35, Martin Jerabek a écrit : Hi! I recently started to use Org Babel for C++ programs. One of the programs outputs several lines with double-quoted strings, similar to this: #+NAME: doublequotes_cpp #+begin_src cpp :includes :results output verbatim raw std::cout << "\"line 1\"\n"; std::cout << "\"line 2\"\n"; std::cout << "\"line 3\"\n"; #+end_src #+RESULTS: doublequotes_cpp line 1 As you can see, only the first line is copied to the RESULTS block, and it is stripped of the double quotes. I tracked down the problem to org-babel-read (in ob-core.el). org-babel-C-execute (in ob-C.el) calls this function with the output of the C++ program. The problem is the following line: ((eq (string-to-char cell) ?\") (read cell)) i.e. if the output of the program starts with a double quote, it is passed to read which reads only the first string and also removes the double quotes, resulting in the observed output. The original version of this piece of code was added with the following commit: commit 60a8ba556d682849eafb0f84e689967cd2965549 Author: Eric Schulte Date: Wed Mar 2 07:55:39 2011 -0700 ob: read string variable values wrapped in double quotes, removing the quotes * lisp/ob.el (org-babel-read): Read string variable values wrapped in double quotes, removing the quotes. AFAICT this modification was done in response to the email thread "[Orgmode] org-babel-read should have option NOT to interpret as elisp" started on 2011-02-27, more specifically the email on "Wed, 02 Mar 2011 07:56:45 -0700" from Eric Schulte. This was obviously done for parsing variables in the header line, not for the program output, but the Babel C mode uses org-babel-read also for the output. I assumed that ":results output verbatim raw" would prevent any postprocessing of the output but this is not the case for C mode. I am not sure how to fix this without breaking backward compatibility. I assume it should be fixed directly in org-babel-C-execute, not in a central function like org-babel-read to minimize the impact. Surprisingly (for me) the equivalent shell script works as expected: #+NAME: doublequotes #+begin_src shell :results output verbatim raw echo '"line 1"' echo '"line 2"' echo '"line 3"' #+end_src #+RESULTS: doublequotes "line 1" "line 2" "line 3" because org-babel-execute:shell does not process the output with org-babel-read. I do not know if languages other than the C family (C, C++, D) are affected. At the very least, the documentation of org-babel-read should be expanded to document the fact that if the CELL parameter starts with a double quote, it is processed by the read function. Best regards Martin Jerabek
Re: Babel C-mode corrupts double-quoted strings in output
This looks like a bug in ob-C.el Around line 196 we should replace (org-babel-read results t) with results In this way, ob-C.el will look more like ob-shell.el Let me see what are the consequences with such a fix. Thanks Martin for investigating deep in the sources! Regards Le 31/08/2022 à 18:35, Martin Jerabek a écrit : Hi! I recently started to use Org Babel for C++ programs. One of the programs outputs several lines with double-quoted strings, similar to this: #+NAME: doublequotes_cpp #+begin_src cpp :includes :results output verbatim raw std::cout << "\"line 1\"\n"; std::cout << "\"line 2\"\n"; std::cout << "\"line 3\"\n"; #+end_src #+RESULTS: doublequotes_cpp line 1 As you can see, only the first line is copied to the RESULTS block, and it is stripped of the double quotes. I tracked down the problem to org-babel-read (in ob-core.el). org-babel-C-execute (in ob-C.el) calls this function with the output of the C++ program. The problem is the following line: ((eq (string-to-char cell) ?\") (read cell)) i.e. if the output of the program starts with a double quote, it is passed to read which reads only the first string and also removes the double quotes, resulting in the observed output. The original version of this piece of code was added with the following commit: commit 60a8ba556d682849eafb0f84e689967cd2965549 Author: Eric Schulte Date: Wed Mar 2 07:55:39 2011 -0700 ob: read string variable values wrapped in double quotes, removing the quotes * lisp/ob.el (org-babel-read): Read string variable values wrapped in double quotes, removing the quotes. AFAICT this modification was done in response to the email thread "[Orgmode] org-babel-read should have option NOT to interpret as elisp" started on 2011-02-27, more specifically the email on "Wed, 02 Mar 2011 07:56:45 -0700" from Eric Schulte. This was obviously done for parsing variables in the header line, not for the program output, but the Babel C mode uses org-babel-read also for the output. I assumed that ":results output verbatim raw" would prevent any postprocessing of the output but this is not the case for C mode. I am not sure how to fix this without breaking backward compatibility. I assume it should be fixed directly in org-babel-C-execute, not in a central function like org-babel-read to minimize the impact. Surprisingly (for me) the equivalent shell script works as expected: #+NAME: doublequotes #+begin_src shell :results output verbatim raw echo '"line 1"' echo '"line 2"' echo '"line 3"' #+end_src #+RESULTS: doublequotes "line 1" "line 2" "line 3" because org-babel-execute:shell does not process the output with org-babel-read. I do not know if languages other than the C family (C, C++, D) are affected. At the very least, the documentation of org-babel-read should be expanded to document the fact that if the CELL parameter starts with a double quote, it is processed by the read function. Best regards Martin Jerabek
Re: Babel C-mode corrupts double-quoted strings in output
I would advise you not to use org babel for compiled languages. There is just too much stuff that doesn't work well and multifile dependencies & build systems are just plain hard to get right. Debugging is annoying and getting org babel to tell you compiler where the source actually came from is impossible, if the language even supports #line and #file directives (Rust still doesn't). Immanuel Litzroth On Wed, Aug 31, 2022 at 6:38 PM Martin Jerabek wrote: > > Hi! > > I recently started to use Org Babel for C++ programs. One of the programs > outputs several lines with double-quoted strings, similar to this: > > #+NAME: doublequotes_cpp > #+begin_src cpp :includes :results output verbatim raw > std::cout << "\"line 1\"\n"; > std::cout << "\"line 2\"\n"; > std::cout << "\"line 3\"\n"; > #+end_src > > #+RESULTS: doublequotes_cpp > line 1 > > As you can see, only the first line is copied to the RESULTS block, and it is > stripped of the double quotes. > > I tracked down the problem to org-babel-read (in ob-core.el). > org-babel-C-execute (in ob-C.el) calls this function with the output of the > C++ program. The problem is the following line: > > ((eq (string-to-char cell) ?\") (read cell)) > > i.e. if the output of the program starts with a double quote, it is passed to > read which reads only the first string and also removes the double quotes, > resulting in the observed output. > > The original version of this piece of code was added with the following > commit: > > commit 60a8ba556d682849eafb0f84e689967cd2965549 > Author: Eric Schulte > Date: Wed Mar 2 07:55:39 2011 -0700 > > ob: read string variable values wrapped in double quotes, removing the > quotes > > * lisp/ob.el (org-babel-read): Read string variable values wrapped in > double quotes, removing the quotes. > > AFAICT this modification was done in response to the email thread "[Orgmode] > org-babel-read should have option NOT to interpret as elisp" started on > 2011-02-27, more specifically the email on "Wed, 02 Mar 2011 07:56:45 -0700" > from Eric Schulte. This was obviously done for parsing variables in the > header line, not for the program output, but the Babel C mode uses > org-babel-read also for the output. > > I assumed that ":results output verbatim raw" would prevent any > postprocessing of the output but this is not the case for C mode. > > I am not sure how to fix this without breaking backward compatibility. I > assume it should be fixed directly in org-babel-C-execute, not in a central > function like org-babel-read to minimize the impact. Surprisingly (for me) > the equivalent shell script works as expected: > > #+NAME: doublequotes > #+begin_src shell :results output verbatim raw > echo '"line 1"' > echo '"line 2"' > echo '"line 3"' > #+end_src > > #+RESULTS: doublequotes > "line 1" > "line 2" > "line 3" > > because org-babel-execute:shell does not process the output with > org-babel-read. I do not know if languages other than the C family (C, C++, > D) are affected. > > At the very least, the documentation of org-babel-read should be expanded to > document the fact that if the CELL parameter starts with a double quote, it > is processed by the read function. > > Best regards > Martin Jerabek > -- -- A man must either resolve to point out nothing new or to become a slave to defend it. -- Sir Isaac Newton
Babel C-mode corrupts double-quoted strings in output
Hi! I recently started to use Org Babel for C++ programs. One of the programs outputs several lines with double-quoted strings, similar to this: #+NAME: doublequotes_cpp #+begin_src cpp :includes :results output verbatim raw std::cout << "\"line 1\"\n"; std::cout << "\"line 2\"\n"; std::cout << "\"line 3\"\n"; #+end_src #+RESULTS: doublequotes_cpp line 1 As you can see, only the first line is copied to the RESULTS block, and it is stripped of the double quotes. I tracked down the problem to org-babel-read (in ob-core.el). org-babel-C-execute (in ob-C.el) calls this function with the output of the C++ program. The problem is the following line: ((eq (string-to-char cell) ?\") (read cell)) i.e. if the output of the program starts with a double quote, it is passed to read which reads only the first string and also removes the double quotes, resulting in the observed output. The original version of this piece of code was added with the following commit: commit 60a8ba556d682849eafb0f84e689967cd2965549 Author: Eric Schulte Date: Wed Mar 2 07:55:39 2011 -0700 ob: read string variable values wrapped in double quotes, removing the quotes * lisp/ob.el (org-babel-read): Read string variable values wrapped in double quotes, removing the quotes. AFAICT this modification was done in response to the email thread "[Orgmode] org-babel-read should have option NOT to interpret as elisp" started on 2011-02-27, more specifically the email on "Wed, 02 Mar 2011 07:56:45 -0700" from Eric Schulte. This was obviously done for parsing variables in the header line, not for the program output, but the Babel C mode uses org-babel-read also for the output. I assumed that ":results output verbatim raw" would prevent any postprocessing of the output but this is not the case for C mode. I am not sure how to fix this without breaking backward compatibility. I assume it should be fixed directly in org-babel-C-execute, not in a central function like org-babel-read to minimize the impact. Surprisingly (for me) the equivalent shell script works as expected: #+NAME: doublequotes #+begin_src shell :results output verbatim raw echo '"line 1"' echo '"line 2"' echo '"line 3"' #+end_src #+RESULTS: doublequotes "line 1" "line 2" "line 3" because org-babel-execute:shell does not process the output with org-babel-read. I do not know if languages other than the C family (C, C++, D) are affected. At the very least, the documentation of org-babel-read should be expanded to document the fact that if the CELL parameter starts with a double quote, it is processed by the read function. Best regards Martin Jerabek
