Attaching the updated org-matlab.patch On Thu, Jan 2, 2025 at 6:55 PM John C <john.ciolfi...@gmail.com> wrote:
> Hi > > Thanks for the feedback. See updated org-matlab.patch. Note > code evaluation will now error out when using an out-of-date matlab-mode > (rather than generating a warning) because when using an older matlab-mode, > code block evaluation doesn't work. We need to wait for the matlab-shell to > be ready. Without that capability found in newer matlab-mode, the errors > are very hard to understand. I also added tests. Using steps outlined in > the ob-matlab-test.el comment to setup for MATLAB testing, I get on Linux: > > matlab-mode, version 6.3 > Detected MATLAB R2024b (24.2) -- Loading history file > matlab-shell: starting server with name server-2055614 > passed 127/1250 ob-matlab/results-file-graphics (9.712776 sec) > passed 128/1250 ob-matlab/results-file-graphics-with-space (0.332004 > sec) > passed 129/1250 ob-matlab/results-output (0.013681 sec) > passed 130/1250 ob-matlab/results-output-latex (0.459980 sec) > passed 131/1250 ob-matlab/results-output-preserve-whitespace > (0.281001 sec) > passed 132/1250 ob-matlab/results-output-reuse-a (0.039130 sec) > passed 133/1250 ob-matlab/results-output-reuse-b (0.043762 sec) > passed 134/1250 ob-matlab/results-output-reuse-clear (0.301597 sec) > passed 135/1250 ob-matlab/results-verbatim (0.172567 sec) > > You mentioned that we should expect org users to understand sessions, > however the doc on sessions is a bit light. Searching > https://orgmode.org/org.html for "session", we see it used two different > ways "Emacs session" and babel code block evaluations sessions described in > "Using sessions". What is there is okay, but doesn't really help one > understand when they should/shouldn't use babel code block evaluation > sessions. What would be nice is if "Using sessions" were titled "Babel code > block evaluation sessions" containing the existing content minus the "Only > languages that provide interactive evaluation ..." paragraph because this > is vague. In addition, each language should have a way of describing what > :session means to them, perhaps by extracting this info from ob-LANGUAGE.el > or maybe ob-LANGUAGE.org, or maybe something else that inserts the language > specific into the org manual. > > With good MATLAB integration in org-mode, we'll see scientists and > engineers who have limited programming background leverage org-mode for > their scientific papers. Not having to worry about more advanced concepts > like sessions is nice, so thanks for letting us default the :session for > MATLAB to make it work as one would expect. > > Here's the commit info: > > ob-matlab.el: improve MATLAB support > > * lisp/ob-matlab.el (header): Update URL for MATLAB > > * lisp/ob-octave.el (org-babel-octave-evaluate): Fixed MATLAB support > - Deprecate variables related to MATLAB Emacs Link and removed the code. > Emacs Link capability was removed from MATLAB release R2009a, 15 years > ago. > - Fixed the following type of org block evaluation: > 1) #+begin_src matlab :results verbatim > 2) #+begin_src matlab :results output > 3) #+begin_src matlab :results output latex > 4) #+begin_src matlab :results file graphics > which aid in writing scientific papers. > - Minor point, the correct spelling of MATLAB when referencing the > product is > all upper case. > > * lisp/ob-comint.el: enhanced org-babel-comint-with-output > - The META argument of org-babel-comint-with-output now supports an > optional > STRIP-REGEXPS which can be used to remove content from the returned > output. > > * lisp/org.el > - Add MATLAB as one of the options for org-babel-load-languages > > * mk/default.mk > - Add support for testing matlab code blocks. > > * testing/examples/ob-matlab-test.org, testing/lisp/test-ob-matlab.el > - Test for matlab code block. > > * testing/org-test.el > - Added org-test-get-code-block which is for use by > testing/lisp/test*.el files > to extract code blocks from testing/examples/*.org files for on-the-fly > testing using org-test-with-temp-text. > > * etc/ORG-NEWS (New functions and changes in function arguments): > Added entry "ob-octave: improved MATLAB support" > > Thanks > John > > > On Sun, Dec 29, 2024 at 2:41 AM Ihor Radchenko <yanta...@posteo.net> > wrote: > >> John C <john.ciolfi...@gmail.com> writes: >> >> > See attached org-matlab.patch which addresses all feedback. Here's the >> > commit info. >> >> Thanks! >> See my comments inline. >> >> > +*** ob-matlab: fixed MATLAB support >> > + >> > +Fixed MATLAB babel code blocks processing. MATLAB code blocks, >> ~#+begin_src matlab~, with ~:results >> > +verbatim~, ~:results output~, ~:results output latex~, or ~:results >> file graphics~ now work. Fixes >> > +include (1) waiting for matlab-shell to start before evaluating MATLAB >> code, (2) correctly showing >> > +the results using writematrix, (3) removing the code block lines from >> the result, (4) correctly >> > +handling graphics results by invoking print correctly. To use MATLAB >> with org, you need >> > +https://github.com/MathWorks/Emacs-MATLAB-Mode. >> >> There is no need to provide so many details. >> Just leave the most important things: >> >> 1. MATLAB is no longer broken >> 2. Emacs-MATLAB-Mode is required now >> >> > +*** ob-matlab: MATLAB behavior change >> > + >> > +MATLAB code blocks now reuse the ~MATLAB*~ buffer created by ~M-x >> matlab-shell~, whereas the >> > +prior version started a new shell for each evaluation. The benefit of >> this is that >> > +evaluations are very fast after the first evaluation and that state is >> maintained between >> > +evaluations, which you can clear using the MATLAB ~clear~ command. >> Another benefit of this >> > +behavior is that it is consistent with how MATLAB works. >> >> No need to explain in so much details, I think. >> Just say that MATLAB uses session by default and them mention that users >> may customize `org-babel-default-header-args:matlab' to disable session. >> >> > +(defun org-babel-comint--strip-regexps (result strip-regexps) >> > + "STRIP-REGEXPS from RESULT list of strings." >> > + (dolist (strip-regexp strip-regexps) >> > + (let ((new-result '())) >> > + (dolist (line result) >> > + (setq line (replace-regexp-in-string strip-regexp "" line)) >> > + (when (not (string= line "")) >> > + (setq new-result (append new-result `(,line))))) >> >> It is more efficient to use `push' + `nreverse' instead of `append'. >> >> > -(defvar org-babel-default-header-args:matlab '()) >> > +;; With `org-babel-default-header-args:matlab' set to >> > +;; '((:session . "*MATLAB*"))) >> > +;; ... >> > +;; If you want a new session each time you evaluate a MATLAB code >> block, >> > +;; (setq 'org-babel-default-header-args:matlab '()) >> > +;; However, this will make evaluations slower and is not consistent >> with how >> > +;; MATLAB works. MATLAB is designed for many evaluations. >> > +(defvar org-babel-default-header-args:matlab '((:session . >> "*MATLAB*"))) >> >> You don't need that long comment in the source code. >> If you think that explaining the details about session is necessary (it >> may or may not be, but we should assume that Org users are familiar with >> the notion of sessions in code blocks), please do it in the >> documentation, not in the code. >> >> More generally, your motivation is not specific to matlab. Yet, we >> default to no session in most babel backends. So, it is not a question >> of session being faster or slower, but a question of consistency. >> >> That said, some babel backends do default to session, so I do not oppose >> this change too much. >> >> > +(make-obsolete-variable 'org-babel-matlab-with-emacs-link >> > + "MATLAB removed EmacsLink in R2009a." "2009") >> > + >> > +(make-obsolete-variable 'org-babel-matlab-emacs-link-wrapper-method >> > + "MATLAB removed EmacsLink in R2009a." "2009") >> >> Please use Org version in WHEN argument of `make-obsolete-variable'. >> The WHEN should be "9.8". >> >> > +(defun org-babel-matlab-shell () >> > + "Start and/or wait for MATLAB shell." >> > + (require 'matlab-shell) ;; make `matlab-shell-busy-checker' available >> > + (cond >> > + ((fboundp 'matlab-shell-busy-checker) >> > + ;; Start the shell if needed. `matlab-shell' will reuse existing >> if already running. >> > + (matlab-shell) >> > + ;; If we just started the matlab-shell, wait for the prompt. If >> we do not >> > + ;; wait, then the startup messages will show up in the evaluation >> results. >> > + (matlab-shell-busy-checker 'wait-for-prompt)) >> > + (t >> > + (message (concat "You version of matlab-mode is old.\n" >> > + "Please update, see >> https://github.com/mathworks/Emacs-MATLAB-Mode\n"; >> > + "Updating will eliminate unexpected output in >> your results\n")) >> > + (sit-for 3) >> >> Instead of `message' + `sit-fit', you can simply use `display-warning'. >> It will give users more control. >> >> > +(defun org-babel-body-for-output (body matlabp) >> > + "If MATLABP, fixup BODY for MATLAB output result-type." >> > + (when matlabp >> > + ;; When we send multi-line input to `matlab-shell', we'll see the >> "body" >> > + ;; code lines echoed in the output which is not what one would >> expect. To >> > + ;; remove these unwanted lines, we append a comment "%-<org-eval>" >> to each >> > + ;; line in the body MATLAB code. After we collect the results from >> > + ;; evaluation, we leverage the "%-<org-eval>" to remove the >> unwanted lines. >> > + ;; Example of desired behavior: >> > ... >> >> I think that an important point here is that MATLAB does not echo the >> whole BODY all at once and instead mixes it with the output. Which is >> why we need to do something non-standard to filter out the body in >> matlab specifically. >> >> > + (setq body (replace-regexp-in-string "\n" " %-<org-eval>-\n" body)) >> > + (when (not (string-match "\n\\'" body)) >> > + (setq body (concat body " %-<org-eval>-")))) >> > + body) >> >> Please put this %-<org-eval> into an internal constant and then reuse >> it when building the regexp to filter. >> >> > + (when matlabp >> > + '(;; MATLAB echo's all input lines, so use the >> %-<org-eval> comments to strip >> > + ;; them from the output >> > + "^[^\n]*%-<org-eval>-\n" >> > + ;; Remove starting blank line caused by stripping >> %-<org-eval> >> > + "\\`[[:space:]\r\n]+" >> > + ;; Strip <ERRORTXT> and </ERRORTXT> matlab-shell >> error indicators >> > + "</?ERRORTXT>\n"))) >> >> Same here. Please put these regexps into a constant. >> >> -- >> Ihor Radchenko // yantar92, >> Org mode maintainer, >> Learn more about Org mode at <https://orgmode.org/>. >> Support Org development at <https://liberapay.com/org-mode>, >> or support my work at <https://liberapay.com/yantar92> >> >
diff --git a/etc/ORG-NEWS b/etc/ORG-NEWS index 5d421172f..4f9523cf3 100644 --- a/etc/ORG-NEWS +++ b/etc/ORG-NEWS @@ -162,6 +162,17 @@ bibliography format requires them to be written in title-case. # This also includes changes in function behavior from Elisp perspective. +*** ob-matlab: fixed MATLAB support + +Fixed MATLAB babel code blocks processing. MATLAB code blocks, ~#+begin_src matlab~, with ~:results +verbatim~, ~:results output~, ~:results output latex~, or ~:results file graphics~ now work. To use +MATLAB with org, you need https://github.com/MathWorks/Emacs-MATLAB-Mode. + +*** ob-matlab: MATLAB behavior change + +MATLAB code blocks now reuse the ~MATLAB*~ buffer created by ~M-x matlab-shell~ by default. To +change this behavior, customize ~org-babel-default-header-args:matlab~. + *** ob-sqlite: Added ability to open a database in readonly mode Added option :readonly to ob-sqlite. diff --git a/lisp/ob-comint.el b/lisp/ob-comint.el index b88ac445a..2246badc2 100644 --- a/lisp/ob-comint.el +++ b/lisp/ob-comint.el @@ -101,15 +101,28 @@ PROMPT-REGEXP defaults to `comint-prompt-regexp'." (setq string (substring string (match-end 0)))) string) +(defun org-babel-comint--strip-regexps (result strip-regexps) + "STRIP-REGEXPS from RESULT list of strings." + (dolist (strip-regexp strip-regexps) + (let ((new-result '())) + (dolist (str result) + (setq str (replace-regexp-in-string strip-regexp "" str)) + (when (not (string= str "")) + (push str new-result))) + (setq result (nreverse new-result)))) + result) + (defmacro org-babel-comint-with-output (meta &rest body) "Evaluate BODY in BUFFER and return process output. Will wait until EOE-INDICATOR appears in the output, then return all process output. If REMOVE-ECHO and FULL-BODY are present and -non-nil, then strip echo'd body from the returned output. META -should be a list containing the following where the last two -elements are optional. +non-nil, then strip echoed body from the returned output. If +optional STRIP-REGEXPS, a list of regular expressions, is +present, then all matches will be removed from the returned +output. META should be a list containing the following where the +last three elements are optional. - (BUFFER EOE-INDICATOR REMOVE-ECHO FULL-BODY) + (BUFFER EOE-INDICATOR REMOVE-ECHO FULL-BODY STRIP-REGEXPS) This macro ensures that the filter is removed in case of an error or user `keyboard-quit' during execution of body." @@ -117,7 +130,8 @@ or user `keyboard-quit' during execution of body." (let ((buffer (nth 0 meta)) (eoe-indicator (nth 1 meta)) (remove-echo (nth 2 meta)) - (full-body (nth 3 meta))) + (full-body (nth 3 meta)) + (strip-regexps (nth 4 meta))) `(org-babel-comint-in-buffer ,buffer (let* ((string-buffer "") (comint-output-filter-functions @@ -165,8 +179,12 @@ or user `keyboard-quit' during execution of body." (and ,remove-echo ,full-body (setq string-buffer (org-babel-comint--echo-filter string-buffer ,full-body))) - ;; Filter out prompts. - (org-babel-comint--prompt-filter string-buffer))))) + ;; Filter out prompts from returned output result. + (let ((result (org-babel-comint--prompt-filter string-buffer))) + ;; Remove all matches of STRIP-REGEXPS in returned output result. + (when ,strip-regexps + (setq result (org-babel-comint--strip-regexps result ,strip-regexps))) + result))))) (defun org-babel-comint-input-command (buffer cmd) "Pass CMD to BUFFER. diff --git a/lisp/ob-matlab.el b/lisp/ob-matlab.el index de8deadbe..083dcdec3 100644 --- a/lisp/ob-matlab.el +++ b/lisp/ob-matlab.el @@ -28,11 +28,10 @@ ;;; Requirements: -;; Matlab - -;; matlab.el required for interactive emacs sessions and matlab-mode -;; major mode for source code editing buffer -;; https://matlab-emacs.sourceforge.net/ +;; 1) MATLAB +;; 2) https://github.com/mathworks/Emacs-MATLAB-Mode +;; For matlab-shell to run MATLAB within Emacs and matlab-mode +;; major mode for source code editing buffer ;;; Code: diff --git a/lisp/ob-octave.el b/lisp/ob-octave.el index 005990f20..5e622df9a 100644 --- a/lisp/ob-octave.el +++ b/lisp/ob-octave.el @@ -1,8 +1,8 @@ -;;; ob-octave.el --- Babel Functions for Octave and Matlab -*- lexical-binding: t; -*- +;;; ob-octave.el --- Babel Functions for Octave and MATLAB -*- lexical-binding: t; -*- ;; Copyright (C) 2010-2024 Free Software Foundation, Inc. -;; Author: Dan Davison +;; Author: Dan Davison (Octave), John Ciolfi (MATLAB) ;; Keywords: literate programming, reproducible research ;; URL: https://orgmode.org @@ -30,6 +30,8 @@ ;;; Code: +(require 'cl-seq) + (require 'org-macs) (org-assert-version) @@ -39,7 +41,11 @@ (declare-function matlab-shell "ext:matlab-mode") (declare-function matlab-shell-run-region "ext:matlab-mode") -(defvar org-babel-default-header-args:matlab '()) +(defvar org-babel-default-header-args:matlab '((:session . "*MATLAB*")) + "Reuse the matlab-shell buffer for code block evaluations. +Add the MATLAB clear command to your code block to clear the +MATLAB workspace.") + (defvar org-babel-default-header-args:octave '()) (defvar org-babel-matlab-shell-command "matlab -nosplash" @@ -47,18 +53,42 @@ (defvar org-babel-octave-shell-command "octave -q" "Shell command to run octave as an external process.") -(defvar org-babel-matlab-with-emacs-link nil - "If non-nil use matlab-shell-run-region for session evaluation. -This will use EmacsLink if (matlab-with-emacs-link) evaluates -to a non-nil value.") - -(defvar org-babel-matlab-emacs-link-wrapper-method - "%s -if ischar(ans), fid = fopen('%s', 'w'); fprintf(fid, '%%s\\n', ans); fclose(fid); -else, save -ascii %s ans -end -delete('%s') +(make-obsolete-variable 'org-babel-matlab-with-emacs-link + "MATLAB removed EmacsLink in R2009a." "9.8") + +(make-obsolete-variable 'org-babel-matlab-emacs-link-wrapper-method + "MATLAB removed EmacsLink in R2009a." "9.8") + +(defvar org-babel-matlab-print "print(\"-dpng\", %S);\nans=%S;" + ;; MATLAB command-function duality requires that the file name be specified + ;; without quotes. Using: print -dpng "file.png", would produce a file with + ;; the quotes in the file name on disk. Therefore, use the functional form + ;; to handle files with spaces, print("-dpng", "file.png"). + ;; Example: + ;; #+begin_src matlab :file "sine wave.png" :results file graphics + ;; t = [0 : 0.1 : 2*pi]; + ;; y = sin(t); + ;; plot(t, y); + ;; set(gcf, 'PaperUnits', 'inches', 'PaperPosition', [0 0 4 3]) % Set the size to 4" x 3" + ;; #+end_src + ;; + ;; #+RESULTS: + ;; [[file:sine wave.png]] + "MATLAB format specifier to print current figure to a file.") + +(defvar org-babel-octave-print "print -dpng %S\nans=%S" + "Octave format specifier to print current figure to a file.") + +(defvar org-babel-matlab-wrapper-method + (concat "\ +cd('%s'); +%s +if ~exist('ans', 'var') ans = ''; end; \ +writematrix(ans, '%s', 'Delimiter', 'tab'); ") + "Format specifier used when evaluating MATLAB code blocks. +Arguments are the `default-directory', the MATLAB code, and a result file.txt.") + (defvar org-babel-octave-wrapper-method "%s if ischar(ans), fid = fopen('%s', 'w'); fdisp(fid, ans); fclose(fid); @@ -92,7 +122,10 @@ When MATLABP is non-nil, execute Matlab. Otherwise, execute Octave." (list "set (0, \"defaultfigurevisible\", \"off\");" full-body - (format "print -dpng %S\nans=%S" gfx-file gfx-file)) + (format (if matlabp + org-babel-matlab-print + org-babel-octave-print) + gfx-file gfx-file)) "\n") full-body) result-type matlabp))) @@ -153,6 +186,19 @@ If there is not a current inferior-process-buffer in SESSION then create. Return the initialized session. PARAMS are src block parameters." (org-babel-octave-initiate-session session params 'matlab)) +(defun org-babel-matlab-shell () + "Start and/or wait for MATLAB shell." + (require 'matlab-shell) ;; make `matlab-shell-busy-checker' available + (if (fboundp 'matlab-shell-busy-checker) + (progn + ;; Start the shell if needed. `matlab-shell' will be reused if it is already running. + (matlab-shell) + ;; If we just started the matlab-shell, wait for the prompt. If we do not + ;; wait, then the startup messages will show up in the evaluation results. + (matlab-shell-busy-checker 'wait-for-prompt)) + (user-error "A newer version of matlab-mode is required, see \ +https://github.com/mathworks/Emacs-MATLAB-Mode\n";))) + (defun org-babel-octave-initiate-session (&optional session _params matlabp) "Create an octave inferior process buffer. If there is not a current inferior-process-buffer in SESSION then @@ -165,9 +211,15 @@ Octave session, unless MATLABP is non-nil." (unless (string= session "none") (let ((session (or session (if matlabp "*Inferior Matlab*" "*Inferior Octave*")))) - (if (org-babel-comint-buffer-livep session) session + (if (org-babel-comint-buffer-livep session) + (progn + (when (and matlabp (fboundp 'matlab-shell-busy-checker)) + ;; Can't evaluate if the matlab-shell is currently running code + (matlab-shell-busy-checker 'error-if-busy)) + session) (save-window-excursion - (if matlabp (unless org-babel-matlab-with-emacs-link (matlab-shell)) + (if matlabp + (org-babel-matlab-shell) (run-octave)) (rename-buffer (if (bufferp session) (buffer-name session) (if (stringp session) session (buffer-name)))) @@ -183,79 +235,125 @@ value of the last statement in BODY, as elisp." (org-babel-octave-evaluate-session session body result-type matlabp) (org-babel-octave-evaluate-external-process body result-type matlabp))) +(defun org-babel-octave-wrapper-tmp-file (matlabp) + "Return a local tmp file with name adjusted for MATLABP." + (if matlabp + ;; writematrix requires a file ending with '.txt' + (org-babel-temp-file "matlab-" ".txt") + (org-babel-temp-file "octave-"))) + +(defun org-babel-octave-get-code-to-eval (body tmp-file matlabp) + "Format BODY of the code block for evaluation saving results to TMP-FILE. +If MATLABP, format for MATLAB, else format for Octave." + (if matlabp + (format org-babel-matlab-wrapper-method default-directory body tmp-file) + (format org-babel-octave-wrapper-method body tmp-file tmp-file))) + (defun org-babel-octave-evaluate-external-process (body result-type matlabp) - "Evaluate BODY in an external Octave or Matalab process. + "Evaluate BODY in an external Octave or MATLAB process. Process the result as RESULT-TYPE. Use Octave, unless MATLABP is non-nil." (let ((cmd (if matlabp org-babel-matlab-shell-command org-babel-octave-shell-command))) (pcase result-type (`output (org-babel-eval cmd body)) - (`value (let ((tmp-file (org-babel-temp-file "octave-"))) + (`value (let ((tmp-file (org-babel-process-file-name + (org-babel-octave-wrapper-tmp-file matlabp) + 'noquote))) (org-babel-eval cmd - (format org-babel-octave-wrapper-method body - (org-babel-process-file-name tmp-file 'noquote) - (org-babel-process-file-name tmp-file 'noquote))) + (org-babel-octave-get-code-to-eval body tmp-file matlabp)) (org-babel-octave-import-elisp-from-file tmp-file)))))) +(defvar org-babel-octave--matlab-line-indicator " %-<org-eval>\n" + "Comment appended to each code line being evaluated.") + +(defvar org-babel-octave--matlab-error-indicator-re "</?ERRORTXT>\r?\n?" + "MATLAB shell error indicators.") + +(defun org-babel-octave-body-for-output (body matlabp) + "If MATLABP, fix up BODY for MATLAB output result-type." + (when matlabp + ;; MATLAB does not echo the whole BODY all at once and instead mixes input + ;; with the output. Therefore, when we send multi-line input to + ;; `matlab-shell', we'll see the "body" code lines echoed in the output + ;; which is not what one would expect. To remove these unwanted lines, + ;; we append `org-babel-octave--matlab-line-indicator' to each line in the BODY. + ;; We leverage this indicator to remove the unwanted lines. + ;; Example of desired behavior: + ;; #+begin_src matlab :results output + ;; disp('The results are:') + ;; a = [1, 2; 3, 4] + ;; b = a * 2 + ;; #+end_src + ;; + ;; #+RESULTS: + ;; #+begin_example + ;; The results are: + ;; + ;; a = + ;; + ;; 1 2 + ;; 3 4 + ;; + ;; b = + ;; + ;; 2 4 + ;; 6 8 + ;; #+end_example + (setq body (replace-regexp-in-string "\r" "" body)) ;; CRLF => LF + (setq body (concat (string-trim-right body) "\n")) ;; Ensure a single final newline + (setq body (replace-regexp-in-string "\n" org-babel-octave--matlab-line-indicator body))) + body) + (defun org-babel-octave-evaluate-session (session body result-type &optional matlabp) "Evaluate BODY in SESSION." - (let* ((tmp-file (org-babel-temp-file (if matlabp "matlab-" "octave-"))) - (wait-file (org-babel-temp-file "matlab-emacs-link-wait-signal-")) + (let* ((tmp-file (org-babel-octave-wrapper-tmp-file matlabp)) (full-body (pcase result-type (`output (mapconcat #'org-babel-chomp - (list body org-babel-octave-eoe-indicator) "\n")) + (list (org-babel-octave-body-for-output body matlabp) + org-babel-octave-eoe-indicator) + "\n")) (`value - (if (and matlabp org-babel-matlab-with-emacs-link) - (concat - (format org-babel-matlab-emacs-link-wrapper-method - body - (org-babel-process-file-name tmp-file 'noquote) - (org-babel-process-file-name tmp-file 'noquote) wait-file) "\n") - (mapconcat - #'org-babel-chomp - (list (format org-babel-octave-wrapper-method - body - (org-babel-process-file-name tmp-file 'noquote) - (org-babel-process-file-name tmp-file 'noquote)) - org-babel-octave-eoe-indicator) "\n"))))) - (raw (if (and matlabp org-babel-matlab-with-emacs-link) - (save-window-excursion - (with-temp-buffer - (insert full-body) - (write-region "" 'ignored wait-file nil nil nil 'excl) - (matlab-shell-run-region (point-min) (point-max)) - (message "Waiting for Matlab Emacs Link") - (while (file-exists-p wait-file) (sit-for 0.01)) - "")) ;; matlab-shell-run-region doesn't seem to - ;; make *matlab* buffer contents easily - ;; available, so :results output currently - ;; won't work - (org-babel-comint-with-output - (session - (if matlabp - org-babel-octave-eoe-indicator - org-babel-octave-eoe-output) - t full-body) - (insert full-body) (comint-send-input nil t)))) - results) + (mapconcat + #'org-babel-chomp + (list (org-babel-octave-get-code-to-eval body tmp-file matlabp) + org-babel-octave-eoe-indicator) + "\n")))) + (raw-results + (org-babel-comint-with-output + (session + (if matlabp + org-babel-octave-eoe-indicator + org-babel-octave-eoe-output) + t full-body ;; Remove echo'd full-body from result + (when matlabp + `(;; MATLAB echo's input interleaved w/output, so strip inputs + ,(concat "^.*" org-babel-octave--matlab-line-indicator) + ;; Strip matlab-shell error indicators + ,org-babel-octave--matlab-error-indicator-re))) + (insert full-body) (comint-send-input nil t)))) (pcase result-type (`value (org-babel-octave-import-elisp-from-file tmp-file)) (`output - (setq results - (if matlabp - (cdr (reverse (delete "" (mapcar #'org-strip-quotes - (mapcar #'org-trim raw))))) - (cdr (member org-babel-octave-eoe-output - (reverse (mapcar #'org-strip-quotes - (mapcar #'org-trim raw))))))) - (mapconcat #'identity (reverse results) "\n"))))) + (if matlabp + (let* ((stripped (delete "" (mapcar #'org-strip-quotes raw-results))) + ;; Trim extra newline, keeping MATLAB's newlines + (trimmed (mapcar (lambda (str) + (replace-regexp-in-string "\n\n\\'" "\n" str)) + stripped)) + (reversed (cdr (reverse trimmed)))) + (concat (string-trim (mapconcat #'identity (reverse reversed)) "[\r\n]+") + "\n")) + (let ((reversed (cdr (member org-babel-octave-eoe-output + (reverse (mapcar #'org-strip-quotes + (mapcar #'org-trim raw-results))))))) + (mapconcat #'identity (reverse reversed)))))))) (defun org-babel-octave-import-elisp-from-file (file-name) "Import data from FILE-NAME. diff --git a/lisp/org.el b/lisp/org.el index 819a82eb9..1e46d238e 100644 --- a/lisp/org.el +++ b/lisp/org.el @@ -332,9 +332,10 @@ requirement." (const :tag "Lisp" lisp) (const :tag "Lua" lua) (const :tag "Makefile" makefile) + (const :tag "MATLAB" matlab) (const :tag "Maxima" maxima) (const :tag "OCaml" ocaml) - (const :tag "Octave and MatLab" octave) + (const :tag "Octave" octave) (const :tag "Org" org) (const :tag "Perl" perl) (const :tag "Processing" processing) diff --git a/mk/default.mk b/mk/default.mk index c5ea1ba01..2635d3a98 100644 --- a/mk/default.mk +++ b/mk/default.mk @@ -53,7 +53,7 @@ BTEST_POST = # -L <path-to>/ert # needed for Emacs23, Emacs24 has ert built in # -L <path-to>/ess # needed for running R tests # -L <path-to>/htmlize # need at least version 1.34 for source code formatting -BTEST_OB_LANGUAGES = awk C fortran maxima lilypond octave perl python java sqlite eshell calc +BTEST_OB_LANGUAGES = awk C fortran matlab maxima lilypond octave perl python java sqlite eshell calc # R # requires ESS to be installed and configured # ruby # requires inf-ruby to be installed and configured # extra packages to require for testing diff --git a/testing/examples/ob-matlab-test.org b/testing/examples/ob-matlab-test.org new file mode 100644 index 000000000..63500f223 --- /dev/null +++ b/testing/examples/ob-matlab-test.org @@ -0,0 +1,114 @@ +#+Title: A collection of examples for ob-matlab tests +#+OPTIONS: ^:nil + +* Test MATLAB results output +:PROPERTIES: +:ID: 99332277-5e0e-4834-a3ad-ebcaddb855ab +:END: + +#+begin_src matlab :results output + disp('The results are:') + a = [1, 2; 3, 4] + b = a * 2 +#+end_src + +#+RESULTS: +#+begin_example +The results are: + +a = + + 1 2 + 3 4 + +b = + + 2 4 + 6 8 +#+end_example + +Following validates whitespace is preserved + +#+begin_src matlab :results output + disp(' +---+') + disp(' |one|') + disp(' +---+') + disp(newline) + disp([' ', num2str(1)]); + disp(newline) + disp(' +---+') + disp(' |two|') + disp(' +---+') + disp(newline) + disp([' ', num2str(2)]); + disp(newline) +#+end_src + +* Test reuse of MATLAB buffer with results output +:PROPERTIES: +:ID: a68db9ff-efdf-42b9-85fc-174015cd1f77 +:END: + +#+begin_src matlab :results output + a = 123 +#+end_src + +#+RESULTS: +: a = +: +: 123 + +#+begin_src matlab :exports both :results output + b = a + 1000 +#+end_src + +#+RESULTS: +: b = +: +: 1123 + +Following should give: Unrecognized function or variable 'b'. + +#+begin_src matlab :results output + clear + c = b * 2 +#+end_src + +#+RESULTS: +: Unrecognized function or variable 'b'. + +* Test results verbatim +:PROPERTIES: +:ID: 278047b6-4b87-4852-9050-e3e99fcaabb8 +:END: + +#+begin_src matlab :results verbatim + a = 2 + 3; + ans = magic(a); +#+end_src + +* Test results output latex +:PROPERTIES: +:ID: 7a8190be-d674-4944-864e-6fdaa7362585 +:END: + +#+begin_src matlab :results output latex + m = [4*pi, 3*pi; 2*pi, pi]; + result = latex(sym(m)); + disp(result) +#+end_src + +* Test results file graphics +:PROPERTIES: +:ID: 5bee8841-a898-4135-b44b-f1bd5465ceed +:END: + +#+begin_src matlab :results file graphics :file NAME.png + t = [0 : 0.1 : 2*pi]; + y = sin(t); + plot(t, y); + set(gcf, 'PaperUnits', 'inches', 'PaperPosition', [0 0 4 3]) % Set the size to 4" x 3" +#+end_src + + +# LocalWords: ebcaddb efdf fc fcaabb fdaa ceed sinewave diff --git a/testing/lisp/test-ob-matlab.el b/testing/lisp/test-ob-matlab.el new file mode 100644 index 000000000..476e5b1e4 --- /dev/null +++ b/testing/lisp/test-ob-matlab.el @@ -0,0 +1,191 @@ +;;; test-ob-matlab.el --- tests for ob-matlab.el -*- lexical-binding: t; -*- + +;;; Commentary: + +;; Copyright 2024 Free Software Foundation +;; Authors: John Ciolfi + +;; This file is not part of GNU Emacs. + +;; This program is free software; you can redistribute it and/or modify +;; it under the terms of the GNU General Public License as published by +;; the Free Software Foundation, either version 3 of the License, or +;; (at your option) any later version. + +;; This program is distributed in the hope that it will be useful, +;; but WITHOUT ANY WARRANTY; without even the implied warranty of +;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +;; GNU General Public License for more details. + +;; You should have received a copy of the GNU General Public License +;; along with this program. If not, see <https://www.gnu.org/licenses/>. + +;; ---------- +;; Code blocks reside in: ../examples/ob-matlab-test.org +;; To run test-ob-matlab.el using latest matlab-mode and org-mode: +;; 1. Download and build latest org-mode, https://orgmode.org/worg/org-contribute.html +;; Example: +;; cd ~/github +;; git clone https://git.sv.gnu.org/git/emacs/org-mode.git +;; make -C org-mode compile -j32 && make prefix=~/github/org-mode-install install +;; 2. Download and build latest matlab-mode, https://github.com/MathWorks/Emacs-MATLAB-Mode +;; Example: +;; cd ~/github +;; git clone https://github.com/mathworks/Emacs-MATLAB-Mode.git +;; make -C Emacs-MATLAB-Mode -j32 +;; 3. Test uses matlab-shell which requires that "matlab" is on the PATH: +;; Example: +;; cd ~/github/org-mode +;; env PATH=/path/to/MATLAB-INSTALL/bin:$PATH \ +;; make -j32 test \ +;; BTEST_POST="-L ~/github/Emacs-MATLAB-Mode -l ~/Emacs-MATLAB-Mode/matlab-autoload.el" + +;;; Code: + +(require 'ob-core) + +(org-test-for-executable "matlab") + +(unless (fboundp 'matlab-shell) + (signal 'missing-test-dependency '("Support for MATLAB code blocks"))) + +;;-------------------------------;; +;; Basic ":results output" tests ;; +;;-------------------------------;; +(ert-deftest ob-matlab/results-output () + "Test matlab :results output code block." + (let ((expected "\ +The results are: + +a = + + 1 2 + 3 4 + +b = + + 2 4 + 6 8 +")) + (org-test-at-id "99332277-5e0e-4834-a3ad-ebcaddb855ab" + (org-babel-next-src-block) + (should (equal expected (org-babel-execute-src-block)))))) + +(ert-deftest ob-matlab/results-output-preserve-whitespace () + "Test matlab :results output code block preserves whitespace." + (let ((expected "\ + +---+ + |one| + +---+ + + 1 + + +---+ + |two| + +---+ + + 2 +")) + (org-test-at-id "99332277-5e0e-4834-a3ad-ebcaddb855ab" + (org-babel-next-src-block 2) + (should (equal expected (org-babel-execute-src-block)))))) + +;;------------------------------------------------;; +;; MATLAB workspace reuse tests (:results output) ;; +;;------------------------------------------------;; + +(ert-deftest ob-matlab/results-output-reuse-a () + "Test matlab :results output defining variable, a." + (let ((expected "\ +a = + + 123 +")) + (org-test-at-id "a68db9ff-efdf-42b9-85fc-174015cd1f77" + (org-babel-next-src-block) + (should (equal expected (org-babel-execute-src-block)))))) + +(ert-deftest ob-matlab/results-output-reuse-b () + "Test matlab :results output reusing variable, a, to compute variable, b." + (let ((expected "\ +b = + + 1123 +")) + (org-test-at-id "a68db9ff-efdf-42b9-85fc-174015cd1f77" + (org-babel-next-src-block 2) + (should (equal expected (org-babel-execute-src-block)))))) + +(ert-deftest ob-matlab/results-output-reuse-clear () + "Test matlab :results output with clear resulting in error. +Also validates that we strip the </?ERRORTXT> matlab-shell indicators." + (let ((expected "\ +Unrecognized function or variable 'b'. +")) + (org-test-at-id "a68db9ff-efdf-42b9-85fc-174015cd1f77" + (org-babel-next-src-block 3) + (should (equal expected (org-babel-execute-src-block)))))) + +;;---------------------------;; +;; ":results verbatim" tests ;; +;;---------------------------;; +(ert-deftest ob-matlab/results-verbatim () + "Test matlab :results verbatim." + (let ((expected '((17 24 1 8 15) + (23 5 7 14 16) + (4 6 13 20 22) + (10 12 19 21 3) + (11 18 25 2 9)))) + (org-test-at-id "278047b6-4b87-4852-9050-e3e99fcaabb8" + (org-babel-next-src-block) + (should (equal expected (org-babel-execute-src-block)))))) + +;;-------------------------------;; +;; ":results output latex" tests ;; +;;-------------------------------;; + +(ert-deftest ob-matlab/results-output-latex () + "Test matlab :results output latex." + (let ((expected "\ +\\left(\\begin{array}{cc} 4\\,\\pi & 3\\,\\pi \\\\ 2\\,\\pi & \\pi \\end{array}\\right) +")) + (org-test-at-id "7a8190be-d674-4944-864e-6fdaa7362585" + (org-babel-next-src-block) + (let ((got (org-babel-execute-src-block))) + (should (equal expected got)))))) + + +;;--------------------------------;; +;; ":results file graphics" tests ;; +;;--------------------------------;; + +(ert-deftest ob-matlab/results-file-graphics () + "Test matlab :results file graphics." + (let ((code-block (org-test-get-code-block "5bee8841-a898-4135-b44b-f1bd5465ceed")) + (temp-file-png (make-temp-file "test-ob-matlab-" nil ".png"))) + (setq code-block (replace-regexp-in-string "NAME\\.png" temp-file-png code-block)) + (unwind-protect + (org-test-with-temp-text + code-block + (org-babel-execute-src-block) + (should (search-forward (format "[[file:%s]]" temp-file-png) nil nil)) + (should (file-readable-p temp-file-png))) + (delete-file temp-file-png)))) + +(ert-deftest ob-matlab/results-file-graphics-with-space () + "Test matlab :results file graphics using a file name with a space." + (let ((code-block (org-test-get-code-block "5bee8841-a898-4135-b44b-f1bd5465ceed")) + (temp-file-png (make-temp-file "test ob-matlab-" nil ".png"))) + (setq code-block (replace-regexp-in-string "NAME\\.png" temp-file-png code-block)) + (unwind-protect + (org-test-with-temp-text + code-block + (org-babel-execute-src-block) + (should (search-forward (format "[[file:%s]]" temp-file-png) nil nil)) + (should (file-readable-p temp-file-png))) + (delete-file temp-file-png)))) + +(provide 'test-ob-matlab) +;;; test-ob-matlab.el ends here (emacs-lisp-checkdoc) + +;; LocalWords: fboundp ebcaddb efdf fc ERRORTXT fcaabb fdaa ceed setq env BTEST diff --git a/testing/org-test.el b/testing/org-test.el index 52d38c3fd..93dfa46fc 100644 --- a/testing/org-test.el +++ b/testing/org-test.el @@ -133,6 +133,27 @@ currently executed.") (unless (or visited-p (not to-be-removed)) (kill-buffer to-be-removed))))) +(defun org-test-get-code-block (id &optional count) + "Get the COUNT, defaulting to first, code block after ID." + (let* ((id-location (org-id-find id)) + (id-file (car id-location)) + (visited-p (get-file-buffer id-file)) + code-block + to-be-removed) + (unwind-protect + (save-window-excursion + (save-match-data + (org-id-goto id) + (org-babel-next-src-block count) + (beginning-of-line) + (setq code-block (buffer-substring-no-properties + (point) + (re-search-forward "^[ \t]*#\\+end_src[ \t\r]*\n"))) + (setq to-be-removed (current-buffer)))) + (unless (or visited-p (not to-be-removed)) + (kill-buffer to-be-removed))) + code-block)) + (defmacro org-test-in-example-file (file &rest body) "Execute body in the Org example file." (declare (indent 1) (debug t))
