Re: Bug: Following a link to a #+NAME causes '(wrong-type-argument stringp nil)' [9.3 (release_9.3 @ /usr/share/emacs/28.0.50/lisp/org/)]

2020-12-01 Thread Dima Kogan
Kyle Meyer  writes:

> Thanks for the report.  As suggested in Dante's reply, the blank line
> following #+name is invalid syntax.
>
>   https://orgmode.org/worg/dev/org-syntax.html#Affiliated_keywords
>
> On maint (3bb073b63), the type error is now avoided for invalid name
> keywords.  No attempt is made to match these keywords, though, so your
> example won't lead to a match aside from through a fuzzy text search
> when org-link-search-must-match-exact-headline is nil.

Thank you Kyle, Dante. I was using the wrong thing. I should have been
using the CUSTOM_ID property instead of the #+NAME tag. This was my
fault. I guess the fact that #+NAME was ALMOST working the way I wanted
it to increased my confusion. And the unhelpful error message.

Thanks again



Bug: Org exporter: broken-link errors can't be debugged with debug-on-error [9.3 (release_9.3 @ /usr/share/emacs/28.0.50/lisp/org/)]

2020-11-30 Thread Dima Kogan
Hi. I'm using the org included with a very recent emacs built from git.

I have a large project consisting of many .org files that I'm exporting
to html. Somewhere there's a broken link, so when I export the project I
get

  user-error: Unable to resolve link: "figures/blahblahblah.svg"

This doesn't tell me where the problem is, specifically, so to find out
I

  (setq debug-on-error t)

and go again. But something about the org code is preventing the
debugger from triggering on this. That really shouldn't be happening.
This is an error that's causing the export to give up and quit, and the
debugger should come up (when debug-on-error).

Also, it would be nice if the error message reported the source file and
line number of the bad link.

Thanks!
Please Cc me in replies; I'm not subscribed to the list.

Emacs  : GNU Emacs 28.0.50 (build 1, x86_64-pc-linux-gnu, GTK+ Version 3.24.23, 
cairo version 1.16.0)
 of 2020-09-29, unofficial emacs-snapshot build: http://emacs.secretsauce.net
Package: Org mode version 9.3 (release_9.3 @ /usr/share/emacs/28.0.50/lisp/org/)



Bug: Following a link to a #+NAME causes '(wrong-type-argument stringp nil)' [9.3 (release_9.3 @ /usr/share/emacs/28.0.50/lisp/org/)]

2020-11-22 Thread Dima Kogan
Hi. I'm using the org included with a recent build from emacs git. I
have this tst.org:




[[name][link]]

* heading
#+NAME: name

text




I open it with 'emacs -Q'. I move the point to the link at the top, and
C-c C-o to follow the link. This doesn't work:

  Debugger entered--Lisp error: (wrong-type-argument stringp nil)
string-match("[ \f\11\n\15\13]+" nil 0)
split-string(nil)
org-link-search("name" 3)
org-link-open((link (:type "fuzzy" :path "name" :format bracket :raw-link 
"name" :application nil :search-option nil :begin 1 :end 15 :contents-begin 9 
:contents-end 13 :post-blank 0 :parent (paragraph (:begin 1 :end 17 
:contents-begin 1 :contents-end 16 :post-blank 1 :post-affiliated 1 :parent 
nil nil)
org-open-at-point(nil)
funcall-interactively(org-open-at-point nil)

Apparently org doesn't like the empty line right after the #+NAME tag.
Removing that empty line makes it work.

Thanks!
 

Emacs  : GNU Emacs 28.0.50 (build 1, x86_64-pc-linux-gnu, GTK+ Version 3.24.23, 
cairo version 1.16.0)
 of 2020-09-29, unofficial emacs-snapshot build: http://emacs.secretsauce.net
Package: Org mode version 9.3 (release_9.3 @ /usr/share/emacs/28.0.50/lisp/org/)



Nicer export of an ipython-notebook style file

2020-03-10 Thread Dima Kogan
Hi.

Recently I used org-babel to create documentation for a python plotting
library. The end-product of this documentation is a sequence of code
snippets and images (the result of evaluating the snippets).

A wrinkle is that I didn't want to use org to do the export. I'd use org
to generate all the images, and then push the .org to github, and use
its (half-assed) renderer to display the results.

This mostly worked out of the box, but some elisp was needed to get this
done fully. So this email is meant to

1. Ask the question "what is the right way to have done this?"

2. Talk about the elisp I wrote to get this done, and maybe get some of
   it merged upstream

The final product lives here:

  https://github.com/dkogan/gnuplotlib/blob/master/guide/guide.org

The big "init" block at the end is the emacs lisp in question. I can't
get github to not display it, but that's a separate thing.

Now, each required piece, one at a time.


Each python block needs to know the file it should write the results to.
Org and python need to agree about this. It looks like some org-babel
backends do this (gnuplot for example), but in general there's no way. I
wrote a lisp thing to make ALL the org-babel headers for that snippet
available to the python block in a dict called "_org_babel_params". So
the python code writes its output to _org_babel_params['_file']. Clearly
this is specific to python, but any backend that has any sort of
associative array could maybe support something like this. The code:

  (defun dima-org-babel-python-var-to-python (var)
"Convert an elisp value to a python variable.
Like the original, but supports (a . b) cells and symbols
"
(if (listp var)
(if (listp (cdr var))
(concat "[" (mapconcat #'org-babel-python-var-to-python var ", ") 
"]")
  (format "\"\"\"%s\"\"\"" var))
  (if (symbolp var)
  (format "\"\"\"%s\"\"\"" var)
(if (eq var 'hline)
org-babel-python-hline-to
  (format
   (if (and (stringp var) (string-match "[\n\r]" var)) "\"\"%S\"\"" 
"%S")
   (if (stringp var) (substring-no-properties var) var))
  (defun dima-alist-to-python-dict (alist)
"Generates a string defining a python dict from the given alist"
(let ((keyvalue-list
   (mapcar (lambda (x)
 (format "%s = %s, "
 (replace-regexp-in-string
  "[^a-zA-Z0-9_]" "_"
  (symbol-name (car x)))
 (dima-org-babel-python-var-to-python (cdr x
   alist)))
  (concat
   "dict( "
   (apply 'concat keyvalue-list)
   ")")))
  (defun dima-org-babel-python-pass-all-params (f params)
(cons
 (concat
  "_org_babel_params = "
  (dima-alist-to-python-dict params))
 (funcall f params)))
  (advice-add
   #'org-babel-variable-assignments:python
   :around #'dima-org-babel-python-pass-all-params)

Now that org-babel can tell python where to write its output, we need to
tell org-babel this. I don't actually care what the output file is
called, as long as it's unique. So each of my org-babel snippets does
not specify the :file at all, instead I advice
org-babel-execute-src-block to set a :file with "guide-%d.svg" where the
%d is an integer that's incremented with each block:

  (defun dima-org-babel-python-unique-plot-filename
  (f  arg info params)
(funcall f arg info
 (cons (cons ':file
 (format "guide-%d.svg"
 (condition-case nil
 (setq dima-unique-plot-number (1+ 
dima-unique-plot-number))
   (error (setq dima-unique-plot-number 0)
   params)))
  (advice-add
   #'org-babel-execute-src-block
   :around #'dima-org-babel-python-unique-plot-filename)

I'd like the count to start from 0 each time I (org-babel-execute-buffer):

  (defun dima-reset-unique-plot-number
  ( args)
  (setq dima-unique-plot-number 0))
  (advice-add
   #'org-babel-execute-buffer
   :after #'dima-reset-unique-plot-number)


Was there a better way to do this? Any of these advices upstreamable?

Thanks.



[O] bug#22472: 25.0.50; org-mode: latex fragments can't be un-rendered after a revert

2016-02-06 Thread Dima Kogan
Nicolas Goaziou <m...@nicolasgoaziou.fr> writes:

> Dima Kogan <d...@secretsauce.net> writes:
>
>> Ah. Thanks for explaining. The patch helps somewhat, but one can still
>> get into an inconsistent state:
>>
>>   1. Write \(1 + 2\)
>>   2. Toggle overlay with C-c C-x C-l
>>   3. M-x revert-buffer
>>
>> After the revert, the overlay remains, which is arguably OK, since the
>> text has not changed. However after the revert org doesn't realize that
>> the overlay is still up: org-latex-fragment-image-overlays is nil.
>
> Indeed. 
>
> `org-latex-fragment-image-overlays' is a local variable and
> `revert-buffer' calls `normal-mode', which, in turn, calls
> `fundamental-mode'. The latter calls `kill-all-local-variables'.
> Information is lost.
>
> I don't think that `org-latex-fragment-image-overlays' is useful,
> anyway. So I removed it, along with that bug, hopefully.

Yes, things look consistent after your patch.


>> I think the overlays should all disappear on a revert.
>
> I'm not convinced that an unmodified overlay should disappear.

Sure. If we can keep everything consistent, then keeping the overlays is
probably better.

Thank you very much! I'm closing this bug.





[O] bug#22472: 25.0.50; org-mode: latex fragments can't be un-rendered after a revert

2016-02-04 Thread Dima Kogan
Nicolas Goaziou <m...@nicolasgoaziou.fr> writes:

> Dima Kogan <d...@secretsauce.net> writes:
>
>> Nicolas Goaziou <m...@nicolasgoaziou.fr> writes:
>>
>>> Would the following patch (applied on maint) solve the problem?
>>
>> Hi. I didn't observe anything acting differently with this patch. What
>> did you see it do? Was it supposed to unrender on revert, or to keep the
>> render, but update the state?
>
> As the commit message says, it is supposed to remove the overlay when
> text below is modified (e.g. when using a replace-regexp). E.g.,
>
>  1. Write \(1 + 2\)
>
>  2. Toggle overlay with C-c C-x C-l
>
>  3. M-% 2 RET 3 RET
>
>  4. The overlay should have disappeared
>
> I didn't test it with the `revert-buffer' command, but I would be
> surprised that it doesn't run any modification hook.

Ah. Thanks for explaining. The patch helps somewhat, but one can still
get into an inconsistent state:

  1. Write \(1 + 2\)
  2. Toggle overlay with C-c C-x C-l
  3. M-x revert-buffer

After the revert, the overlay remains, which is arguably OK, since the
text has not changed. However after the revert org doesn't realize that
the overlay is still up: org-latex-fragment-image-overlays is nil.

I think the overlays should all disappear on a revert.

Thanks!





[O] bug#22472: 25.0.50; org-mode: latex fragments can't be un-rendered after a revert

2016-02-04 Thread Dima Kogan
Nicolas Goaziou  writes:

> Would the following patch (applied on maint) solve the problem?

Hi. I didn't observe anything acting differently with this patch. What
did you see it do? Was it supposed to unrender on revert, or to keep the
render, but update the state?





[O] bug#22472: 25.0.50; org-mode: latex fragments can't be un-rendered after a revert

2016-02-03 Thread Dima Kogan
Hi. Thank you for replying. I just tried with the latest daily snapshot
from org-mode.org, and this bug is still valid, although the problem at
least becomes recoverable without killing and re-opening the org buffer.


Bastien Guerry <b...@gnu.org> writes:

> Dima Kogan <d...@secretsauce.net> writes:
>
>> 3. C-c C-x C-l
>>
>>This renders the latex fragment to show the equation graphically. It
>>also tells the user (in the mini-buffer) that C-c C-c will remove the
>>rendering, and go back to text
>
> Latest stable version from Org does not use C-c C-c to remove the
> rendering.

Indeed C-c C-c no longer does this. There're no instructions in the
mini-buffer about how to unrender anymore. C-c C-x C-l is supposed
unrender anything that's rendered, and this works.


>> 4. M-x revert-buffer
>>
>>At this point I would expect the render to revert to text, but it
>>doesn't. This isn't necessarily a bug. However...
>
> (Why would reverting the buffer unrender the LaTeX fragment if the
> text file hasn't change?)

The latest org-mode still keeps the renders on a revert. This is a bug
because when a revert happens, emacs does not assume that the file
hasn't changed. In fact, it COULD have changed, so keeping the (possibly
out-of-date) render up is just wrong. Contrived example:

1. tst.org contains $\sqrt x$
2. C-c C-x C-l to render it
3. in the shell (outside of emacs): perl -p -i -e s/sqrt/sin/ tst.org
4. M-x revert-buffer
   Observe that the render stays up, even though the latex the render
   represents is no longer there

>> 5. C-c C-c
>>
>>Here I ask emacs to un-render the fragment, but it does nothing. If
>>#4 isn't a bug, then this is definitely a bug.

Right. But I can now use C-c C-x C-l again. Continuing instructions from
above

5. C-c C-x C-l
   One would expect this to unrender the equation, but org thinks that
   it is not rendered, so it attempts to render it. You get a new render
   of \sin x

6. C-c C-x C-l
   This is now an unrender to get back to the new text



So a revert puts emacs into an inconsistent state, where the equation is
rendered, but the source may or may not be there anymore, and where org
doesn't think there's a render there at all.





[O] bug#22472: 25.0.50; org-mode: latex fragments can't be un-rendered after a revert

2016-02-03 Thread Dima Kogan
Dima Kogan <d...@secretsauce.net> writes:

> So a revert puts emacs into an inconsistent state, where the equation is
> rendered, but the source may or may not be there anymore, and where org
> doesn't think there's a render there at all.

This looks like an issue in emacs, rather than org. I think the right
solution is for a revert to delete all overlays. I.e.:

  (add-hook 'before-revert-hook 'delete-all-overlays)

This generally works, but is unideal because:


1. We'd want this to apply to org buffers, not to all emacs buffers

2. This works only if revert-buffer-function is nil or
revert-buffer--default.


If we don't want to delete overlays on revert, then
org-latex-fragment-image-overlays must be properly set to reflect the
overlay state, AND emacs should make sure the overlay is still valid on
revert.

Any particular thoughts?





Re: [O] Adding new table rows/cols in a formula update

2014-10-11 Thread Dima Kogan
Nicolas Goaziou m...@nicolasgoaziou.fr writes:

 Thanks for your work on this, the patch looks good but it cannot be
 accepted as a tiny change.  Would you be willing to sign the FSF
 copyright assignment papers?

 Actually, it's not a tiny change. He signed copyright assignment
 already. See http://permalink.gmane.org/gmane.emacs.orgmode/91352

Here's the same patch without the TINYCHANGE marker, if that's helpful.

From 6a361837f1f7b71a02ab5b918509def84c9fa43e Mon Sep 17 00:00:00 2001
From: Dima Kogan d...@secretsauce.net
Date: Tue, 30 Sep 2014 22:36:21 -0700
Subject: [PATCH] org-table: Field formulas can now create columns as needed

* org-table.el (org-table-formula-create-columns): New variable.
(org-table-recalculate): Use the new org-table-formula-make-new-cols
customization to control whether org creates new columns when
a formula explicitly targets them.
---
 etc/ORG-NEWS   |  6 +
 lisp/org-table.el  | 35 +++---
 testing/lisp/test-org-table.el | 56 ++
 3 files changed, 94 insertions(+), 3 deletions(-)

diff --git a/etc/ORG-NEWS b/etc/ORG-NEWS
index 1af54ad..0a5af68 100644
--- a/etc/ORG-NEWS
+++ b/etc/ORG-NEWS
@@ -90,6 +90,12 @@ docstrings (including ~orgtbl-to-generic~) for details.
 *** Non-floating minted listings in Latex export
 It is not possible to specify =#+attr_latex: :float nil= in conjunction with
 source blocks exported by the minted package.
+*** Field formulas can now create columns as needed
+Previously, evaluating formulas that referenced out-of-bounds columns
+would throw an error. A new variable
+~org-table-formula-create-columns~ was added to adjust this
+behavior. It is now possible to silently add new columns, to do so
+with a warning or to explicitly ask the user each time.
 ** Miscellaneous
 *** File names in links accept are now compatible with URI syntax
 Absolute file names can now start with =///= in addition to =/=. E.g.,
diff --git a/lisp/org-table.el b/lisp/org-table.el
index 7607ead..14c68d6 100644
--- a/lisp/org-table.el
+++ b/lisp/org-table.el
@@ -354,6 +354,18 @@ portability of tables.
 	  (const :tag Stick to hline nil)
 	  (const :tag Error on attempt to cross error)))
 
+(defcustom org-table-formula-create-columns nil
+  Non-nil means that evaluation of a field formula can add new
+columns if an out-of-bounds field is being set.
+  :group 'org-table-calculation
+  :version 24.5
+  :package-version '(Org . 8.3)
+  :type '(choice
+	  (const :tag Setting an out-of-bounds field generates an error (default) nil)
+	  (const :tag Setting an out-of-bounds field silently adds columns as needed t)
+	  (const :tag Setting an out-of-bounds field adds columns as needed, but issues a warning message warn)
+	  (const :tag When setting an out-of-bounds field, the user is prompted prompt)))
+
 (defgroup org-table-import-export nil
   Options concerning table import and export in Org-mode.
   :tag Org Table Import Export
@@ -3125,9 +3137,26 @@ known that the table will be realigned a little later anyway.
   (while (setq eq (pop eqlname1))
 	(message Re-applying formula to field: %s (car eq))
 	(org-goto-line (nth 1 eq))
-	(org-table-goto-column (nth 2 eq))
-	(org-table-eval-formula nil (nth 3 eq) 'noalign 'nocst
-'nostore 'noanalysis))
+	(let ((column-target (nth 2 eq)))
+	  (when ( column-target 1000)
+	(user-error Formula column target too large))
+	  (let* ((column-count (progn (end-of-line)
+  (1- (org-table-current-column
+		 (create-new-column
+		  (and ( column-target column-count)
+		   (or (eq org-table-formula-create-columns t)
+			   (and
+			(eq org-table-formula-create-columns 'warn)
+			(progn
+			  (org-display-warning Out-of-bounds formula added columns)
+			  t))
+			   (and
+			(eq org-table-formula-create-columns 'prompt)
+			(yes-or-no-p Out-of-bounds formula. Add columns?))
+	(org-table-goto-column column-target nil create-new-column))
+
+	  (org-table-eval-formula nil (nth 3 eq) 'noalign 'nocst
+  'nostore 'noanalysis)))
 
   (org-goto-line thisline)
   (org-table-goto-column thiscol)
diff --git a/testing/lisp/test-org-table.el b/testing/lisp/test-org-table.el
index e083683..d99db27 100644
--- a/testing/lisp/test-org-table.el
+++ b/testing/lisp/test-org-table.el
@@ -1550,6 +1550,62 @@ See also `test-org-table/copy-field'.
 	 (progn (search-forward # END RECEIVE ORGTBL table)
 		(match-beginning 0)))
 
+(ert-deftest test-org-table/field-formula-outside-table ()
+  If `org-table-formula-create-columns' is nil, then a formula
+that references an out-of-bounds column should do nothing. If it
+is t, then new columns should be added as needed
+
+  (let ((org-table-formula-create-columns nil))
+
+;; need condition-case to trap the out-of-bounds user-error
+(condition-case
+	nil
+	(org-test-table-target-expect
+	 
+| 2 |
+| 4 |
+| 8 |
+
+	 
+| 2 |
+| 4 |
+| 8 |
+
+	 1
+	 #+TBLFM: @1

Re: [O] Adding new table rows/cols in a formula update

2014-10-10 Thread Dima Kogan
Nicolas Goaziou m...@nicolasgoaziou.fr writes:

 Would you mind providing a test for it in org-test-table.el and an entry
 in ORG-NEWS?

 The commit message should be something like the following...

OK. New patch attached.

From 273e642c937f0e12a2f71cf6499415406d708629 Mon Sep 17 00:00:00 2001
From: Dima Kogan d...@secretsauce.net
Date: Tue, 30 Sep 2014 22:36:21 -0700
Subject: [PATCH] org-table: Field formulas can now create columns as needed

* org-table.el (org-table-formula-create-columns): New variable.
(org-table-recalculate): Use the new org-table-formula-make-new-cols
customization to control whether org creates new columns when
a formula explicitly targets them.

TINYCHANGE
---
 etc/ORG-NEWS   |  6 +
 lisp/org-table.el  | 35 +++---
 testing/lisp/test-org-table.el | 56 ++
 3 files changed, 94 insertions(+), 3 deletions(-)

diff --git a/etc/ORG-NEWS b/etc/ORG-NEWS
index 1af54ad..0a5af68 100644
--- a/etc/ORG-NEWS
+++ b/etc/ORG-NEWS
@@ -90,6 +90,12 @@ docstrings (including ~orgtbl-to-generic~) for details.
 *** Non-floating minted listings in Latex export
 It is not possible to specify =#+attr_latex: :float nil= in conjunction with
 source blocks exported by the minted package.
+*** Field formulas can now create columns as needed
+Previously, evaluating formulas that referenced out-of-bounds columns
+would throw an error. A new variable
+~org-table-formula-create-columns~ was added to adjust this
+behavior. It is now possible to silently add new columns, to do so
+with a warning or to explicitly ask the user each time.
 ** Miscellaneous
 *** File names in links accept are now compatible with URI syntax
 Absolute file names can now start with =///= in addition to =/=. E.g.,
diff --git a/lisp/org-table.el b/lisp/org-table.el
index 7607ead..14c68d6 100644
--- a/lisp/org-table.el
+++ b/lisp/org-table.el
@@ -354,6 +354,18 @@ portability of tables.
 	  (const :tag Stick to hline nil)
 	  (const :tag Error on attempt to cross error)))
 
+(defcustom org-table-formula-create-columns nil
+  Non-nil means that evaluation of a field formula can add new
+columns if an out-of-bounds field is being set.
+  :group 'org-table-calculation
+  :version 24.5
+  :package-version '(Org . 8.3)
+  :type '(choice
+	  (const :tag Setting an out-of-bounds field generates an error (default) nil)
+	  (const :tag Setting an out-of-bounds field silently adds columns as needed t)
+	  (const :tag Setting an out-of-bounds field adds columns as needed, but issues a warning message warn)
+	  (const :tag When setting an out-of-bounds field, the user is prompted prompt)))
+
 (defgroup org-table-import-export nil
   Options concerning table import and export in Org-mode.
   :tag Org Table Import Export
@@ -3125,9 +3137,26 @@ known that the table will be realigned a little later anyway.
   (while (setq eq (pop eqlname1))
 	(message Re-applying formula to field: %s (car eq))
 	(org-goto-line (nth 1 eq))
-	(org-table-goto-column (nth 2 eq))
-	(org-table-eval-formula nil (nth 3 eq) 'noalign 'nocst
-'nostore 'noanalysis))
+	(let ((column-target (nth 2 eq)))
+	  (when ( column-target 1000)
+	(user-error Formula column target too large))
+	  (let* ((column-count (progn (end-of-line)
+  (1- (org-table-current-column
+		 (create-new-column
+		  (and ( column-target column-count)
+		   (or (eq org-table-formula-create-columns t)
+			   (and
+			(eq org-table-formula-create-columns 'warn)
+			(progn
+			  (org-display-warning Out-of-bounds formula added columns)
+			  t))
+			   (and
+			(eq org-table-formula-create-columns 'prompt)
+			(yes-or-no-p Out-of-bounds formula. Add columns?))
+	(org-table-goto-column column-target nil create-new-column))
+
+	  (org-table-eval-formula nil (nth 3 eq) 'noalign 'nocst
+  'nostore 'noanalysis)))
 
   (org-goto-line thisline)
   (org-table-goto-column thiscol)
diff --git a/testing/lisp/test-org-table.el b/testing/lisp/test-org-table.el
index e083683..d99db27 100644
--- a/testing/lisp/test-org-table.el
+++ b/testing/lisp/test-org-table.el
@@ -1550,6 +1550,62 @@ See also `test-org-table/copy-field'.
 	 (progn (search-forward # END RECEIVE ORGTBL table)
 		(match-beginning 0)))
 
+(ert-deftest test-org-table/field-formula-outside-table ()
+  If `org-table-formula-create-columns' is nil, then a formula
+that references an out-of-bounds column should do nothing. If it
+is t, then new columns should be added as needed
+
+  (let ((org-table-formula-create-columns nil))
+
+;; need condition-case to trap the out-of-bounds user-error
+(condition-case
+	nil
+	(org-test-table-target-expect
+	 
+| 2 |
+| 4 |
+| 8 |
+
+	 
+| 2 |
+| 4 |
+| 8 |
+
+	 1
+	 #+TBLFM: @1$2=5)
+  ('user-error t)))
+
+  (let ((org-table-formula-create-columns t))
+
+;; make sure field formulas work
+(org-test-table-target-expect
+ 
+| 2 |
+| 4 |
+| 8

Re: [O] Adding new table rows/cols in a formula update

2014-10-03 Thread Dima Kogan
Nick Dokos ndo...@gmail.com writes:

 Nicolas Goaziou m...@nicolasgoaziou.fr writes:

 Does it need sanity checking? A typo in a formula can create billions
 and billions[fn:1] of columns inadvertently.

Sure. Updated patch attached.

From 06f09e58a0f8b62429dda2faa913e297218b67b2 Mon Sep 17 00:00:00 2001
From: Dima Kogan d...@secretsauce.net
Date: Tue, 30 Sep 2014 22:36:21 -0700
Subject: [PATCH] org-table.el: field formulas can now create columns as needed

(org-table-recalculate): use the new org-table-formula-make-new-cols
customization to control whether org creates new columns when a
formula explicitly targets them

TINYCHANGE
---
 lisp/org-table.el | 35 ---
 1 file changed, 32 insertions(+), 3 deletions(-)

diff --git a/lisp/org-table.el b/lisp/org-table.el
index 7607ead..14c68d6 100644
--- a/lisp/org-table.el
+++ b/lisp/org-table.el
@@ -354,6 +354,18 @@ portability of tables.
 	  (const :tag Stick to hline nil)
 	  (const :tag Error on attempt to cross error)))
 
+(defcustom org-table-formula-create-columns nil
+  Non-nil means that evaluation of a field formula can add new
+columns if an out-of-bounds field is being set.
+  :group 'org-table-calculation
+  :version 24.5
+  :package-version '(Org . 8.3)
+  :type '(choice
+	  (const :tag Setting an out-of-bounds field generates an error (default) nil)
+	  (const :tag Setting an out-of-bounds field silently adds columns as needed t)
+	  (const :tag Setting an out-of-bounds field adds columns as needed, but issues a warning message warn)
+	  (const :tag When setting an out-of-bounds field, the user is prompted prompt)))
+
 (defgroup org-table-import-export nil
   Options concerning table import and export in Org-mode.
   :tag Org Table Import Export
@@ -3125,9 +3137,26 @@ known that the table will be realigned a little later anyway.
   (while (setq eq (pop eqlname1))
 	(message Re-applying formula to field: %s (car eq))
 	(org-goto-line (nth 1 eq))
-	(org-table-goto-column (nth 2 eq))
-	(org-table-eval-formula nil (nth 3 eq) 'noalign 'nocst
-'nostore 'noanalysis))
+	(let ((column-target (nth 2 eq)))
+	  (when ( column-target 1000)
+	(user-error Formula column target too large))
+	  (let* ((column-count (progn (end-of-line)
+  (1- (org-table-current-column
+		 (create-new-column
+		  (and ( column-target column-count)
+		   (or (eq org-table-formula-create-columns t)
+			   (and
+			(eq org-table-formula-create-columns 'warn)
+			(progn
+			  (org-display-warning Out-of-bounds formula added columns)
+			  t))
+			   (and
+			(eq org-table-formula-create-columns 'prompt)
+			(yes-or-no-p Out-of-bounds formula. Add columns?))
+	(org-table-goto-column column-target nil create-new-column))
+
+	  (org-table-eval-formula nil (nth 3 eq) 'noalign 'nocst
+  'nostore 'noanalysis)))
 
   (org-goto-line thisline)
   (org-table-goto-column thiscol)
-- 
2.0.0



Re: [O] Adding new table rows/cols in a formula update

2014-10-01 Thread Dima Kogan
 Dima Kogan li...@dima.secretsauce.net writes:
 
  Suppose I have this .org file:
 
   |   |
   #+TBLFM: @1$2=5
 
  It's a 1x1 table with a formula. The formula sets a cell that's out of
  bounds in the table, so evaluating this formula results in an error.
  How set-in-stone is this behavior? I haven't dug too deeply into the
  code, but are there fundamental assumptions here? Would a patch that
  extends the table before applying such a formula be too naive in some
 way?

 Subhan Michael Tindall subh...@familycareinc.org writes:

 I would include a customization variable to control this behavior, defaulting 
 to whatever the current behavior is:
 IE:
 (setq org-calc-extend-file nil) default system behavior
 (setq org-calc-extend-file t) always silently extend rows
 (setq org-calc-extend-file warn) issue warning in message buffer that line 
 was extended
 (setq org-calc-extend-file prompt) prompt user y/n on whether or not to 
 extend column

OK. Patch attached.

From 3b6581c647cb87f0d3e8cee94ce2fb1fb122d3fd Mon Sep 17 00:00:00 2001
From: Dima Kogan d...@secretsauce.net
Date: Tue, 30 Sep 2014 22:36:21 -0700
Subject: [PATCH] Field formulas can now add columns as needed

The org-table-formula-make-new-cols customization controls whether and how this
is done
---
 lisp/org-table.el | 27 ++-
 1 file changed, 26 insertions(+), 1 deletion(-)

diff --git a/lisp/org-table.el b/lisp/org-table.el
index 7607ead..f2933ed 100644
--- a/lisp/org-table.el
+++ b/lisp/org-table.el
@@ -354,6 +354,16 @@ portability of tables.
 	  (const :tag Stick to hline nil)
 	  (const :tag Error on attempt to cross error)))
 
+(defcustom org-table-formula-make-new-cols nil
+  Non-nil means that evaluation of a field formula can add new
+columns if an out-of-bounds field is being set.
+  :group 'org-table-calculation
+  :type '(choice
+	  (const :tag Setting an out-of-bounds field generates an error (default) nil)
+	  (const :tag Setting an out-of-bounds field silently adds columns as needed t)
+	  (const :tag Setting an out-of-bounds field adds columns as needed, but issues a warning message warn)
+	  (const :tag When setting an out-of-bounds field, the user is prompted prompt)))
+
 (defgroup org-table-import-export nil
   Options concerning table import and export in Org-mode.
   :tag Org Table Import Export
@@ -3125,7 +3135,22 @@ known that the table will be realigned a little later anyway.
   (while (setq eq (pop eqlname1))
 	(message Re-applying formula to field: %s (car eq))
 	(org-goto-line (nth 1 eq))
-	(org-table-goto-column (nth 2 eq))
+	(let* ((column-target (nth 2 eq))
+	   (column-count (progn (end-of-line)
+(1- (org-table-current-column
+	   (create-new-column
+		(and ( column-target column-count)
+		 (or (eq org-table-formula-make-new-cols t)
+			 (and
+			  (eq org-table-formula-make-new-cols 'warn)
+			  (progn
+			(org-display-warning Out-of-bounds formula added columns)
+			t))
+			 (and
+			  (eq org-table-formula-make-new-cols 'prompt)
+			  (yes-or-no-p Out-of-bounds formula. Add columns?))
+	  (org-table-goto-column column-target nil create-new-column))
+
 	(org-table-eval-formula nil (nth 3 eq) 'noalign 'nocst
 'nostore 'noanalysis))
 
-- 
2.0.0



Re: [O] Adding new table rows/cols in a formula update

2014-10-01 Thread Dima Kogan
Hi. Thanks for replying.


Nicolas Goaziou m...@nicolasgoaziou.fr writes:

 Thanks for your patch. Some comments follow.

 From 3b6581c647cb87f0d3e8cee94ce2fb1fb122d3fd Mon Sep 17 00:00:00 2001
 From: Dima Kogan d...@secretsauce.net
 Date: Tue, 30 Sep 2014 22:36:21 -0700
 Subject: [PATCH] Field formulas can now add columns as needed

 The org-table-formula-make-new-cols customization controls whether and how 
 this
 is done

 Your commit message is missing information and TINYCHANGE at its end.
 See http://orgmode.org/worg/org-contribute.html#sec-5

OK. Added. By the way, I signed the copyright-assignment paperwork for
Emacs itself. Would I need to do this again for org-mode?



 +(defcustom org-table-formula-make-new-cols nil

 What about `org-table-formula-create-columns'?

Sure, why not.



 +  Non-nil means that evaluation of a field formula can add new
 +columns if an out-of-bounds field is being set.

 First line needs to be complete. E.g.,

   Non-nil means a field formula can create a new column.

 +  :group 'org-table-calculation
 +  :type '(choice
 +  (const :tag Setting an out-of-bounds field generates an error 
 (default) nil)
 +  (const :tag Setting an out-of-bounds field silently adds columns as 
 needed t)
 +  (const :tag Setting an out-of-bounds field adds columns as needed, 
 but issues a warning message warn)
 +  (const :tag When setting an out-of-bounds field, the user is
 prompted prompt)))

 You need to add :version and :package-version keywords.

OK. I'm not sure what those values should be, so please double-check.


  (defgroup org-table-import-export nil
Options concerning table import and export in Org-mode.
:tag Org Table Import Export
 @@ -3125,7 +3135,22 @@ known that the table will be realigned a little later 
 anyway.
(while (setq eq (pop eqlname1))
  (message Re-applying formula to field: %s (car eq))
  (org-goto-line (nth 1 eq))
 -(org-table-goto-column (nth 2 eq))
 +(let* ((column-target (nth 2 eq))
 +   (column-count (progn (end-of-line)
 +(1- (org-table-current-column
 +   (create-new-column
 +(and ( column-target column-count)

 Is this check really necessary? Doesn't `org-table-goto-column' already
 figures it out before creating a new column?

This is necessary if we want to be able to warn the user or to prompt
them ONLY if a new column has to be made. I.e. if we're not looking past
the bounds of the table the user should never be pestered. If the
customization variable is only t/nil then this can be dramatically
simplified, as you have observed.

By the way, is (org-display-warning) the preferred way to produce a
warning? It does into a new buffer, not into *Messages*.

Tweaked patch attached.

From 7b3ab8eeffb2047b966c624707766ec29a416583 Mon Sep 17 00:00:00 2001
From: Dima Kogan d...@secretsauce.net
Date: Tue, 30 Sep 2014 22:36:21 -0700
Subject: [PATCH] org-table.el: field formulas can now create columns as needed

(org-table-recalculate): use the new org-table-formula-make-new-cols
customization to control whether org creates new columns when a
formula explicitly targets them

TINYCHANGE
---
 lisp/org-table.el | 29 -
 1 file changed, 28 insertions(+), 1 deletion(-)

diff --git a/lisp/org-table.el b/lisp/org-table.el
index 7607ead..090bb75 100644
--- a/lisp/org-table.el
+++ b/lisp/org-table.el
@@ -354,6 +354,18 @@ portability of tables.
 	  (const :tag Stick to hline nil)
 	  (const :tag Error on attempt to cross error)))
 
+(defcustom org-table-formula-create-columns nil
+  Non-nil means that evaluation of a field formula can add new
+columns if an out-of-bounds field is being set.
+  :group 'org-table-calculation
+  :version 24.5
+  :package-version '(Org . 8.3)
+  :type '(choice
+	  (const :tag Setting an out-of-bounds field generates an error (default) nil)
+	  (const :tag Setting an out-of-bounds field silently adds columns as needed t)
+	  (const :tag Setting an out-of-bounds field adds columns as needed, but issues a warning message warn)
+	  (const :tag When setting an out-of-bounds field, the user is prompted prompt)))
+
 (defgroup org-table-import-export nil
   Options concerning table import and export in Org-mode.
   :tag Org Table Import Export
@@ -3125,7 +3137,22 @@ known that the table will be realigned a little later anyway.
   (while (setq eq (pop eqlname1))
 	(message Re-applying formula to field: %s (car eq))
 	(org-goto-line (nth 1 eq))
-	(org-table-goto-column (nth 2 eq))
+	(let* ((column-target (nth 2 eq))
+	   (column-count (progn (end-of-line)
+(1- (org-table-current-column
+	   (create-new-column
+		(and ( column-target column-count)
+		 (or (eq org-table-formula-create-columns t)
+			 (and
+			  (eq org-table-formula-create-columns 'warn)
+			  (progn
+			(org-display-warning Out-of-bounds formula added columns)
+			t))
+			 (and
+			  (eq org-table-formula-create

Re: [O] Adding new table rows/cols in a formula update

2014-09-30 Thread Dima Kogan
Dima Kogan li...@dima.secretsauce.net writes:

 Suppose I have this .org file:

  |   |
  #+TBLFM: @1$2=5

 It's a 1x1 table with a formula. The formula sets a cell that's out of
 bounds in the table, so evaluating this formula results in an error. How
 set-in-stone is this behavior? I haven't dug too deeply into the code,
 but are there fundamental assumptions here? Would a patch that extends
 the table before applying such a formula be too naive in some way?

Here's a tiny patch that adds the columns (not rows) as needed. Is this
reasonable?

From 93e9927dd49d100036853963e899c8b6af5325de Mon Sep 17 00:00:00 2001
From: Dima Kogan d...@secretsauce.net
Date: Tue, 30 Sep 2014 12:27:26 -0700
Subject: [PATCH] org-table: field formulas can now create new columns as
 needed

---
 lisp/org-table.el | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/lisp/org-table.el b/lisp/org-table.el
index 7607ead..31365ad 100644
--- a/lisp/org-table.el
+++ b/lisp/org-table.el
@@ -3125,7 +3125,7 @@ known that the table will be realigned a little later anyway.
   (while (setq eq (pop eqlname1))
 	(message Re-applying formula to field: %s (car eq))
 	(org-goto-line (nth 1 eq))
-	(org-table-goto-column (nth 2 eq))
+	(org-table-goto-column (nth 2 eq) nil 'force)
 	(org-table-eval-formula nil (nth 3 eq) 'noalign 'nocst
 'nostore 'noanalysis))
 
-- 
2.0.0



[O] Adding new table rows/cols in a formula update

2014-09-29 Thread Dima Kogan
Hi.

Suppose I have this .org file:

 |   |
 #+TBLFM: @1$2=5

It's a 1x1 table with a formula. The formula sets a cell that's out of
bounds in the table, so evaluating this formula results in an error. How
set-in-stone is this behavior? I haven't dug too deeply into the code,
but are there fundamental assumptions here? Would a patch that extends
the table before applying such a formula be too naive in some way?

Thanks!

dima