[O] Orgtbl-mode in latex: escaped braces and dollars, and other arbitrary transformations

2014-06-26 Thread Thibaut Verron 
Hello,

I'm forwarding this question asked on stackexchange: 
http://tex.stackexchange.com/questions/186605/with-orgtbl-how-to-ensure-
that-braces-and-dollars-are-not-escaped 

After some investigation, it seems that the behavior is hidden deep in the 
export routines, and I was wisely suggested to ask the question on this list 
instead.

I have given some tex-related details in the linked question, including some 
motivations and an example, the tl;dr is that in some conditions, the 
orgtbl-to-latex exporter will perform arbitrary escape of some characters in 
the cells, or other kind of transformations:

 $\text{test}$ 
is exported verbatim (OK).

But
 \pbox{test}
becomes
 \pbox\{test\}

 {test}
becomes
 \{test\}

 {$test$}
becomes
 \{\$test\$\}

And the exporter seems to be trying to be smart, because it will still 
ensure that the result is correct:

 {$\infty$}
becomes
 \{\$$\infty$\$\}

The weirdest of all might be this one:
 \pbox{Foo: \\${bar= (2^{3},1)}$, ${baz= (8^{4})}$}
becomes 
 \pbox\{Foo: \\${bar= (2^{3},1)}$, \$\{baz= (8$^{\text{4}}$)\}\$\}

(Note how the two mathematical expressions recieve different treatment, and 
the decision to insert \text around the exponent!)

The option `:no-export`, as expected, has no effect, since it only controls 
whether `#_^%` are escaped or not. 

Is this a known feature, or a bug? And is there a known workaround?

Thanks,

Thibaut Verron

Re: [O] Orgtbl-mode in latex: escaped braces and dollars, and other arbitrary transformations

2014-06-26 Thread Nicolas Goaziou 
Hello,

Thibaut Verron thibaut.ver...@gmail.com writes:

 I'm forwarding this question asked on stackexchange: 
 http://tex.stackexchange.com/questions/186605/with-orgtbl-how-to-ensure-
 that-braces-and-dollars-are-not-escaped 

 After some investigation, it seems that the behavior is hidden deep in the 
 export routines, and I was wisely suggested to ask the question on this list 
 instead.

 I have given some tex-related details in the linked question, including some 
 motivations and an example, the tl;dr is that in some conditions, the 
 orgtbl-to-latex exporter will perform arbitrary escape of some characters in 
 the cells, or other kind of transformations:

  $\text{test}$ 
 is exported verbatim (OK).

 But
  \pbox{test}
 becomes
  \pbox\{test\}

You should upgrade to Org 8.

  {test}
 becomes
  \{test\}

This is intended.

  {$test$}
 becomes
  \{\$test\$\}

This is to be expected as matching single dollar math snippets is
fragile. I suggest to use \(...\) instead: {\(test\)}

 And the exporter seems to be trying to be smart, because it will still 
 ensure that the result is correct:

  {$\infty$}
 becomes
  \{\$$\infty$\$\}

Ditto. Use \(...\). Or, better, {\infty} as \infty is an Org entity
which will properly translated into LaTeX code.

 The weirdest of all might be this one:
  \pbox{Foo: \\${bar= (2^{3},1)}$, ${baz= (8^{4})}$}
 becomes 
  \pbox\{Foo: \\${bar= (2^{3},1)}$, \$\{baz= (8$^{\text{4}}$)\}\$\}

You cannot write raw LaTeX macros with complicated arguments (e.g.,
containing braces) in an Org buffer. In this case, you have to tell the
exporter it is LaTeX code and don't expect it to find it out:

  @@latex:\pbox{Foo: \\${bar= (2^{3},1)}$, ${baz= (8^{4})}$}@@

This will be ignored in any exporter but latex (and beamer).

 The option `:no-export`, as expected, has no effect, since it only controls 
 whether `#_^%` are escaped or not. 

This option doesn't exist anymore in Org 8.

 Is this a known feature, or a bug? And is there a known workaround?

Actually, this is consistent if you understand the limitations of Org.
As a rule of thumb, avoid using $..$ constructs and macros with
convoluted arguments (but \hfill{} and \vspace{1cm} are fine) when
writing raw LaTeX in an Org buffer.

For anything more complicated, use @@latex:...@@
or #+BEGIN_LATEX...#+END_LATEX (inline and non-inline version).

Again, all this assumes you are using Org 8.

HTH,


Regards,

-- 
Nicolas Goaziou

Re: [O] Orgtbl-mode in latex: escaped braces and dollars, and other arbitrary transformations

2014-06-26 Thread Thibaut Verron 

 Now, are these limitations of Org really preventing it from exporting a
 string verbatim? That would seem like the most logical default in this
 situation, wouldn't it? (Disclaimer: I don't understand the limitations of
 Org, so these last questions may be ridiculous to someone who does)



Apparently not, the following quick attempt seems to be doing the job fine
enough:

  (defun tv/orgtbl-to-latex-verbatim (table params)
(flet ((org-export-string-as
(string backend optional b e)
string))
 (orgtbl-to-latex table params)))

However, it is extra dirty, and ignoring so many parameters in a function
is probably not safe. :-)
Is there really no similar mechanism built-in org? Should I polish this
function and submit a patch? Or am I running into a wall?

Thanks,

Thibaut Verron

Re: [O] Orgtbl-mode in latex: escaped braces and dollars, and other arbitrary transformations

2014-06-26 Thread Thibaut Verron 
2014-06-26 12:38 GMT+02:00 Nicolas Goaziou m...@nicolasgoaziou.fr:

 Hello,

 Thibaut Verron thibaut.ver...@gmail.com writes:

  I'm forwarding this question asked on stackexchange:
  http://tex.stackexchange.com/questions/186605/with-orgtbl-how-to-ensure-
  that-braces-and-dollars-are-not-escaped
 
  After some investigation, it seems that the behavior is hidden deep in
 the
  export routines, and I was wisely suggested to ask the question on this
 list
  instead.
 
  I have given some tex-related details in the linked question, including
 some
  motivations and an example, the tl;dr is that in some conditions, the
  orgtbl-to-latex exporter will perform arbitrary escape of some
 characters in
  the cells, or other kind of transformations:
 
   $\text{test}$
  is exported verbatim (OK).
 
  But
   \pbox{test}
  becomes
   \pbox\{test\}

 You should upgrade to Org 8.

   {test}
  becomes
   \{test\}

 This is intended.

   {$test$}
  becomes
   \{\$test\$\}

 This is to be expected as matching single dollar math snippets is
 fragile. I suggest to use \(...\) instead: {\(test\)}

  And the exporter seems to be trying to be smart, because it will still
  ensure that the result is correct:
 
   {$\infty$}
  becomes
   \{\$$\infty$\$\}

 Ditto. Use \(...\). Or, better, {\infty} as \infty is an Org entity
 which will properly translated into LaTeX code.

  The weirdest of all might be this one:
   \pbox{Foo: \\${bar= (2^{3},1)}$, ${baz= (8^{4})}$}
  becomes
   \pbox\{Foo: \\${bar= (2^{3},1)}$, \$\{baz= (8$^{\text{4}}$)\}\$\}

 You cannot write raw LaTeX macros with complicated arguments (e.g.,
 containing braces) in an Org buffer. In this case, you have to tell the
 exporter it is LaTeX code and don't expect it to find it out:

   @@latex:\pbox{Foo: \\${bar= (2^{3},1)}$, ${baz= (8^{4})}$}@@

 This will be ignored in any exporter but latex (and beamer).

  The option `:no-export`, as expected, has no effect, since it only
 controls
  whether `#_^%` are escaped or not.

 This option doesn't exist anymore in Org 8.

  Is this a known feature, or a bug? And is there a known workaround?

 Actually, this is consistent if you understand the limitations of Org.
 As a rule of thumb, avoid using $..$ constructs and macros with
 convoluted arguments (but \hfill{} and \vspace{1cm} are fine) when
 writing raw LaTeX in an Org buffer.

 For anything more complicated, use @@latex:...@@
 or #+BEGIN_LATEX...#+END_LATEX (inline and non-inline version).

 Again, all this assumes you are using Org 8.

 HTH,


 Regards,

 --
 Nicolas Goaziou


Thank you for your answer,

I forgot to repeat the information about my environment I gave on
stackexchange. In particular, this is Org 8.2.6.

Also, this is not an org buffer, but a latex buffer with the orgtbl minor
mode. Indeed I should have mentioned it in the body of the question, it's
easily missed in the title.

However, I understand that the underlying mechanism is the same as the one
used to export an org buffer to latex, and that it probably suffers from
the same limitations.
Now, are these limitations of Org really preventing it from exporting a
string verbatim? That would seem like the most logical default in this
situation, wouldn't it? (Disclaimer: I don't understand the limitations of
Org, so these last questions may be ridiculous to someone who does)

Thanks,

Thibaut Verron

PS. Indeed \(..\) does work for the dollars, thank you for the tip. I
should have pointed out that \bgroup ... \egroup works as a replacement for
outermost pairs of braces too. However, I do not have any solution for the
\pbox{...} thing. And I would prefer a more robust solution which would not
require me to change the way I write the tables, because otherwise, I'd
still risk facing new unexportable constructs at random times.

Re: [O] Orgtbl-mode in latex: escaped braces and dollars, and other arbitrary transformations

2014-06-26 Thread Nicolas Goaziou 
Thibaut Verron thibaut.ver...@gmail.com writes:

 Now, are these limitations of Org really preventing it from exporting a
 string verbatim? That would seem like the most logical default in this
 situation, wouldn't it?

I disagree in the general case. The most logical default for Org is to
treat contents as plain text and ensure that the export conforms to what
appears in the buffer. As a bonus, it can leave LaTeX code as-is when it
recognizes some, which depends on Org's understanding of LaTeX syntax
(hence the limitations I'm talking about).

Now, in the context of a LaTeX buffer using orgtbl minor mode, it could
make sense in some situations to treat cell contents verbatim. I don't
think it should be the default, but there could be an option for that.
Anyway, there's a solution, see below.

 Apparently not, the following quick attempt seems to be doing the job fine
 enough:

   (defun tv/orgtbl-to-latex-verbatim (table params)
 (flet ((org-export-string-as
 (string backend optional b e)
 string))
  (orgtbl-to-latex table params)))

 However, it is extra dirty, and ignoring so many parameters in a function
 is probably not safe. :-)

I think defining your own translator function is the way to go. For
example, the following (untested) could work:

  (defun my-orgtbl-to-latex-verbatim (table params)
(let* ((alignment (mapconcat (lambda (x) (if x r l))
   org-table-last-alignment ))
 (params2
  (list
   :tstart (concat \\begin{tabular}{ alignment })
   :tend \\end{tabular}
   :lstart  :lend   :sep   
   :efmt %s\\,(%s) :hline \\hline)))
  (orgtbl-to-generic table (org-combine-plists params2 params


Regards,

-- 
Nicolas Goaziou

Re: [O] Orgtbl-mode in latex: escaped braces and dollars, and other arbitrary transformations

2014-06-26 Thread Thibaut Verron 
2014-06-26 15:17 GMT+02:00 Nicolas Goaziou m...@nicolasgoaziou.fr:

 Thibaut Verron thibaut.ver...@gmail.com writes:

  Now, are these limitations of Org really preventing it from exporting
 a
  string verbatim? That would seem like the most logical default in this
  situation, wouldn't it?

 I disagree in the general case. The most logical default for Org is to
 treat contents as plain text and ensure that the export conforms to what
 appears in the buffer. As a bonus, it can leave LaTeX code as-is when it
 recognizes some, which depends on Org's understanding of LaTeX syntax
 (hence the limitations I'm talking about).


Of course, I did not mean it for org-mode buffers! ;)



 Now, in the context of a LaTeX buffer using orgtbl minor mode, it could
 make sense in some situations to treat cell contents verbatim. I don't
 think it should be the default, but there could be an option for that.
 Anyway, there's a solution, see below.

  Apparently not, the following quick attempt seems to be doing the job
 fine
  enough:
 
(defun tv/orgtbl-to-latex-verbatim (table params)
  (flet ((org-export-string-as
  (string backend optional b e)
  string))
   (orgtbl-to-latex table params)))
 
  However, it is extra dirty, and ignoring so many parameters in a function
  is probably not safe. :-)

 I think defining your own translator function is the way to go. For
 example, the following (untested) could work:

   (defun my-orgtbl-to-latex-verbatim (table params)
 (let* ((alignment (mapconcat (lambda (x) (if x r l))
org-table-last-alignment ))
  (params2
   (list
:tstart (concat \\begin{tabular}{ alignment })
:tend \\end{tabular}
:lstart  :lend   :sep   
:efmt %s\\,(%s) :hline \\hline)))
   (orgtbl-to-generic table (org-combine-plists params2 params


Indeed it does work.

But, unless I am mistaken, this is exactly the definition given here:
http://orgmode.org/manual/Translator-functions.html#Translator-functions
and so I was not wrong, this used to work as I expected.

I suspect that this change (regression?) will cause problems to a lot of
other users when they will upgrade their org to the current version.

Would changing the last lines of `orgtbl-to-latex` to something like this
work as a long-term solution?

(require 'ox-latex)
(let* ((*orgtbl-verbatim* (plist-get params :verbatim))
   (backend (if *orgtbl-verbatim* nil 'latex)))
  (orgtbl-to-generic table (org-combine-plists params2 params)
backend

Thanks for your time,

Thibaut verron

Re: [O] Orgtbl-mode in latex: escaped braces and dollars, and other arbitrary transformations

2014-06-26 Thread Nicolas Goaziou 
Thibaut Verron thibaut.ver...@gmail.com writes:

 Would changing the last lines of `orgtbl-to-latex` to something like this
 work as a long-term solution?

 (require 'ox-latex)
 (let* ((*orgtbl-verbatim* (plist-get params :verbatim))
(backend (if *orgtbl-verbatim* nil 'latex)))
   (orgtbl-to-generic table (org-combine-plists params2 params)
 backend

The check should happen in `orgtbl-to-generic', which is responsible for
translating cell contents.

Or, better, backend could become a parameter, e.g., :contents, and not
an optional argument anymore. Then one could override it with specific
params.

Also, docstrings and documentation should be updated accordingly.


Regards,

-- 
Nicolas Goaziou