Re: [O] [RFC] Org linting library

[Charles C. Berry]

On Sun, 19 Apr 2015, Nicolas Goaziou wrote:


Hello,

The following library implements linting for Org syntax. The sole public
function is `org-lint', which see.



Nice!

Comments below.


Internally, the library defines a new structure: `org-lint-checker',
with the following slots:



[snip]



Checks currently implemented are:

 - duplicates CUSTOM_ID properties
 - duplicate NAME values
 - duplicate targets
 - duplicate footnote definitions
 - orphaned affiliated keywords


This generates an orphaned keyword warning, but seems innocuous and is 
what babel renders.


#+NAME: abc
#+BEGIN_SRC emacs-lisp
nil
#+END_SRC

#+RESULTS: abc



 - obsolete affiliated keywords
 - missing language in src blocks
 - NAME values with a colon
 - wrong header arguments in src blocks


IFF the header args are in the #+BEGIN_SRC line.

#+HEADER lines and header-arg properties are not screened AFAICS.

[snip]

Chuck

Re: [O] Error when using org-ctrl-c-ctrl-c to add tags

[Vicente Vera]

Worked for me.

2015-04-19 13:05 GMT-03:00 Marc Ihm :
>
> Okay, this one only tries to fix, what seems broken ...
> Could someone please check ?
>
> best regards
> Marc
>
>
> diff --git a/lisp/org.el b/lisp/org.el
> index 24b3a69..8a00847 100755
> --- a/lisp/org.el
> +++ b/lisp/org.el
> @@ -10515,8 +10515,7 @@ from."
> completing-read-function)
>'completing-read)))
>(concat (car args))
> -  (if (and (consp (nth 1 args))
> -  (consp (car (nth 1 args
> +  (if (consp (car (nth 1 args)))
>   (mapcar 'car (nth 1 args))
> (nth 1 args))
>(cddr args)

Re: [O] [RFC] Org linting library

[Marco Wahl]

Hi!

Nicolas Goaziou  writes:

> The following library implements linting for Org syntax. The sole public
> function is `org-lint', which see.

Great, thanks a lot.  Already found some issues with some org-files.

> Results are displayed in a special "*Org Lint*" buffer with a dedicated
> major mode, derived from `tabulated-list-mode'. In addition to the usual
> key-bindings inherited from it, "C-j" displays problematic line reported
> under point and "RET" jumps to it.

Cool.

> Feedback welcome.

I think it's too much that org-lint tries to follow ssh links.  This
slowed down the org-linting considerably in my case.

#v+
2278 low   Link to non-existent local file 
"///ssh:root@127.0.0.1#:/etc/httpd/conf/httpd.conf"
#v-


My 2ct,  Marco
-- 
http://www.wahlzone.de
GPG: 0x49010A040A3AE6F2

Re: [O] Target and link text normalised to `orgtargetn'

[Nicolas Goaziou]

Rasmus  writes:

> That's what I meant.  Or rather a wrapper like org-latex--label.  A
> mapping like the one that was reverted for ox-latex only.  Or are there
> pitfalls in that approach?

It will not give you predictability either since you cannot guess "4" in
"sec:4". Also, it is dangerous since a user could use \label{sec:4} for
something different. Hence the `gensym'-like approach: we're pretty safe
with "orgheadline4".

What is the real benefit of "sec:4" over "orgheadline4"? Aesthetics?

> It does not IMO. I would rather not label sections manually.

I don't understand that part. Would you mind elaborating a bit?


Regards,

Re: [O] Target and link text normalised to `orgtargetn'

[Rasmus]

Nicolas Goaziou  writes:


>> I realize.  /I/ think we should do better.
>
> The only way to do better I can think of is to let "ox-latex.el"
> implements its own labelling scheme, without relying on
> `org-export-get-reference', which is not mandatory anyway.

That's what I meant.  Or rather a wrapper like org-latex--label.  A
mapping like the one that was reverted for ox-latex only.  Or are there
pitfalls in that approach?

> Anyway, I'm not sure we need to bother since
> `org-latex-prefer-user-labels' fills the gap.

It does not IMO.  I would rather not label sections manually.

—Rasmus

-- 
Enough with the bla bla!

Re: [O] [RFC] Org linting library

[Nicolas Goaziou]

Rasmus  writes:

> It looks like a great tool.  Especially given the fact that the org format
> still seems to be developing.

Indeed. We can use it as another channel for syntax deprecation.

> Anyway, some very premature notes:
>
> One venue that may worth pursuing is better error messages during export,
> when links do not exist.  E.g. after a link fails to resolve, an org-lint
> buffer is opened to ease tracking down the error.

I think the error message during export is as good as the one provided
by `org-lint', minus the line and the jump.

> Perhaps, org-lint should also check that INCLUDE keywords correctly
> resolves, e.g. whether #section exits in file.org:
>
> #+INCLUDE: "file.org::#section"

Good idea. I implemented it along with a check for SETUPFILE keywords.
Updated file attached.

> Perhaps that would induce nasty recursions, though...

`org-lint' only checks current document, it doesn't enter INCLUDE files,
so there's no danger here.

Thanks for the feedback.


Regards,

>From 1b182320599a435de7ab85f64ba984d1e5ef4502 Mon Sep 17 00:00:00 2001
From: Nicolas Goaziou 
Date: Wed, 15 Apr 2015 22:24:15 +0200
Subject: [PATCH] Implement Org linting

* lisp/org-lint.el: New file.
---
 lisp/org-lint.el | 748 +++
 1 file changed, 748 insertions(+)
 create mode 100644 lisp/org-lint.el

diff --git a/lisp/org-lint.el b/lisp/org-lint.el
new file mode 100644
index 000..a514b8b
--- /dev/null
+++ b/lisp/org-lint.el
@@ -0,0 +1,748 @@
+;;; org-lint.el --- Linting for Org documents-*- lexical-binding: t; -*-
+
+;; Copyright (C) 2015  Free Software Foundation
+
+;; Author: Nicolas Goaziou 
+;; Keywords: outlines, hypermedia, calendar, wp
+
+;; This program is free software; you can redistribute it and/or modify
+;; it under the terms of the GNU General Public License as published by
+;; the Free Software Foundation, either version 3 of the License, or
+;; (at your option) any later version.
+
+;; This program is distributed in the hope that it will be useful,
+;; but WITHOUT ANY WARRANTY; without even the implied warranty of
+;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+;; GNU General Public License for more details.
+
+;; You should have received a copy of the GNU General Public License
+;; along with this program.  If not, see .
+
+;;; Commentary:
+
+;; This library implements linting for Org syntax.  The sole public
+;; function is `org-lint', which see.
+
+;; Internally, the library defines a new structure:
+;; `org-lint-checker', with the following slots:
+
+;;   - NAME: Unique check identifier, as a symbol.  The check is done
+;; calling the function `org-lint-NAME' with one mandatory
+;; argument, the parse tree describing the current Org buffer.
+;; Such function calls are wrapped within a `save-excursion' and
+;; point is always at `point-min'.  Its return value has to be an
+;; alist (POSITION MESSAGE) when POSITION refer to the buffer
+;; position of the error, as an integer, and MESSAGE is a strings
+;; describing the error.
+
+;;   - DESCRIPTION: Summary about the check, as a string.
+
+;;   - CATEGORIES: Categories relative to the check, as a list of
+;; symbol.  They are used for filtering when calling `org-lint'.
+;; Checkers not explicitly associated to a category are collected
+;; in the `default' one.
+
+;;   - TRUST: The trust level one can have in the check.  It is either
+;; `low' or `high', depending on the heuristics implemented and
+;; the nature of the check.  This has an indicative value only and
+;; is displayed along reports.
+
+;; All checks have to be listed in `org-lint--checkers'.
+
+;; Results are displayed in a special "*Org Lint*" buffer with
+;; a dedicated major mode, derived from `tabulated-list-mode'.  In
+;; addition to the usual key-bindings inherited from it, "C-j"
+;; displays problematic line reported under point and "RET" jumps to
+;; it.
+
+;; Checks currently implemented are:
+
+;;   - duplicates CUSTOM_ID properties
+;;   - duplicate NAME values
+;;   - duplicate targets
+;;   - duplicate footnote definitions
+;;   - orphaned affiliated keywords
+;;   - obsolete affiliated keywords
+;;   - missing language in src blocks
+;;   - NAME values with a colon
+;;   - wrong header arguments in src blocks
+;;   - misuse of CATEGORY keyword
+;;   - "coderef" links with unknown destination
+;;   - "custom-id" links with unknown destination
+;;   - "fuzzy" links with unknown destination
+;;   - "id" links with unknown destination
+;;   - links to non-existent local files
+;;   - SETUPFILE keywords with non-existent file parameter
+;;   - INCLUDE keywords with wrong link parameter
+;;   - special properties in properties drawer
+;;   - obsolete syntax for PROPERTIES drawers
+;;   - missing definition for footnote references
+;;   - missing reference for footnote definitions
+;;   - non-footnote definitions in foo

Re: [O] Error when using org-ctrl-c-ctrl-c to add tags

[Marc Ihm]

Okay, this one only tries to fix, what seems broken ...
Could someone please check ?

best regards
Marc


diff --git a/lisp/org.el b/lisp/org.el
index 24b3a69..8a00847 100755
--- a/lisp/org.el
+++ b/lisp/org.el
@@ -10515,8 +10515,7 @@ from."
completing-read-function)
   'completing-read)))
   (concat (car args))
-  (if (and (consp (nth 1 args))
-  (consp (car (nth 1 args
+  (if (consp (car (nth 1 args)))
  (mapcar 'car (nth 1 args))
(nth 1 args))
   (cddr args)

Re: [O] Target and link text normalised to `orgtargetn'

[Nicolas Goaziou]

Rasmus  writes:

> I think it's very important that ox-latex outputs are also consumable.  I
> don't want to create another LyX.  Of course we are very far from this
> point.

[...]

> I realize.  /I/ think we should do better.

The only way to do better I can think of is to let "ox-latex.el"
implements its own labelling scheme, without relying on
`org-export-get-reference', which is not mandatory anyway.

Anyway, I'm not sure we need to bother since
`org-latex-prefer-user-labels' fills the gap.


Regards,

Re: [O] Error when using org-ctrl-c-ctrl-c to add tags

[Vicente Vera]

Hello Marc. Yes, upon further inspection it seems the problem is in
org-icompleting-read. This function is complaining that
org-tags-completion--function---one of its arguments---is not a list.

2015-04-19 11:36 GMT-03:00 Marc Ihm :
> Vicente Vera  writes:
>
>> Hi.
>> Don't think there are more details to provide. The error message
>> appears upon pressing C-c C-c (bound to the org-ctrl-c-ctrl-c command)
>> on a headline, doesn't matter which level.
>> It happened in 3 different Org files, one of which was created for testing.
>> Also tried with "emacs -q" but the error persisted.
>>
>> Strange message though, org-tags-completion-function is a function,
>> not a variable, so why is Emacs complaining that it isn't a list?
>> Probably related to commit acf7f47ecd20a48c05f97dc660b00d1850b57e10?
>
> I see this error too; the patch below fixes it for me (and corrects the
> nexting level of 'completing-read too ...)
>
> best regards
> Marc
>
>
> diff --git a/lisp/org.el b/lisp/org.el
> index 8a00847..f440382 100755
> --- a/lisp/org.el
> +++ b/lisp/org.el
> @@ -10512,10 +10512,11 @@ from."
>   (listp (second args)))
>  'org-iswitchb-completing-read)
> (t (or (and (boundp 'completing-read-function)
> -   completing-read-function)
> -  'completing-read)))
> +   completing-read-function))
> +  'completing-read))
>(concat (car args))
> -  (if (consp (car (nth 1 args)))
> +  (if (and (consp (nth 1 args))
> +  (consp (car (nth 1 args
>   (mapcar 'car (nth 1 args))
> (nth 1 args))
>(cddr args)

Re: [O] Target and link text normalised to `orgtargetn'

[Rasmus]

Hi,

Nicolas Goaziou  writes:

>> I hope you had a nice time "offline".
>
> Yes I had, thank you.

Cool!

>> I think the label naming is worse now, and I think it's important to
>> produce good quality labels akin to what you would expect from e.g.
>> AUCTeX.
>
> The goal is completely different. AUCTeX generates good quality labels
> because they are meant for user consumption.

I think it's very important that ox-latex outputs are also consumable.  I
don't want to create another LyX.  Of course we are very far from this
point.

> OTOH, `org-export-get-reference' creates impossible to guess labels, for
> internal use only, e.g., much like what `gensym' does.

I realize.  /I/ think we should do better.

Cheers,
Rasmus

-- 
Lasciate ogni speranza o voi che entrate: siete nella mani di'machellaio

Re: [O] Error when using org-ctrl-c-ctrl-c to add tags

[Rasmus]

Hi Marc (and Vincent),

Marc Ihm  writes:

> Vicente Vera  writes:
>
>> Don't think there are more details to provide. The error message
>> appears upon pressing C-c C-c (bound to the org-ctrl-c-ctrl-c command)
>> on a headline, doesn't matter which level.
>> It happened in 3 different Org files, one of which was created for testing.
>> Also tried with "emacs -q" but the error persisted.

Thanks for confirming.

>> Strange message though, org-tags-completion-function is a function,
>> not a variable, so why is Emacs complaining that it isn't a list?
>> Probably related to commit acf7f47ecd20a48c05f97dc660b00d1850b57e10?
>
> I see this error too; the patch below fixes it for me (and corrects the
> nexting level of 'completing-read too ...)

Thanks for confirming.  Do you (also) see it when starting Emacs with
"emacs -q", as Vincent?

Finally, can you repost post the patch via git am against master?

Thanks,
Rasmus

-- 
Summon the Mothership!

Re: [O] Bug: HTML export ignoring CUSTOM_ID properties

[Rasmus]

Hi,

tftor...@tftorrey.com (T.F. Torrey) writes:

> First, thank you for your incredible work on Org.  I hope you enjoyed
> your days off, but I have to admit that your announcement that you were
> taking the week off worried me.  I seem to remember we lost Carsten and
> Bastien soon after they took a week off.  When (if?) you finally get
> burned out and leave, all Org users will feel the loss.

This remark is uncalled for and very unfair.

> Nicolas Goaziou  writes:
>
>> "beta" is indeed misleading. I suggest to ignore it.
>>
>> As Rasmus pointed out, master is where development happens. Some changes
>> introduced here may break Org. If one such change makes Org unusable for
>> you, you can easily revert Org to an earlier, working, commit, without
>> fuss. Of course, we appreciate if you report the problem encountered
>> beforehand.
>
> Yes, changes on master can and do occasionally break Org, but they are
> *supposed* to work.  You wouldn't leave the spreadsheet functionality in
> an unusable state and just tell people to use 8.2.

To the best of my knowledge nobody makes a living from developing Org.
While everybody tries hard to make Org great, you cannot expect people to
put in work at the time *you* would prefer.  That being said, you are free
to hire somebody to make changes for you.  That way you could gain control
over the timing of particular fixes.

If you use git version of Org it is advisable to learn about "git
checkout".  In the following message I mention the range of changes that
also affect the change that has enraged you so greatly:

   http://permalink.gmane.org/gmane.emacs.orgmode/96890

> But yes, it should be a simple matter to revert the commit that caused
> the problem for me until the problem can be addressed.  That was the
> second thing I looked at.  However, the place where this change happened
> is not obvious in the git logs.  I still don't know where it came from.

See above.  In addition to the tips Nicolas provided you, it is good to
familiarize yourself with recent activity on the list.

> On the contrary, the slant of Org's development lately seems to be first
> to make sure users don't make any mistakes, and then to follow their
> instructions.

{{Citation needed}}

>>> As you said, they aren't your changes and it isn't your decision.
>>
>> I overlooked the problem in HTML and made a mistake. It happens, more
>> often than I would like. However, you are not required to be obnoxious
>> about it. It helps no one.
>
> Your mistakes are very rare, and your work is sincerely appreciated.  I
> think your comment about my response is out of context, and I'm not sure
> your final statement is true.  My polite comments were summarily
> dismissed, but now anyone who depended on CUSTOM_ID has been helped.

Nobody dismissed your "polite" comments:

However, in html custom_id serves as an important measure to
facilitate css customization, e.g. on a section level-basis.

Thus, I think it is a bug, unless there is a better way to allow
per-section css. I will look at this later unless somebody beets me to
it.



>> The problem should be fixed in 0449b785b4b58ec16e1aac126634de70eee519a4.
>> Thank you for reporting it.
>
> Thank you for your prompt action, but can I ask what you mean by
> "fixed"?  Have you decided to revert CUSTOM_ID to its previous
> functionality?  Are you still planning on changing its functionality
> and/or meaning?  Are you still planning on throwing warnings or errors in
> the event of duplicate or invalid CUSTOM_ID's?

Other commands that you may study further are "git log" and "git show".
I often use the following page as well:

   http://orgmode.org/cgit.cgi/org-mode.git

I hope this helps.

—Rasmus

-- 
Spil noget med Slayer!

Re: [O] Error when using org-ctrl-c-ctrl-c to add tags

[Marc Ihm]

Vicente Vera  writes:

> Hi.
> Don't think there are more details to provide. The error message
> appears upon pressing C-c C-c (bound to the org-ctrl-c-ctrl-c command)
> on a headline, doesn't matter which level.
> It happened in 3 different Org files, one of which was created for testing.
> Also tried with "emacs -q" but the error persisted.
>
> Strange message though, org-tags-completion-function is a function,
> not a variable, so why is Emacs complaining that it isn't a list?
> Probably related to commit acf7f47ecd20a48c05f97dc660b00d1850b57e10?

I see this error too; the patch below fixes it for me (and corrects the
nexting level of 'completing-read too ...)

best regards
Marc


diff --git a/lisp/org.el b/lisp/org.el
index 8a00847..f440382 100755
--- a/lisp/org.el
+++ b/lisp/org.el
@@ -10512,10 +10512,11 @@ from."
  (listp (second args)))
 'org-iswitchb-completing-read)
(t (or (and (boundp 'completing-read-function)
-   completing-read-function)
-  'completing-read)))
+   completing-read-function))
+  'completing-read))
   (concat (car args))
-  (if (consp (car (nth 1 args)))
+  (if (and (consp (nth 1 args))
+  (consp (car (nth 1 args
  (mapcar 'car (nth 1 args))
(nth 1 args))
   (cddr args)

Re: [O] Target and link text normalised to `orgtargetn'

[Nicolas Goaziou]

Rasmus  writes:

> I hope you had a nice time "offline".

Yes I had, thank you.

> I think the label naming is worse now, and I think it's important to
> produce good quality labels akin to what you would expect from e.g.
> AUCTeX.

The goal is completely different. AUCTeX generates good quality labels
because they are meant for user consumption.

OTOH, `org-export-get-reference' creates impossible to guess labels, for
internal use only, e.g., much like what `gensym' does.

> However, I have reverted cf7d64fgit per your request.

Thank you.

Regards,

Re: [O] [RFC] Org linting library

[Rasmus]

Hi,

Nicolas Goaziou  writes:

> Since it relies on lexical binding, `pcase' and `string-prefix-p', it
> cannot be added to Org 8.3, but can make it into Org 8.4, if deemed
> useful enough.

It looks like a great tool.  Especially given the fact that the org format
still seems to be developing.

Anyway, some very premature notes:

One venue that may worth pursuing is better error messages during export,
when links do not exist.  E.g. after a link fails to resolve, an org-lint
buffer is opened to ease tracking down the error.

Perhaps, org-lint should also check that INCLUDE keywords correctly
resolves, e.g. whether #section exits in file.org:

#+INCLUDE: "file.org::#section" 

Perhaps that would induce nasty recursions, though...

Thanks,
Rasmus

-- 
Enough with the bla bla!

Re: [O] Target and link text normalised to `orgtargetn'

[Suvayu Ali]

On Sun, Apr 19, 2015 at 03:56:22PM +0200, Nicolas Goaziou wrote:
> Suvayu Ali  writes:
> 
> > Ah, that was my report, that option did not work for me completely.
> > See: http://thread.gmane.org/gmane.emacs.orgmode/96887/focus=96888.
> >
> > To summarise quickly, with the above option set to t all
> > references/links behave as I expect, except the ones where the target is
> > a `bare target'.
> >
> > Does that mean there was indeed a bug?
> 
> Yes, Rasmus fixed it in f1548e11fe2972819bc48b88c6094b11150e5c8a. I just
> fixed Rasmus' fix wrt radio targets.

Ah, okk.  Up to speed now.

Cheers,

-- 
Suvayu

Open source is the future. It sets us free.

Re: [O] Target and link text normalised to `orgtargetn'

[Rasmus]

Hi Nicolas,

I hope you had a nice time "offline".

Nicolas Goaziou  writes:

> This is why I suggest to rever commit cf7d64f. It also introduces
> a non-alphanumeric character ":", even though `org-export-get-reference'
> uses only alphanumeric ones by portability concern (it is even specified
> in its docstring).

I think the label naming is worse now, and I think it's important to
produce good quality labels akin to what you would expect from
e.g. AUCTeX.  However, I have reverted cf7d64fgit per your request.

Perhaps a solution for high-quality labels in ox-latex can be introduced
later a la org-latex--label.

> Commit f1548e1 is different. I actually mostly agree with it (though
> I was confused at first with the docstring change, which needs to be
> fixed),

I guess the fix is 88108f652f0d4ddb4250cb89c2453df1fc9ee671.  Yes?

> but it breaks radio targets.

I agree, and I actually though I had left out that part when pushing.
Thanks for fixing.

—Rasmus

-- 
However beautiful the theory, you should occasionally look at the evidence

Re: [O] Target and link text normalised to `orgtargetn'

[Nicolas Goaziou]

Suvayu Ali  writes:

> Ah, that was my report, that option did not work for me completely.
> See: http://thread.gmane.org/gmane.emacs.orgmode/96887/focus=96888.
>
> To summarise quickly, with the above option set to t all
> references/links behave as I expect, except the ones where the target is
> a `bare target'.
>
> Does that mean there was indeed a bug?

Yes, Rasmus fixed it in f1548e11fe2972819bc48b88c6094b11150e5c8a. I just
fixed Rasmus' fix wrt radio targets.

Regards,

Re: [O] Target and link text normalised to `orgtargetn'

[Suvayu Ali]

Hi Nicolas,

On Sun, Apr 19, 2015 at 03:40:07PM +0200, Nicolas Goaziou wrote:
> Suvayu Ali  writes:
> 
> > Okay I see it now, in your last email I got a bit confused, but now I
> > follow.  With your solution I use both label and reference using LaTeX,
> > bypassing Org.  Of course this leaves me with links not working inside
> > Org.  I was eventually going to use filters to transform the ref
> > commands to the alternate ones from refstyle and varioref to get both
> > LaTeX and Org side working as I wanted.  Something like this:
> >
> >   <>
> >
> >   [[sec:interesting]]
> >
> > Then transform \ref{sec:interesting} → \secref{sec:interesting} with a
> > filter; that way I would have my cake and eat it too!  I see your point
> > though.
> 
> You can do that with a non-nil `org-latex-prefer-user-labels'.
> Otherwise, <> becomes \label{whatever}, defeating your
> filter.

Ah, that was my report, that option did not work for me completely.
See: http://thread.gmane.org/gmane.emacs.orgmode/96887/focus=96888.

To summarise quickly, with the above option set to t all
references/links behave as I expect, except the ones where the target is
a `bare target'.

Does that mean there was indeed a bug?

Cheers,

-- 
Suvayu

Open source is the future. It sets us free.

Re: [O] Target and link text normalised to `orgtargetn'

[Nicolas Goaziou]

Suvayu Ali  writes:

> Okay I see it now, in your last email I got a bit confused, but now I
> follow.  With your solution I use both label and reference using LaTeX,
> bypassing Org.  Of course this leaves me with links not working inside
> Org.  I was eventually going to use filters to transform the ref
> commands to the alternate ones from refstyle and varioref to get both
> LaTeX and Org side working as I wanted.  Something like this:
>
>   <>
>
>   [[sec:interesting]]
>
> Then transform \ref{sec:interesting} → \secref{sec:interesting} with a
> filter; that way I would have my cake and eat it too!  I see your point
> though.

You can do that with a non-nil `org-latex-prefer-user-labels'.
Otherwise, <> becomes \label{whatever}, defeating your
filter.


Regards,

[O] [RFC] Org linting library

[Nicolas Goaziou]

Hello,

The following library implements linting for Org syntax. The sole public
function is `org-lint', which see.

Internally, the library defines a new structure: `org-lint-checker',
with the following slots:

  - NAME: Unique check identifier, as a symbol. The check is done
calling the function `org-lint-NAME' with one mandatory argument,
the parse tree describing the current Org buffer. Such function
calls are wrapped within a `save-excursion' and point is always at
`point-min'. Its return value has to be an alist (POSITION MESSAGE)
when POSITION refer to the buffer position of the error, as an
integer, and MESSAGE is a strings describing the error.

  - DESCRIPTION: Summary about the check, as a string.

  - CATEGORIES: Categories relative to the check, as a list of symbol.
They are used for filtering when calling `org-lint'. Checkers not
explicitly associated to a category are collected in the `default'
one.

  - TRUST: The trust level one can have in the check. It is either `low'
or `high', depending on the heuristics implemented and the nature of
the check. This has an indicative value only and is displayed along
reports.

All checks have to be listed in `org-lint--checkers'.

Results are displayed in a special "*Org Lint*" buffer with a dedicated
major mode, derived from `tabulated-list-mode'. In addition to the usual
key-bindings inherited from it, "C-j" displays problematic line reported
under point and "RET" jumps to it.

Checks currently implemented are:

  - duplicates CUSTOM_ID properties
  - duplicate NAME values
  - duplicate targets
  - duplicate footnote definitions
  - orphaned affiliated keywords
  - obsolete affiliated keywords
  - missing language in src blocks
  - NAME values with a colon
  - wrong header arguments in src blocks
  - misuse of CATEGORY keyword
  - "coderef" links with unknown destination
  - "custom-id" links with unknown destination
  - "fuzzy" links with unknown destination
  - "id" links with unknown destination
  - links to non-existent local files
  - special properties in properties drawer
  - obsolete syntax for PROPERTIES drawers
  - missing definition for footnote references
  - missing reference for footnote definitions
  - non-footnote definitions in footnote section
  - probable invalid keywords
  - invalid blocks
  - probable incomplete drawers
  - obsolete QUOTE section

Since it relies on lexical binding, `pcase' and `string-prefix-p', it
cannot be added to Org 8.3, but can make it into Org 8.4, if deemed
useful enough.

Feedback welcome.


Regards,

-- 
Nicolas Goaziou0x80A93738
>From 8324f1422953caab5566950f989e95fd4325a0b0 Mon Sep 17 00:00:00 2001
From: Nicolas Goaziou 
Date: Wed, 15 Apr 2015 22:24:15 +0200
Subject: [PATCH] Implement Org linting

* lisp/org-lint.el: New file.
---
 lisp/org-lint.el | 681 +++
 1 file changed, 681 insertions(+)
 create mode 100644 lisp/org-lint.el

diff --git a/lisp/org-lint.el b/lisp/org-lint.el
new file mode 100644
index 000..009a503
--- /dev/null
+++ b/lisp/org-lint.el
@@ -0,0 +1,681 @@
+;;; org-lint.el --- Linting for Org documents-*- lexical-binding: t; -*-
+
+;; Copyright (C) 2015  Free Software Foundation
+
+;; Author: Nicolas Goaziou 
+;; Keywords: outlines, hypermedia, calendar, wp
+
+;; This program is free software; you can redistribute it and/or modify
+;; it under the terms of the GNU General Public License as published by
+;; the Free Software Foundation, either version 3 of the License, or
+;; (at your option) any later version.
+
+;; This program is distributed in the hope that it will be useful,
+;; but WITHOUT ANY WARRANTY; without even the implied warranty of
+;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+;; GNU General Public License for more details.
+
+;; You should have received a copy of the GNU General Public License
+;; along with this program.  If not, see .
+
+;;; Commentary:
+
+;; This library implements linting for Org syntax.  The sole public
+;; function is `org-lint', which see.
+
+;; Internally, the library defines a new structure:
+;; `org-lint-checker', with the following slots:
+
+;;   - NAME: Unique check identifier, as a symbol.  The check is done
+;; calling the function `org-lint-NAME' with one mandatory
+;; argument, the parse tree describing the current Org buffer.
+;; Such function calls are wrapped within a `save-excursion' and
+;; point is always at `point-min'.  Its return value has to be an
+;; alist (POSITION MESSAGE) when POSITION refer to the buffer
+;; position of the error, as an integer, and MESSAGE is a strings
+;; describing the error.
+
+;;   - DESCRIPTION: Summary about the check, as a string.
+
+;;   - CATEGORIES: Categories relative to the check, as a list of
+;; symbol.  They are used for filtering when calling `org-lint

Re: [O] Target and link text normalised to `orgtargetn'

[Suvayu Ali]

Hi Nicolas,

On Sun, Apr 19, 2015 at 02:57:09PM +0200, Nicolas Goaziou wrote:
> Suvayu Ali  writes:
> 
> > Sorry, I don't think I quite understand.  AFAIU, targets like
> > <> have to be unique even inside the Org buffer for links
> > like this to work: [[interesting][very cool]], regardless of export.  So
> > then why rename them during export?
> 
> There's no guarantee that, e.g., \label{interesting} is a valid syntax.
> Think about <<100%>> for LaTeX. Actually such links were broken for
> non-ASCII characters before the patch.
> 
> Note that this is transparent for the user: if <> becomes
> \label{foo}, [[interesting][very cool]] automatically becomes
> \hyperref[foo]{very cool}.
> 
> Your problem is different AFAIU. You are relying on the internal pattern
> of generated labels, i.e., you are eating Org's dog food. I advise
> against it.

Yes indeed.  I wanted to use the location information provided by
refstyle, and varioref.  By using \secref or \vpageref I would get
references like: section §3, on page 5, etc.

> If you need explicitly "\label{interesting}", ask Org to write it with,
> e.g., "@@latex:\label{foo}@@" or even better
> 
> #+MACRO: lbl @@latex:\label{$1}@@
> 
>   {{{lbl(foo)}}}
>

Okay I see it now, in your last email I got a bit confused, but now I
follow.  With your solution I use both label and reference using LaTeX,
bypassing Org.  Of course this leaves me with links not working inside
Org.  I was eventually going to use filters to transform the ref
commands to the alternate ones from refstyle and varioref to get both
LaTeX and Org side working as I wanted.  Something like this:

  <>

  [[sec:interesting]]

Then transform \ref{sec:interesting} → \secref{sec:interesting} with a
filter; that way I would have my cake and eat it too!  I see your point
though.

Thanks a lot for the comments.  :)

Cheers,

-- 
Suvayu

Open source is the future. It sets us free.

Re: [O] Target and link text normalised to `orgtargetn'

[Nicolas Goaziou]

Hello,

Suvayu Ali  writes:

> Sorry, I don't think I quite understand.  AFAIU, targets like
> <> have to be unique even inside the Org buffer for links
> like this to work: [[interesting][very cool]], regardless of export.  So
> then why rename them during export?

There's no guarantee that, e.g., \label{interesting} is a valid syntax.
Think about <<100%>> for LaTeX. Actually such links were broken for
non-ASCII characters before the patch.

Note that this is transparent for the user: if <> becomes
\label{foo}, [[interesting][very cool]] automatically becomes
\hyperref[foo]{very cool}.

Your problem is different AFAIU. You are relying on the internal pattern
of generated labels, i.e., you are eating Org's dog food. I advise
against it.

If you need explicitly "\label{interesting}", ask Org to write it with,
e.g., "@@latex:\label{foo}@@" or even better

#+MACRO: lbl @@latex:\label{$1}@@

  {{{lbl(foo)}}}

> How does the above solve the issue?  I was not having problems with
> getting to export \label{foo}, my problem was what do I put as `foo'
> since in the exported TeX file foo was different.
>
> Hopefully I made myself clear, or I'm not misunderstanding something.

Again <> are for [[target]], \label{...} are for
\whateverref{..}. IOW, you are mixing two different concepts. Since you
write "\secref{fig:1}" in your documents, you can also write
"\label{fig:1}" there.

Org can only ensures you that [[target]] always point to <>. No
more, no less.

There is another data point to consider: internal label scheme must not
follow a simple pattern, as a user might use the same and introduce
a label collision. E.g., if <> becomes par:foo, the following
line is problematic:

  <> \label{par:foo} \ref{par:foo} [[foo]]

This is why I suggest to rever commit cf7d64f. It also introduces
a non-alphanumeric character ":", even though `org-export-get-reference'
uses only alphanumeric ones by portability concern (it is even specified
in its docstring).

Commit f1548e1 is different. I actually mostly agree with it (though
I was confused at first with the docstring change, which needs to be
fixed), but it breaks radio targets. It doesn't need to be reverted.


Regards,

-- 
Nicolas Goaziou

Re: [O] Target and link text normalised to `orgtargetn'

[Suvayu Ali]

Hi Nicolas,

On Sat, Apr 18, 2015 at 11:53:01PM +0200, Nicolas Goaziou wrote:
> Hello,
> 
> Rasmus  writes:
> 
> > Suvayu Ali  writes:
> >
> >> Hi Rasmus,
> >>
> >> On Thu, Apr 16, 2015 at 12:57:46AM +0200, Rasmus wrote:
> >>> 
> >>> Could you try the attached patches and see if they solve your issues?
> >>
> >> Seems to work nicely.  I'll keep using them rest of the week.
> >
> > Pushed.
> >
> > I removed support for radio targets as I don't really see the point in
> > those supporting custom labeling.  They are transcoded as
> >
> > <<>
> > →
> > \hyperref{label}{target}
> >
> > It makes little sense to label be the same as target, though it probably
> > would not create problems.
> 
> I don't see the point of this change. The naming scheme is internal
> anyway. Users are not supposed to use it directly.

Sorry, I don't think I quite understand.  AFAIU, targets like
<> have to be unique even inside the Org buffer for links
like this to work: [[interesting][very cool]], regardless of export.  So
then why rename them during export?

> If you want \label{foo} in the output, you can write
> @@latex:\label{foo}@@.
> 
> Targets are for Org links and are expected to work across all back-ends,
> so the problem they solve is slightly different.

How does the above solve the issue?  I was not having problems with
getting to export \label{foo}, my problem was what do I put as `foo'
since in the exported TeX file foo was different.

Hopefully I made myself clear, or I'm not misunderstanding something.

Cheers,

-- 
Suvayu

Open source is the future. It sets us free.

Re: [O] Bug: HTML export ignoring CUSTOM_ID properties

[Nicolas Goaziou]

Hello,

tftor...@tftorrey.com (T.F. Torrey) writes:

> Yes, changes on master can and do occasionally break Org, but they are
> *supposed* to work.  You wouldn't leave the spreadsheet functionality in
> an unusable state and just tell people to use 8.2.

CUSTOM_ID was *supposed* to work after this change. Actually, it worked,
except in two use cases (custom CSS and linking from outside Org) in one
specific export back-end.

> But yes, it should be a simple matter to revert the commit that caused
> the problem for me until the problem can be addressed.  That was the
> second thing I looked at.  However, the place where this change happened
> is not obvious in the git logs.  I still don't know where it came
> from.

AFAICT, a single commit affected "ox-html" significantly lately:
459033265295723cbfb0fccb3577acbfdc9d0285. Anyway, you may also bisect to
find the problematic commit; doing so might also improve the bug report.

> I did see that most (maybe all) of your changes are accompanied by
> tests.  I'm not very familiar with the testing.  Are the tests
> restricted to merely checking if the code explodes?

You cannot test everything. Moreover, we do not write tests specific to
export back-ends (i.e., "ox.el" is extensively tested but not
"ox-latex.el" or, in this case "ox-html.el").

> They aren't lying because they don't claim to allow only valid ID's.
> They produce valid ID's on their own, but when a user calls for a
> specific ID (the {#clinton} construct in Markdown comes to mind), they
> just do what the user tells them to do.  Which is a good thing.

No, it isn't a good thing in all cases. 

In some circumstances, the user creates an ID possibly without knowing
about it, e.g. with targets and radio targets. How Org handles these
objects is an implementation detail, and shouldn't be thrown at the face
of users.

However, it happened recently to some user (see
). The culprit was
`org-export-solidify-link-text', a very wrong function. I didn't explain
the problem because of its internal nature, but it seems I should have.

For the record, `org-export-solidify-link-text' was just turning any
character non alphanumerical and not among "_.-:" into a hyphen. This
function was also applied to many things, including CUSTOM_ID. IOW,
"clinton" became "clinton", but also both "clintén" and "clintàn" became
"clint-n". So, basically CUSTOM_ID were already broken for anyone using
non ASCII characters.

Of course, if it had been only about CUSTOM_ID, the solution would have
been to simply remove the call to `org-export-solidify-link-text' and
let the user handle it. But there are also radio targets, and to
a lesser extent, targets, which are expected to be human-readable. E.g.,
if, for some reason, I need to write <<<éé>>> and <<<àà>>> in some
document, I certainly don't want them to both refer to "--" ID. Note
they could also be <<>> or <<>>, and could be
exported through HTML, LaTeX, etc. all with different expectations for
their IDs.

Since there is no reason to impose restrictions about them on the user
and the fact they are Org specific features, using an internal reference
is fine in this case. However, `org-export-solidify-link-text' is not
the answer, as it is not bijective. Therefore, I implemented
`org-export-get-reference', which relies on a very basic and portable
set of characters (alphanumeric ones) while still ensuring stability of
references. Of course, predictability is not achieved, but it wasn't
before either, excepted in the most simple cases.

Furthermore it isn't a problem in practice since users are not expected
to (and shouldn't) rely on these references externally. Here, CUSTOM_ID
in the context of HTML export is the exception, not the rule.

I hope this clarifies the purpose of the change.

> In my view, the purpose of tools such as Org that convert documents to
> HTML is to do what the user tells them to do, even if that means
> creating invalid HTML.  On many occasions in the past, and probably some
> in the present and the future, I have used conversion tools to produce
> technically invalid HTML as in intermediate format to be further
> processed by XSLT to a final product.  A tool that refused to produce
> invalid HTML would be no help at all.  In fact, I'm not aware of any
> tool that disallows that except maybe for some beginner level things.
>
> On the contrary, the slant of Org's development lately seems to be first
> to make sure users don't make any mistakes, and then to follow their
> instructions.

Again, this change isn't about protecting users, but fixing an incorrect
function, alas very much used across code base and affecting many users.

>> I overlooked the problem in HTML and made a mistake. It happens, more
>> often than I would like. However, you are not required to be obnoxious
>> about it. It helps no one.
>
> Your mistakes are very rare, and your work is sincerely appreciated.  I
> think your comment about my re