Re: attachment: link type export to HTML invalid attach dir

2020-02-27 Thread Nicolas Goaziou
Hello,

Gustav Wikström  writes:

> Surely custom code for file links. Haven't dug much deeper since the
> issue is solved by the attached patch.

Certainly, but that means `org-export-link-as-file' is not working at
some level. Since adding a hook is better, and the function was not used
anymore, I removed it: it is not necessary.

> I'll apply the patch then, unless already taken care of by you or
> Bastien! :)

Applied. Thank you.

Regards,

-- 
Nicolas Goaziou



RE: attachment: link type export to HTML invalid attach dir

2020-02-27 Thread Gustav Wikström
Hello,

> -Original Message-
> From: Nicolas Goaziou 
> Sent: den 26 februari 2020 16:23
> To: Gustav Wikström 
> Cc: Bastien ; emacs-orgmode@gnu.org
> Subject: Re: attachment: link type export to HTML invalid attach dir
> 
> Hello,
> 
> Gustav Wikström  writes:
> 
> > Expanding attachment-links in the buffer makes the link type no longer
> > be attachment.
> 
> What makes you think that? The link type is preserved during the
> substitution, according to the (concat "attachment" ...).

What I meant is that the content of the link, even though mentioned to be 
"attachment" cannot be reasoned about and worked with in the same way as an 
unexpanded attachment link. Thus, it's not the same type of link any longer. 
It's effectively a "file" link when expanded. But this discussion doesn't carry 
much meaning any longer I suppose, given the agreement to apply the patch.

> > I would prefer if we explicitly set the link type to
> > files here instead. Storing intermediate state in an attachment link
> > types makes less sense imo.
> >
> > One issue with the current way it's done is that images are treated
> > differently between attachment links and file links. For HTML exports,
> > file links are wrapped in a div with class figure where expanded
> > attachment links are not.
> 
> It would be nice to investigate why this happens.

Surely custom code for file links. Haven't dug much deeper since the issue is 
solved by the attached patch.

> > Letting org-attach-expand-links do the full
> > transform to file links would solve that issue. That also
> > means :export is not needed for org-link-set-parameters.
> >
> > Patch attached if you agree to this.
> 
> Of course! This is what I suggested, like, 6 weeks ago (although, at
> this time, I was thinking about having the substitution in "ox.el").

Well yes... I had other ideas at that point and would rather see attachment 
links as something of its own. But we're not in that place now, and the 
decision was to treat link types that can be expanded into file links just like 
that. You've kindly fixed the code so Org mode can do that now, something I'm 
sure I couldn't have done myself anyhow. So let's do it as you suggested from 
the beginning. I'll apply the patch then, unless already taken care of by you 
or Bastien! :)

Kindly
Gustav


Re: attachment: link type export to HTML invalid attach dir

2020-02-26 Thread Nicolas Goaziou
Hello,

Gustav Wikström  writes:

> Expanding attachment-links in the buffer makes the link type no longer
> be attachment.

What makes you think that? The link type is preserved during the
substitution, according to the (concat "attachment" ...).

> I would prefer if we explicitly set the link type to
> files here instead. Storing intermediate state in an attachment link
> types makes less sense imo.
>
> One issue with the current way it's done is that images are treated
> differently between attachment links and file links. For HTML exports,
> file links are wrapped in a div with class figure where expanded
> attachment links are not.

It would be nice to investigate why this happens.

> Letting org-attach-expand-links do the full
> transform to file links would solve that issue. That also
> means :export is not needed for org-link-set-parameters.
>
> Patch attached if you agree to this.

Of course! This is what I suggested, like, 6 weeks ago (although, at
this time, I was thinking about having the substitution in "ox.el").

Regards,

-- 
Nicolas Goaziou



RE: attachment: link type export to HTML invalid attach dir

2020-02-25 Thread Gustav Wikström
Hi Bastien and Nicolas,

> -Original Message-
> From: Bastien 
> Sent: den 22 februari 2020 14:32
> To: Nicolas Goaziou 
> Cc: Gustav Wikström ; emacs-orgmode@gnu.org
> Subject: Re: attachment: link type export to HTML invalid attach dir
> 
> [...]
>
> Nicolas Goaziou  writes:
> 
> > I'd like to hear from Gustav, too, since this affects the changes he
> > introduced. If you are in a hurry, we can also apply it. Up to you.
> 
> I just applied it - Gustav let us know if it fits your needs and if it
> does not introduce regressions.

It does work. Thanks. One issue though, with org-attach-expand-links:

+(defun org-attach-expand-links (_)
+  "Expand links in current buffer.
+It is meant to be added to `org-export-before-parsing-hook'."
+  (save-excursion
+(while (re-search-forward "attachment:" nil t)
+  (let ((link (org-element-context)))
+   (when (and (eq 'link (org-element-type link))
+  (string-equal "attachment"
+(org-element-property :type link)))
+ (let* ((description (and (org-element-property :contents-begin link)
+  (buffer-substring-no-properties
+   (org-element-property :contents-begin link)
+   (org-element-property :contents-end link
+(file (org-element-property :path link))
+(new-link (org-link-make-string
+   (concat "attachment:" (org-attach-expand file))
+   description)))
+   (goto-char (org-element-property :end link))
+   (skip-chars-backward " \t")
+   (delete-region (org-element-property :begin link) (point))

Expanding attachment-links in the buffer makes the link type no longer be 
attachment. I would prefer if we explicitly set the link type to files here 
instead. Storing intermediate state in an attachment link types makes less 
sense imo.

One issue with the current way it's done is that images are treated differently 
between attachment links and file links. For HTML exports, file links are 
wrapped in a div with class figure where expanded attachment links are not. 
Letting org-attach-expand-links do the full transform to file links would solve 
that issue. That also means :export is not needed for org-link-set-parameters.

Patch attached if you agree to this.

> >> Do you want to do this for 9.4 or can it be done later on?
> >
> > This can be done later on. I don't have time to work on this. Of
> > course, if someone wants to do it, that would be great, too.
> 
> Okay, thanks.  Noted for Org > 9.4.
> 
> Best,
> 
> --
>  Bastien


org-attach.patch
Description: org-attach.patch


Re: attachment: link type export to HTML invalid attach dir

2020-02-22 Thread Bastien
Hi,

Thanks for your answer.

Nicolas Goaziou  writes:

> I'd like to hear from Gustav, too, since this affects the changes he
> introduced. If you are in a hurry, we can also apply it. Up to you.

I just applied it - Gustav let us know if it fits your needs and if it
does not introduce regressions.

>> Do you want to do this for 9.4 or can it be done later on?
>
> This can be done later on. I don't have time to work on this. Of course,
> if someone wants to do it, that would be great, too.

Okay, thanks.  Noted for Org > 9.4.

Best,

-- 
 Bastien



Re: attachment: link type export to HTML invalid attach dir

2020-02-22 Thread Nicolas Goaziou
Hello,

Bastien  writes:

> Thanks a lot -- feel free to apply it when you think it's okay.

I'd like to hear from Gustav, too, since this affects the changes he
introduced. If you are in a hurry, we can also apply it. Up to you.

> Also, perhaps can you or Gustav suggest a way to test the new feature
> so that we can test this new code?

I don't use the feature. I just tried to avoid introducing regressions.

It is related to how attachments are opened, and exported. I guess
testing should focus on this, in particular export of attached images,
i.e., attachements with an image extension.

> Do you want to do this for 9.4 or can it be done later on?

This can be done later on. I don't have time to work on this. Of course,
if someone wants to do it, that would be great, too.

Regards,

-- 
Nicolas Goaziou



Re: attachment: link type export to HTML invalid attach dir

2020-02-20 Thread Bastien
Hi Nicolas,

Nicolas Goaziou  writes:

> This second patch removes most of the "attachment" leakage in the code
> base (mainly "org-element.el" and "ox-*.el"). The features should be the
> same as before. Let me know if there's anything wrong.

Thanks a lot -- feel free to apply it when you think it's okay.

Also, perhaps can you or Gustav suggest a way to test the new feature
so that we can test this new code?

> The only thing left is to refactor the ability to display "attachment"
> links as inline images. For now, this is hard-coded in "org.el", but
> this should not be the case.
>
> As a third phase, I suggest to add a new parameter in
> `org-link-parameters', for example :inlineable. This parameter would be
> either a boolean, or a function transforming the path. For example,
> "file" links should set it to t and "attachment" links to
> `org-attach-expand'. Then `org-display-inline-images' should collect
> link types with this flag, apply the transformation, and display images
> accordingly.

Do you want to do this for 9.4 or can it be done later on?

Thanks,

-- 
 Bastien



Re: attachment: link type export to HTML invalid attach dir

2020-02-20 Thread Nicolas Goaziou
Following up,

Nicolas Goaziou  writes:

> This is the first part of the suggested changes. These do not touch
> attachment modifications, but rather improve the tooling in "ol.el".

This second patch removes most of the "attachment" leakage in the code
base (mainly "org-element.el" and "ox-*.el"). The features should be the
same as before. Let me know if there's anything wrong.

The only thing left is to refactor the ability to display "attachment"
links as inline images. For now, this is hard-coded in "org.el", but
this should not be the case.

As a third phase, I suggest to add a new parameter in
`org-link-parameters', for example :inlineable. This parameter would be
either a boolean, or a function transforming the path. For example,
"file" links should set it to t and "attachment" links to
`org-attach-expand'. Then `org-display-inline-images' should collect
link types with this flag, apply the transformation, and display images
accordingly.

Regards,

-- 
Nicolas Goaziou
>From e5ccffcf617f3d04d97840873c0b16913eb65369 Mon Sep 17 00:00:00 2001
From: Nicolas Goaziou 
Date: Thu, 20 Feb 2020 09:29:21 +0100
Subject: [PATCH] Do not leak "attachment" links

* lisp/ol.el (org-link-open): Remove "attachment" for special cases.
* lisp/org-attach.el (org-attach-expand-links):
(org-attach-follow): New functions.
(org-attach-link-expand): Remove function.
* lisp/org-element.el (org-element-link-parser):
* lisp/ox-ascii.el (org-ascii-link):
* lisp/ox-html.el (org-html-link):
* lisp/ox-latex.el (org-latex--inline-image):
(org-latex-link):
* lisp/ox-man.el (org-man-link):
* lisp/ox-md.el (org-md-link):
* lisp/ox-odt.el (org-odt-inline-image-rules):
(org-odt-link):
* lisp/ox-texinfo.el (org-texinfo-inline-image-rules):
(org-texinfo-link): Remove "attachment" from special cases.
---
 lisp/ol.el  |  5 +
 lisp/org-attach.el  | 45 +++--
 lisp/org-element.el | 10 +-
 lisp/ox-ascii.el| 23 +--
 lisp/ox-html.el |  6 +-
 lisp/ox-latex.el| 23 ---
 lisp/ox-man.el  | 17 ++---
 lisp/ox-md.el   | 10 ++
 lisp/ox-odt.el  | 11 ++-
 lisp/ox-texinfo.el  | 10 +-
 10 files changed, 66 insertions(+), 94 deletions(-)

diff --git a/lisp/ol.el b/lisp/ol.el
index 76454d2db..e9bed3972 100644
--- a/lisp/ol.el
+++ b/lisp/ol.el
@@ -75,7 +75,6 @@
 (declare-function org-src-source-type "org-src" ())
 (declare-function org-time-stamp-format "org" ( long inactive))
 (declare-function outline-next-heading "outline" ())
-(declare-function org-attach-link-expand "org-attach" (link  buffer-or-name))
 
 
 ;;; Customization
@@ -1027,9 +1026,7 @@ for internal and \"file\" links, or stored as a parameter in
 (pcase type
   ;; Opening a "file" link requires special treatment since we
   ;; first need to integrate search option, if any.
-  ((or "file" "attachment")
-   (when (string= type "attachment")
-	 (setq path (org-attach-link-expand link)))
+  ("file"
(let* ((option (org-element-property :search-option link))
 	  (path (if option (concat path "::" option) path)))
 	 (org-link-open-as-file path
diff --git a/lisp/org-attach.el b/lisp/org-attach.el
index d073291a2..97a7236e4 100644
--- a/lisp/org-attach.el
+++ b/lisp/org-attach.el
@@ -41,6 +41,8 @@
 
 (declare-function dired-dwim-target-directory "dired-aux")
 (declare-function org-element-property "org-element" (property element))
+(declare-function org-element-type "org-element" (element))
+(declare-function org-export-link-as-file "org-export" (path description backend info))
 
 (defgroup org-attach nil
   "Options concerning attachments in Org mode."
@@ -646,22 +648,36 @@ See `org-attach-open'."
 Basically, this adds the path to the attachment directory."
   (expand-file-name file (org-attach-dir)))
 
-(defun org-attach-link-expand (link  buffer-or-name)
-  "Return the full path to the attachment in the LINK element.
-Takes LINK which is a link element, as defined by
-`org-element-link-parser'.  If LINK `:type' is attachment the
-full path to the attachment is expanded and returned.  Otherwise,
-return nil.  If BUFFER-OR-NAME is specified, LINK is expanded in
-that buffer, otherwise current buffer is assumed."
-  (let ((type (org-element-property :type link))
-	(file (org-element-property :path link))
-	(pos (org-element-property :begin link)))
-(when (string= type "attachment")
-  (with-current-buffer (or buffer-or-name (current-buffer))
-	(goto-char pos)
-	(org-attach-expand file)
+(defun org-attach-expand-links (_)
+  "Expand links in current buffer.
+It is meant to be added to `org-export-before-parsing-hook'."
+  (save-excursion
+(while (re-search-forward "attachment:" nil t)
+  (let ((link (org-element-context)))
+	(when (and (eq 'link (org-element-type link))
+		   (string-equal "attachment"
+ (org-element-property :type link)))
+	  (let* ((description (and 

Re: attachment: link type export to HTML invalid attach dir

2020-02-17 Thread Bastien
Hi Nicolas,

Nicolas Goaziou  writes:

> Thinking about it, we can stick to a single solution, e.g., the
> `condition-case' workaround. Therefore we do not need to introduce a new
> keyword. However, all "ol-" libraries need to be updated.

I think the release of 9.4 is an important one, so it's probably a
good time to advertize changes that may break existing ol-* libraries
that we are not aware of -- provided we clearly explain what needs to
be changed, of course.

> I'll do that instead.

Thanks,

-- 
 Bastien



Re: attachment: link type export to HTML invalid attach dir

2020-02-17 Thread Nicolas Goaziou
Hello,

Bastien  writes:

> I'm not sure why the distinction between :open and :follow is needed
> in the context of the discussion for exporting attachments: is it
> because attachment need to be "open", not just followed?

The motivation is that the new behaviour for `:follow' is not compatible
with the previous one, since the signature changed. So, we have two
options: either we write a workaround, as is done for :export (i.e.,
`condition-case' with `wrong-number-of-arguments'), or we add new
keywords so old code still works during transition.

Thinking about it, we can stick to a single solution, e.g., the
`condition-case' workaround. Therefore we do not need to introduce a new
keyword. However, all "ol-" libraries need to be updated.

I'll do that instead.

Regards,

-- 
Nicolas Goaziou



Re: attachment: link type export to HTML invalid attach dir

2020-02-17 Thread Bastien
Nicolas Goaziou  writes:

>> FWIW, I'm for delaying the removal for 9.5, not 9.4.
>
> Sure, I added it back.

Thanks!

-- 
 Bastien



Re: attachment: link type export to HTML invalid attach dir

2020-02-17 Thread Nicolas Goaziou
Hello,

Bastien  writes:

> FWIW, I'm for delaying the removal for 9.5, not 9.4.

Sure, I added it back.

Regards,

-- 
Nicolas Goaziou



Re: attachment: link type export to HTML invalid attach dir

2020-02-16 Thread Bastien
Hi Nicolas,

Nicolas Goaziou  writes:

> That is subconscious, actually. I considered removing them,
> then hesitated, and it all ended up in an intermediate state. Removing
> it properly would require to remove :application property in the parser,
> too. At some point, this functionality needs to go, but it is not
> necessarily now.

FWIW, I'm for delaying the removal for 9.5, not 9.4.

-- 
 Bastien



Re: attachment: link type export to HTML invalid attach dir

2020-02-16 Thread Bastien
Hi Nicolas,

Nicolas Goaziou  writes:

> This is the first part of the suggested changes. These do not touch
> attachment modifications, but rather improve the tooling in "ol.el".

Thanks for working on this--and Kyle for the careful review.

> `org-link-parameters' accepts a new property, :open, which is
> like :follow, but function installed there is called with a universal
> prefix argument.

I'm not sure why the distinction between :open and :follow is needed
in the context of the discussion for exporting attachments: is it
because attachment need to be "open", not just followed?

> Also, :export function is called with a fourth argument, the export info
> channel. I updated the export back-ends to handle the new signature, and
> provided a compatibility layers for link libraries in the wild, which
> may still use three arguments.
>
> There are two new tools: `org-link-open-as-file' and
> `org-export-link-as-file'. They can be used as helper functions for,
> respectively, :open and :export functions.

> WDYT?

LGTM, feel free to go ahead when you want.

Thanks,

-- 
 Bastien



Re: attachment: link type export to HTML invalid attach dir

2020-02-16 Thread Nicolas Goaziou
Hello,

Kyle Meyer  writes:

> It looks like a good direction to go.

Thank you for the feedback!

I fixed the various typos you mentioned.

> As far as I can see (by looking at the diff and quickly testing), the
> file+{sys,emacs} functionality is being removed.  That's probably
> okay/intended, given that it was marked as deprecated in 9.0.  But
> perhaps it's worth mentioning in ORG-NEWS that this functionality has
> now been removed entirely.

Good catch! That is subconscious, actually. I considered removing them,
then hesitated, and it all ended up in an intermediate state. Removing
it properly would require to remove :application property in the parser,
too. At some point, this functionality needs to go, but it is not
necessarily now. I can add it back if needed. Otherwise, I will remove
it completely in a subsequent patch.

Removing this feature may not break existing documents, however. Indeed,
with the suggested tooling, we may simply make "file+sys" and
"file+emacs" aliases for "file" in "org-compat.el". IOW, they would do
nothing more than regular file links, except maybe send an obnoxious
message to the user (linter already does this).

I didn't write the required ORG-NEWS entry because we are not settled on
the final design. I agree all the changes discussed here have to be
mentioned in that file.

Regards,

-- 
Nicolas Goaziou



Re: attachment: link type export to HTML invalid attach dir

2020-02-15 Thread Kyle Meyer
Nicolas Goaziou  writes:

> In a nutshell:
>
> `org-link-parameters' accepts a new property, :open, which is
> like :follow, but function installed there is called with a universal
> prefix argument. It could replace :follow altogether, but for
> compatibility reasons, they both live side by side for the moment.
>
> Also, :export function is called with a fourth argument, the export info
> channel. I updated the export back-ends to handle the new signature, and
> provided a compatibility layers for link libraries in the wild, which
> may still use three arguments.
>
> There are two new tools: `org-link-open-as-file' and
> `org-export-link-as-file'. They can be used as helper functions for,
> respectively, :open and :export functions.
>
> WDYT?

It looks like a good direction to go.

> Subject: [PATCH 1/2] Extend export tooling in link parameters
[...]
>  (defcustom org-link-parameters nil
> -  "An alist of properties that defines all the links in Org mode.
> +  "Alist of properties that defines all the links in Org mode.
> +
[...]
> +`:export'
> +
> +  Function that accepts three mandatory arguments :

nit: extra space before colon

> +  Optionally, it may accept the export communication channel as
> +  a fourth argument, in case you need using Org Export tooling,

s/need using/need to use/ ?

> Subject: [PATCH 2/2] ol: Extend open tooling in link parameters
[...]
> -  Function that takes the link path (a string) as an argument and
> -  \"opens\" the link.
> +  Function used to follow the link, when `org-open-at-point' is
> +  command runs on it.

Delete "is"?  (Also, assuming the "is" is dropped, to my ears it'd read
better with a "the" in front of org-open-at-point.)

> [ @@ -990,37 +998,14 @@ (defun org-link-open (link  arg) ]
> -  (if (string-match "[*?{]" (file-name-nondirectory path))
> -   (dired path)
> - ;; Look into `org-link-parameters' in order to find
> - ;; a DEDICATED-FUNCTION to open file.  The function will be
> - ;; applied on raw link instead of parsed link due to the
> - ;; limitation in `org-add-link-type' ("open" function called
> - ;; with a single argument).  If no such function is found,
> - ;; fallback to `org-open-file'.
> - (let* ((option (org-element-property :search-option link))
> -(app (org-element-property :application link))
> -(dedicated-function
> - (org-link-get-parameter (if app (concat type "+" app) type)
> - :follow)))

As far as I can see (by looking at the diff and quickly testing), the
file+{sys,emacs} functionality is being removed.  That's probably
okay/intended, given that it was marked as deprecated in 9.0.  But
perhaps it's worth mentioning in ORG-NEWS that this functionality has
now been removed entirely.

> +(defun org-link-open-as-file (path arg)
> +  "Pretend PATH is a file name and open it.
> +
> +According to \"file\"-link syntax, PATH may include addition

s/addition/additional/

> +search options, separated from the file name with \"::\".
> +
> +This function is meant to be used as a possible tool for `:open'
> +property in `org-link-parameters.'"

nit: I think the period should go after the quote (regardless of the
strange preference in American English to put commas and periods within
quotes).



Re: attachment: link type export to HTML invalid attach dir

2020-02-15 Thread Nicolas Goaziou
Hello,

Bastien  writes:

>>> Now, I'm not sure who wants to try implementing Nicolas suggestion to
>>> let ox.el expand attachments into file links: since you both know the
>>> issue better than I do, I suggest one of you can try?
>>
>> I will try to propose a patch by tomorrow evening.
>
> Thanks!
>
> If things are okay, I'll then have some time on sunday to prepare for
> the release on monday.

This is the first part of the suggested changes. These do not touch
attachment modifications, but rather improve the tooling in "ol.el".

In a nutshell:

`org-link-parameters' accepts a new property, :open, which is
like :follow, but function installed there is called with a universal
prefix argument. It could replace :follow altogether, but for
compatibility reasons, they both live side by side for the moment.

Also, :export function is called with a fourth argument, the export info
channel. I updated the export back-ends to handle the new signature, and
provided a compatibility layers for link libraries in the wild, which
may still use three arguments.

There are two new tools: `org-link-open-as-file' and
`org-export-link-as-file'. They can be used as helper functions for,
respectively, :open and :export functions.

WDYT?

Regards,

-- 
Nicolas Goaziou
>From 59172eb6c5edd5e74a6fd248e975065437bf5072 Mon Sep 17 00:00:00 2001
From: Nicolas Goaziou 
Date: Fri, 14 Feb 2020 10:00:15 +0100
Subject: [PATCH 1/2] Extend export tooling in link parameters

* lisp/ol.el (org-link-parameters): Allow a fourth "info" argument for
`:export' property.  Expound docstring.
* lisp/ox.el (org-export-custom-protocol-maybe): Accept a fourth
optional argument.
* lisp/ox-ascii.el (org-ascii--describe-links):
(org-ascii-link):
* lisp/ox-beamer.el (org-beamer-link):
* lisp/ox-html.el (org-html-link):
* lisp/ox-latex.el (org-latex-link):
* lisp/ox-man.el (org-man-link):
* lisp/ox-md.el (org-md-link):
* lisp/ox-odt.el (org-odt-link):
* lisp/ox-org.el (org-org-link):
* lisp/ox-texinfo.el (org-texinfo-link):
* contrib/lisp/ox-groff.el (org-groff-link): Provide expected fourth
argument.
* lisp/ox.el (org-export-link-as-file): New function.
* lisp/ol.el (org-link-parameters): Add reference to new function in docstring.
* testing/lisp/test-ox.el (test-org-export/link-as-file): Add tests.
(test-org-export/custom-protocol-maybe): Update tests.
---
 contrib/lisp/ox-groff.el |  2 +-
 lisp/ol.el   | 99 ++--
 lisp/ox-ascii.el |  4 +-
 lisp/ox-beamer.el|  2 +-
 lisp/ox-html.el  |  2 +-
 lisp/ox-latex.el |  2 +-
 lisp/ox-man.el   |  4 +-
 lisp/ox-md.el|  2 +-
 lisp/ox-odt.el   |  2 +-
 lisp/ox-org.el   |  4 +-
 lisp/ox-texinfo.el   |  2 +-
 lisp/ox.el   | 37 +++
 testing/lisp/test-ox.el  | 32 +++--
 13 files changed, 146 insertions(+), 48 deletions(-)

diff --git a/contrib/lisp/ox-groff.el b/contrib/lisp/ox-groff.el
index 9f7d3f11f..9f0a32432 100644
--- a/contrib/lisp/ox-groff.el
+++ b/contrib/lisp/ox-groff.el
@@ -1248,7 +1248,7 @@ INFO is a plist holding contextual information.  See
 ((string= type "file") (org-export-file-uri raw-path))
 (t raw-path
 (cond
- ((org-export-custom-protocol-maybe link desc 'groff))
+ ((org-export-custom-protocol-maybe link desc 'groff info))
  ;; Image file.
  (imagep (org-groff-link--inline-image link info))
  ;; import groff files
diff --git a/lisp/ol.el b/lisp/ol.el
index ce53b3e69..f850a5313 100644
--- a/lisp/ol.el
+++ b/lisp/ol.el
@@ -86,42 +86,93 @@
   :group 'org)
 
 (defcustom org-link-parameters nil
-  "An alist of properties that defines all the links in Org mode.
+  "Alist of properties that defines all the links in Org mode.
+
 The key in each association is a string of the link type.
-Subsequent optional elements make up a plist of link properties.
+Subsequent optional elements make up a property list for that
+type.
+
+All properties ar optional.  However, the most important ones
+are, in this order, `:follow', `:export', and `:store', described
+below.
+
+`:follow'
+
+  Function that takes the link path (a string) as an argument and
+  \"opens\" the link.
+
+`:export'
+
+  Function that accepts three mandatory arguments :
+  - the path, as a string,
+  - the description as a string, or nil,
+  - the export back-end.
+
+  Optionally, it may accept the export communication channel as
+  a fourth argument, in case you need using Org Export tooling,
+  e.g., `org-export-link-as-file'.
+
+  When nil, export for that type of link is delegated to the
+  back-end.
+
+`:store'
+
+  Function responsible for storing the link.  See the function
+  `org-store-link-functions' for a description of the expected
+  arguments.
+
+Additional properties provide more specific control over the
+link.
+
+`:activate-func'
+
+  Function to run at the end of Font Lock activation.  It must
+  accept four arguments:
+  - the 

Re: attachment: link type export to HTML invalid attach dir

2020-02-14 Thread Bastien
Nicolas Goaziou  writes:

>> Now, I'm not sure who wants to try implementing Nicolas suggestion to
>> let ox.el expand attachments into file links: since you both know the
>> issue better than I do, I suggest one of you can try?
>
> I will try to propose a patch by tomorrow evening.

Thanks!

If things are okay, I'll then have some time on sunday to prepare for
the release on monday.

-- 
 Bastien



Re: attachment: link type export to HTML invalid attach dir

2020-02-14 Thread Nicolas Goaziou
Hello,

Bastien  writes:

> Now, I'm not sure who wants to try implementing Nicolas suggestion to
> let ox.el expand attachments into file links: since you both know the
> issue better than I do, I suggest one of you can try?

I will try to propose a patch by tomorrow evening.

Regards,

-- 
Nicolas Goaziou



Re: attachment: link type export to HTML invalid attach dir

2020-02-14 Thread Bastien
Okay, I've done my home work :)

TL;DR: (1) The change is fresh, let's take the time to work together
at making it right before 9.4*; (2) the current state with the line
3216 in org-element.el is not okay; (3) let's try implementing what
Nicolas suggests wrt to attachment=>file link resolution.

* Sorry that I rushed you into having this discussion by announcing
  the release with such a short notice, I should have seen this was
  not resolve.

Now into more details.

- The new feature for attachments is nice and useful.  And I'm quite
  happy Gustav is now more acquainted with org-element.el -- we need
  more people with a good knowledge of that core file (thanks!)

- My definition of Org core link types = the minimal set of types to
  which other types can be translated to.  Given this definition, an
  attachment is not a core link type as it can resolve to file type.

- org-element.el should only know about core link types.

- Link type resolution should happen at exporting time: I think the
  design consistency is worth the potential parsing time cost.

- One advantage of this approach (on top of design consistency) is
  that exporters won't each depend on org-attach.el to deal with file
  attachments, as they do right now.

Now, I'm not sure who wants to try implementing Nicolas suggestion to
let ox.el expand attachments into file links: since you both know the
issue better than I do, I suggest one of you can try?

We are not pressed by time, so whoever has the energy to finish this.

Thanks!

-- 
 Bastien



Re: attachment: link type export to HTML invalid attach dir

2020-02-14 Thread Bastien
Hi Nicolas and Gustav,

Nicolas Goaziou  writes:

> I'm Cc'ing Bastien because that particular point should be solved before
> any release, IMO.

Thanks for adding me to the loop.

I'll take the time today or tomorrow to understand what is at stake
here and of course, I won't release Org 9.4 before it is sorted out.

Everyone can take a break from this thread :)

-- 
 Bastien



RE: attachment: link type export to HTML invalid attach dir

2020-02-13 Thread Gustav Wikström
Btw, the change 20d293b4a was done after a filed bug report. Maybe should be 
mentioned as well. The change was not something I just felt like doing just for 
the fun of it.

Regards
G

> -Original Message-
> From: Kyle Meyer 
> Sent: den 14 februari 2020 03:42
> To: Gustav Wikström ; Nicolas Goaziou
> 
> Cc: Bastien Guerry ; emacs-orgmode@gnu.org
> Subject: RE: attachment: link type export to HTML invalid attach dir
> 
> Gustav Wikström  writes:
> 
> >> Gustav Wikström  writes:
> >>
> >> > Ah, you mean the reference on line 3216. No, I don't think it can
> >> > be removed. And I honestly don't think it should be either. It's
> >> > there to let attachment links mirror the peculiarities of file
> >> > links. It's needed for feature compatibility. I don't see the
> issue with that.
> >> > It's a core link type and it needs the information. That
> particular
> >> > logic doesn't affect the parse tree. It's there only to give
> >> > attachment links the same properties as file links.
> >>
> >> I disagree. This is not a core link type. The issue is that the
> >> parser should self-contained. Please use a different way to obtain
> >> the information; we already discussed it and suggested multiple
> solutions.
> >
> > Maybe time for Bastien to step in. Because I can't remove the
> > reference to attachment in org-element.el without breaking it's
> > functionality. Which, btw, was broken before adding the reference in
> > org-element.el. The thing that started this discussion in the first
> > place. We're in a better place now.
> 
> It seems unfair to say you can't remove it because it would break
> functionality.  You committed 20d293b4a (Give link parser knowledge of
> attachment link expanded path, 2020-01-18) without posting it to the
> list [0] and giving Nicolas a chance to comment, which you've agreed
> was too hasty [1].  Misjudging the situation is of course okay, but
> please don't use that as a reason to keep a change that would not have
> landed if you had submitted it for discussion.
> 
> [0]: https://lists.gnu.org/archive/html/emacs-orgmode/2020-
> 01/msg00155.html
> [1]: https://lists.gnu.org/archive/html/emacs-orgmode/2020-
> 01/msg00162.html


RE: attachment: link type export to HTML invalid attach dir

2020-02-13 Thread Gustav Wikström
Hi,

> -Original Message-
> From: Kyle Meyer 
> Sent: den 14 februari 2020 03:42
> To: Gustav Wikström ; Nicolas Goaziou
> 
> Cc: Bastien Guerry ; emacs-orgmode@gnu.org
> Subject: RE: attachment: link type export to HTML invalid attach dir
> 
> Gustav Wikström  writes:
> 
> >> Gustav Wikström  writes:
> >>
> >> > Ah, you mean the reference on line 3216. No, I don't think it can
> >> > be removed. And I honestly don't think it should be either. It's
> >> > there to let attachment links mirror the peculiarities of file
> >> > links. It's needed for feature compatibility. I don't see the
> issue with that.
> >> > It's a core link type and it needs the information. That
> particular
> >> > logic doesn't affect the parse tree. It's there only to give
> >> > attachment links the same properties as file links.
> >>
> >> I disagree. This is not a core link type. The issue is that the
> >> parser should self-contained. Please use a different way to obtain
> >> the information; we already discussed it and suggested multiple
> solutions.
> >
> > Maybe time for Bastien to step in. Because I can't remove the
> > reference to attachment in org-element.el without breaking it's
> > functionality. Which, btw, was broken before adding the reference in
> > org-element.el. The thing that started this discussion in the first
> > place. We're in a better place now.
> 
> It seems unfair to say you can't remove it because it would break
> functionality.  You committed 20d293b4a (Give link parser knowledge of
> attachment link expanded path, 2020-01-18) without posting it to the
> list [0] and giving Nicolas a chance to comment, which you've agreed
> was too hasty [1].  Misjudging the situation is of course okay, but
> please don't use that as a reason to keep a change that would not have
> landed if you had submitted it for discussion.
> 
> [0]: https://lists.gnu.org/archive/html/emacs-orgmode/2020-
> 01/msg00155.html
> [1]: https://lists.gnu.org/archive/html/emacs-orgmode/2020-
> 01/msg00162.html

Just a short note before leaving for work. I'm not trying to strong-arm 
anything here. I'm just saying I don't think it's possible to remove the 
reference without breaking functionality. The changes discussed in [0] and [1] 
are in my opinion removed already. The reference to attachment we're talking 
about here is something else. This string reference is not a change in the 
design of org-element.el for links. The other was. And that's rolled back.

I'm sorry if this is perceived as disingenuous and being done with a bad 
agenda. It's not. It's a discussion about design. Yes, we disagree about some 
things. Nothing strange there. Let's talk it out and try to find the middle 
ground. Which is what's going on. I'll respect the direction given by the 
better Org moders out there. But in this case I'm simply stating that the 
principles that needs to apply according to Nicolas will either break 
attachment link functionality (it was broken before my change in 20d293b4a) or 
require quite some thinking and/or code duplication in org-attach.el to get it 
right. And I don't have the capacity for that redesign. Rolling back the change 
to a broken state is ofc. still possible, but that is more of a regression in 
my mind than the string-reference to attachment link types in org-element.el.

Which is why I think it's a decision for the maintainer of the project to step 
in and say if it's fine to have broken attachment links, or if it's fine to let 
attachment be used the way it is on line 3216 in org-element.el, while there is 
hard-coded logic for file links the way it is on that line today.

Regards
Gustav


RE: attachment: link type export to HTML invalid attach dir

2020-02-13 Thread Gustav Wikström
Hi,

> -Original Message-
> From: Nicolas Goaziou 
> Sent: den 14 februari 2020 01:16
> To: Gustav Wikström 
> Cc: emacs-orgmode@gnu.org; Bastien Guerry 
> Subject: Re: attachment: link type export to HTML invalid attach dir
> 
> Gustav Wikström  writes:
> 
> > What makes attachment links not be a core link type?
> 
> 1. Org Attach is not loaded by default
> 2. Implementing Org syntax doesn't require to implement attachments
> 
> Attachment links are in the same category as Docview links, for
> example.
> There is no "docview" occurrence in "org-element.el". We have been
> there already.
> 
> > As I see it we have two categories here: Core = those inside Org
> mode.
> > Not core = those in external libraries. No other distinction needs to
> > be made. If a link type is inside Org mode, let it then use the
> > special properties that Org mode provides.
> 
> Sure thing, but you use the wrong ones!

I meant that Core = inside the Org mode repository of source code files. Both 
Docview and Attachment is, both are mentioned in the manual as part of the 
system. Both should be able to use :search-option inside the Org element parser.

> > Attachment links need to be treated just as file links in most
> > regards. Since file-links have special logic which attachment links
> > also needs. Thus a reference to attachment in org-element.el. Nothing
> > strange here, nothing breaking and the parser is still self-
> contained.
> 
> Attachement links should live only in "org-attach.el": like 99% of
> other links do live in their own library. They do not require a special
> treatment.
> 
> Again, the parser is not self-contained if it references org-attach.el.

It doesn't referencen org-attach.el any longer. That reference is removed. What 
remains is a reference to the string "attachment". You may argue that even that 
is to much. And I can only agree with that vision if you also state that 
org-element shouldn't reference the string "file" either. Anything else is a 
subjective design that puts a sentiment on "more" or "less" importance of link 
types and how to regard if they're "important" enough to be mentioned. I don't 
think you want that in org-element any more than I do.

> > Maybe time for Bastien to step in. Because I can't remove the
> > reference to attachment in org-element.el without breaking it's
> > functionality. Which, btw, was broken before adding the reference in
> > org-element.el. The thing that started this discussion in the first
> > place. We're in a better place now.
> 
> We're not. The way it is implemented is not correct.

Functionally we are - we have working attachment links that are treated in the 
same way as file links through out Org mode. Since there is special treatment 
for file links in Org, attachment links needs the same special treatment to get 
the same behavior. You might see it as a regression in the design of 
org-element.el. To which I've already given arguments about why it's not really 
- and different ways it can be solved - by removing special file-related 
properties from org-element.el. The added string-reference to attachment links 
is there because special string-references to file links are there. If special 
file link references are removed, then clearly attachment links will/should be 
removed as well.

> For opening link, attachment links should use the :follow link
> parameter. The "following" function can extract the search option, if
> any, and the file name, then call `org-open-file' appropriately. You're
> doing it backwards and modify `org-open-file' instead. Additional links
> are not supposed to modify "ol.el".

Sure, that's a principle I can agree to. And have already mentioned that I will 
try to refactor that part by giving the :follow function the same abilities as 
org-open-file currently has. But that won't be completed quite yet.

> For exporting link, it should use the :export link parameter. The
> "exporting" function can extract the search option, if any, and the
> file name. It can then re-create a file link object, and call `org-
> export-data' on it. You're also doing it backwards here and prefer to
> modify each export back-end instead.

Which I've argued about already, saying it will be difficult without having 
file already being treated the same way. I'm not opposed to doing it that way 
in the long run. But let it be done for file links first, then attachment links 
can follow. I don't have the capacity to dig into this on my own.

> > Attachment links works as intended. It would be a pity to have to
> > change that because of your argument that attachment links aren't
> > "core" enough t

RE: attachment: link type export to HTML invalid attach dir

2020-02-13 Thread Kyle Meyer
Gustav Wikström  writes:

>> Gustav Wikström  writes:
>> 
>> > Ah, you mean the reference on line 3216. No, I don't think it can be
>> > removed. And I honestly don't think it should be either. It's there to
>> > let attachment links mirror the peculiarities of file links. It's
>> > needed for feature compatibility. I don't see the issue with that.
>> > It's a core link type and it needs the information. That particular
>> > logic doesn't affect the parse tree. It's there only to give
>> > attachment links the same properties as file links.
>> 
>> I disagree. This is not a core link type. The issue is that the parser
>> should self-contained. Please use a different way to obtain the
>> information; we already discussed it and suggested multiple solutions.
>
> Maybe time for Bastien to step in. Because I can't remove the
> reference to attachment in org-element.el without breaking it's
> functionality. Which, btw, was broken before adding the reference in
> org-element.el. The thing that started this discussion in the first
> place. We're in a better place now.

It seems unfair to say you can't remove it because it would break
functionality.  You committed 20d293b4a (Give link parser knowledge of
attachment link expanded path, 2020-01-18) without posting it to the
list [0] and giving Nicolas a chance to comment, which you've agreed was
too hasty [1].  Misjudging the situation is of course okay, but please
don't use that as a reason to keep a change that would not have landed
if you had submitted it for discussion.

[0]: https://lists.gnu.org/archive/html/emacs-orgmode/2020-01/msg00155.html
[1]: https://lists.gnu.org/archive/html/emacs-orgmode/2020-01/msg00162.html



Re: attachment: link type export to HTML invalid attach dir

2020-02-13 Thread Nicolas Goaziou
Gustav Wikström  writes:

> What makes attachment links not be a core link type? 

1. Org Attach is not loaded by default
2. Implementing Org syntax doesn't require to implement attachments

Attachment links are in the same category as Docview links, for example.
There is no "docview" occurrence in "org-element.el". We have been there
already.

> As I see it we have two categories here: Core = those inside Org mode.
> Not core = those in external libraries. No other distinction needs to
> be made. If a link type is inside Org mode, let it then use the
> special properties that Org mode provides.

Sure thing, but you use the wrong ones!

> Attachment links need to be treated just as file links in most
> regards. Since file-links have special logic which attachment links
> also needs. Thus a reference to attachment in org-element.el. Nothing
> strange here, nothing breaking and the parser is still self-contained.

Attachement links should live only in "org-attach.el": like 99% of other
links do live in their own library. They do not require a special
treatment.

Again, the parser is not self-contained if it references org-attach.el.

> Maybe time for Bastien to step in. Because I can't remove the
> reference to attachment in org-element.el without breaking it's
> functionality. Which, btw, was broken before adding the reference in
> org-element.el. The thing that started this discussion in the first
> place. We're in a better place now. 

We're not. The way it is implemented is not correct.

For opening link, attachment links should use the :follow link
parameter. The "following" function can extract the search option, if
any, and the file name, then call `org-open-file' appropriately. You're
doing it backwards and modify `org-open-file' instead. Additional links
are not supposed to modify "ol.el".

For exporting link, it should use the :export link parameter. The
"exporting" function can extract the search option, if any, and the file
name. It can then re-create a file link object, and call
`org-export-data' on it. You're also doing it backwards here and prefer
to modify each export back-end instead.

> Attachment links works as intended. It would be a pity to have to
> change that because of your argument that attachment links aren't
> "core" enough to be mentioned in org-element.el.

I strongly insist that it should be removed. I gave you ways to solve
the issue. If you need anything else, let me know, but please consider
implementing them.



RE: attachment: link type export to HTML invalid attach dir

2020-02-13 Thread Gustav Wikström
> -Original Message-
> From: Nicolas Goaziou 
> Sent: den 13 februari 2020 22:38
> To: Gustav Wikström 
> Cc: emacs-orgmode@gnu.org; Bastien Guerry 
> Subject: Re: attachment: link type export to HTML invalid attach dir
> 
> Gustav Wikström  writes:
> 
> > Ah, you mean the reference on line 3216. No, I don't think it can be
> > removed. And I honestly don't think it should be either. It's there to
> > let attachment links mirror the peculiarities of file links. It's
> > needed for feature compatibility. I don't see the issue with that.
> > It's a core link type and it needs the information. That particular
> > logic doesn't affect the parse tree. It's there only to give
> > attachment links the same properties as file links.
> 
> I disagree. This is not a core link type. The issue is that the parser
> should self-contained. Please use a different way to obtain the
> information; we already discussed it and suggested multiple solutions.

What makes attachment links not be a core link type? As I see it we have two 
categories here: Core = those inside Org mode. Not core = those in external 
libraries. No other distinction needs to be made. If a link type is inside Org 
mode, let it then use the special properties that Org mode provides. Attachment 
links need to be treated just as file links in most regards. Since file-links 
have special logic which attachment links also needs. Thus a reference to 
attachment in org-element.el. Nothing strange here, nothing breaking and the 
parser is still self-contained.

Maybe time for Bastien to step in. Because I can't remove the reference to 
attachment in org-element.el without breaking it's functionality. Which, btw, 
was broken before adding the reference in org-element.el. The thing that 
started this discussion in the first place. We're in a better place now. 
Attachment links works as intended. It would be a pity to have to change that 
because of your argument that attachment links aren't "core" enough to be 
mentioned in org-element.el.

Regards
Gustav



RE: attachment: link type export to HTML invalid attach dir

2020-02-13 Thread Gustav Wikström
Continuation from previous answer,

> -Original Message-
> From: Nicolas Goaziou 
> Sent: den 13 februari 2020 21:41
> To: Gustav Wikström 
> Cc: emacs-orgmode@gnu.org; Bastien Guerry 
> Subject: Re: attachment: link type export to HTML invalid attach dir
> 
> [...]
> 
> > Given the above paragraph, the :path and :search-option property
> > doesn't make sense in the parser. :raw-link is enough. Less ambiguous
> > names for :path and :search-option would be :file-path
> > and :file-search-option. But that's sub-typing. We've already
> > concluded that that should not belong to the parser.
> 
> I don't have much time, I apologize if I'm not clear.
> 
> I disagree with you conclusion. Sub-typing is necessary in the parser.
> The `link' object is complex, so it needs categories or types. There are
> plain links, angle links, square links. In this last category, there are
> internal links, which include coderefs, fuzzy links, custom id, file
> links, and, "links with an explicit scheme part". For each of these
> categories, it is fine to have specific properties, like search-options.

Sure, that's not too far away from what I suggested in one of my first mails on 
the subject (except I took it a few steps too far maybe). Saying there are 
properties for categories of sub-types is fine but then this needs to be made 
much more explicit than today. 

> Note that this is sub-typing per syntax, not by meaning. This is what
> should not move, unless absolutely necessary. For example, attachment
> links belong to "square links with an explicit 'attachment' scheme
> part". That is all the parser needs to know.

That, and the way to extract the :search-options and :application from the 
link, as is done for file-links. Since attachment links would fit into your 
category of links that needs specific properties, like search-options.

> Now, it is true that [[file:...]] links are put in the same category as
> [[~/...]] links, for practical reasons, i.e., you wouldn't want them to
> differ in meaning. But this is a breach in the categories above.

Perfectly valid in my opinion. [[~/...]] is shorthand for [[file:~/...]] and, 
as long as documented as such, I see no breach. Only one such shorthand is 
possible anyways so not much ambiguity there.

> We might ignore the :search-option part in file links, but it's very
> easy to get, and, more importantly, we must extract the filename, so
> further parsing is necessary.

As you might have figured, I don't hold a very strong opinion on which design 
decision to take. I just want the choice to be based on principles and to be 
without ambiguities. If you say there are sub-types of links that require the 
:search-option, then fine, let it be so. But then we have to make it perfectly 
clear which particular types this applies to. And also make it clear that those 
special properties are only available for built-in types. I.e. to use them one 
have to get the new link-type into Org mode and into org-element.el.

> > I agree that option 1 is suboptimal. :search-option isn't obvious as
> > a property for all link types. Since option 3 is fairly trivial,
> > option 2 isn't necessary either. For attachment links to reuse
> > the :search-option semantics, without the hard-coding we're currently
> > doing, I see one option where attachment link higher order functions
> > could reuse file link higher order functions. Those file link higher
> > order functions don't exist yet as far as I know.
> 
> Higher order functions for what? `org-open-file' is such a higher order
> function for opening file links. There is no higher order function for
> exporting because each exporter handles file links differently. A higher
> order function for parsing doesn't mean much, since the consumer is not
> known yet.

Yes, each exporter handles file links differently. And I'm saying each exporter 
shouldn't handle file links at all. They should delegate that to the file 
exporter higher order function. In the same way that other link types are 
supposed to be dealt with. Principles.

> At this point, the best thing to do is to clarify what you need.
> I probably do not understand it.

I'm trying :) But it's not that /I need/ anything. It's rather the issue of how 
to solve the conundrum we're in. Where you say "attachment" links should work 
differently and not be hardcoded next to file links. And I'm saying it won't 
work unless we refactor how file-links are handled. And how we should have a 
principled approach to these kinds of things.

> > It might be seducing but I'm not sold. I'd rather have an
> > attachment-link exporter explicitly reuse functionality for exporting
> > file links than automatic translation. For that to be possible, there
> > first is a 

Re: attachment: link type export to HTML invalid attach dir

2020-02-13 Thread Nicolas Goaziou
Gustav Wikström  writes:

> Ah, you mean the reference on line 3216. No, I don't think it can be
> removed. And I honestly don't think it should be either. It's there to
> let attachment links mirror the peculiarities of file links. It's
> needed for feature compatibility. I don't see the issue with that.
> It's a core link type and it needs the information. That particular
> logic doesn't affect the parse tree. It's there only to give
> attachment links the same properties as file links.

I disagree. This is not a core link type. The issue is that the parser
should self-contained. Please use a different way to obtain the
information; we already discussed it and suggested multiple solutions.



RE: attachment: link type export to HTML invalid attach dir

2020-02-13 Thread Gustav Wikström
Hi,

Partial answer for now;

> -Original Message-
> From: Nicolas Goaziou 
> Sent: den 13 februari 2020 21:41
> To: Gustav Wikström 
> Cc: emacs-orgmode@gnu.org; Bastien Guerry 
> Subject: Re: attachment: link type export to HTML invalid attach dir
>
> [...]
>
> There is an "attachement" reference left in "org-element.el" at the
> moment. Can it be removed safely?

Ah, you mean the reference on line 3216. No, I don't think it can be removed. 
And I honestly don't think it should be either. It's there to let attachment 
links mirror the peculiarities of file links. It's needed for feature 
compatibility. I don't see the issue with that. It's a core link type and it 
needs the information. That particular logic doesn't affect the parse tree. 
It's there only to give attachment links the same properties as file links.

> I'm Cc'ing Bastien because that particular point should be solved
> before any release, IMO.
> 
> Regards,
> 
> --
> Nicolas Goaziou


Re: attachment: link type export to HTML invalid attach dir

2020-02-13 Thread Nicolas Goaziou
Hello,

Gustav Wikström  writes:

> Unless search-option applies as a general construct for all link types
> I don't think it should be in the parser. Given the discussion we've
> had about design of the parser from before. I.e. if the parser isn't
> supposed to know anything about the specific types themselves, all
> properties of the parsed elements have to make sense for all types of
> links. So either search-option remains but is parsed in exactly the
> same way for all link types, or it's not there at all. And if it's not
> available in the parser, the file link interpreters (that still might
> need to be constructed) gets to parse the :search-option from the
> raw-link as it wishes itself.
>
> Given the above paragraph, the :path and :search-option property
> doesn't make sense in the parser. :raw-link is enough. Less ambiguous
> names for :path and :search-option would be :file-path
> and :file-search-option. But that's sub-typing. We've already
> concluded that that should not belong to the parser.

I don't have much time, I apologize if I'm not clear.

I disagree with you conclusion. Sub-typing is necessary in the parser.
The `link' object is complex, so it needs categories or types. There are
plain links, angle links, square links. In this last category, there are
internal links, which include coderefs, fuzzy links, custom id, file
links, and, "links with an explicit scheme part". For each of these
categories, it is fine to have specific properties, like search-options.

Note that this is sub-typing per syntax, not by meaning. This is what
should not move, unless absolutely necessary. For example, attachment
links belong to "square links with an explicit 'attachment' scheme
part". That is all the parser needs to know.

Now, it is true that [[file:...]] links are put in the same category as
[[~/...]] links, for practical reasons, i.e., you wouldn't want them to
differ in meaning. But this is a breach in the categories above.

We might ignore the :search-option part in file links, but it's very
easy to get, and, more importantly, we must extract the filename, so
further parsing is necessary.

> I agree that option 1 is suboptimal. :search-option isn't obvious as
> a property for all link types. Since option 3 is fairly trivial,
> option 2 isn't necessary either. For attachment links to reuse
> the :search-option semantics, without the hard-coding we're currently
> doing, I see one option where attachment link higher order functions
> could reuse file link higher order functions. Those file link higher
> order functions don't exist yet as far as I know.

Higher order functions for what? `org-open-file' is such a higher order
function for opening file links. There is no higher order function for
exporting because each exporter handles file links differently. A higher
order function for parsing doesn't mean much, since the consumer is not
known yet.

At this point, the best thing to do is to clarify what you need.
I probably do not understand it.

> True that. There is a balance to strike. Maybe it's time to pull in
> the org element document into the manual? I vote for that at least.

I have no strong opinion on that. It may fit into an appendix, indeed,
or even as a section in first appendix: "Hacking", e.g. "Using the
Parser API".

> Ok, sure. Let the syntax be as rigid as possible, and let
> extensibility to that be dealt with in auxiliary parsing/interpreting
> functions. I guess that would be the approach, put in different words.
> Still correct?

It seems correct.

> It might be seducing but I'm not sold. I'd rather have an
> attachment-link exporter explicitly reuse functionality for exporting
> file links than automatic translation. For that to be possible, there
> first is a need for a file link exporter function.

I don't see how that would be possible, since it would be different for
each back-end. Again, do you have a more precise idea about what it
would do?

> And the current implementation (since the patch) is good enough imo,
> for now, and until anyone of us does some file link refactoring.

There is an "attachement" reference left in "org-element.el" at the
moment. Can it be removed safely?

I'm Cc'ing Bastien because that particular point should be solved before
any release, IMO.

Regards,

-- 
Nicolas Goaziou



RE: attachment: link type export to HTML invalid attach dir

2020-02-08 Thread Gustav Wikström
Hi,

> -Original Message-
> From: Nicolas Goaziou 
> Sent: den 7 februari 2020 15:28
> To: Gustav Wikström 
> Cc: emacs-orgmode@gnu.org
> Subject: Re: attachment: link type export to HTML invalid attach dir
> 
> > Hmm, maybe that is so.. Except raw-path is called path (not really an
> > issue though) and there is another property called raw-link. Not sure
> > how to interpret the use of both path and raw-link. And there are
> > things happening between parsing the actual buffer link and storing
> > the raw-link and path parameters.
> 
> There is some normalization involved, indeed. The intent of :raw-link is
> to provide the link almost as it was in the buffer, without any parsing.
> I agree there should not be any `org-link-unescape' and
> `org-link-expand-abbrev' involved there. Something to fix at some point.
> 
> :path, on the contrary is parsed. It is the part of the link between the
> type and the search options, i.e., [[type:path::search]], [[type:path]],
> or [[path]].

Unless search-option applies as a general construct for all link types I don't 
think it should be in the parser. Given the discussion we've had about design 
of the parser from before. I.e. if the parser isn't supposed to know anything 
about the specific types themselves, all properties of the parsed elements have 
to make sense for all types of links. So either search-option remains but is 
parsed in exactly the same way for all link types, or it's not there at all. 
And if it's not available in the parser, the file link interpreters (that still 
might need to be constructed) gets to parse the :search-option from the 
raw-link as it wishes itself. 

Given the above paragraph, the :path and :search-option property doesn't make 
sense in the parser. :raw-link is enough. Less ambiguous names for :path and 
:search-option would be :file-path and :file-search-option. But that's 
sub-typing. We've already concluded that that should not belong to the parser.

> > In the end, what made me consider the sub-typing I've been writing
> > about could very well just have been the ambiguities regarding what's
> > what, and the lack of treatment for parts that arguably could be seen
> > as additional parameters to the link-path. For example the
> > "::extra_content" suffix in file links, that by the parser currently
> > is just a part of the path itself.
> 
> In [[file:name::extra_content]] :path is "name", and :search-option is
> "extra_content". As you noted, :search-option is not valid in non-file
> links, so it belongs to the path.
> 
> It seems there is some friction about this search option part. IIUC, you
> want attachment links to support this link-specific syntax. This is
> indeed not possible. As I commented earlier, letting libraries decide
> what the parser should do is not an option. There are a few options,
> though:
> 
> 1. Allow every link with an explicit type (i.e., not internal links) to
>have a search option, so you can write [[docview:filename::12]] or
>[[attachment:filename::*Section 2]]. IMO this is a waste, because
>most links do not need this, and it could become confusing
>[[https:www.orgmode.org::#sec2]].
> 
> 2. Provide a function in "ol.el" to do the parsing for you, so that
>every new link library doesn't have to re-invent the wheel. E.g.,
>(org-link-extract-search-options "foo::bar") => (:path
>"foo" :search-options "bar").
> 
> 3. Keep that way, i.e., any link library requiring to read the search
>part can do a dumb `match-string` and, in two or three lines of code,
>extract it. IOW, since this is so trivial, why bother?
> 
> WDYT?

I agree that option 1 is suboptimal. :search-option isn't obvious as a property 
for all link types. Since option 3 is fairly trivial, option 2 isn't necessary 
either. For attachment links to reuse the :search-option semantics, without the 
hard-coding we're currently doing, I see one option where attachment link 
higher order functions could reuse file link higher order functions. Those file 
link higher order functions don't exist yet as far as I know.

> > Maybe clearer documentation in the function of what each part is
> > /supposed/ to be, and the design principle that is applied, i.e. that
> > the path is the raw path with options included can help future me and
> > others who might want to understand. Thoughts on that?
> 
> There is some information in the manual, and the Org Syntax document.
> 
> > Hmm, don't really know if a link description should be regarded as
> > content. I for one hadn't considered it until now when you mentioned
> > it! But it preserves space in the parse tree I suppose.
> 
> Thi

Re: attachment: link type export to HTML invalid attach dir

2020-02-07 Thread Nicolas Goaziou
Hello,

Gustav Wikström  writes:

> Hmm, maybe that is so.. Except raw-path is called path (not really an
> issue though) and there is another property called raw-link. Not sure
> how to interpret the use of both path and raw-link. And there are
> things happening between parsing the actual buffer link and storing
> the raw-link and path parameters.

There is some normalization involved, indeed. The intent of :raw-link is
to provide the link almost as it was in the buffer, without any parsing.
I agree there should not be any `org-link-unescape' and
`org-link-expand-abbrev' involved there. Something to fix at some point.

:path, on the contrary is parsed. It is the part of the link between the
type and the search options, i.e., [[type:path::search]], [[type:path]],
or [[path]].

> In the end, what made me consider the sub-typing I've been writing
> about could very well just have been the ambiguities regarding what's
> what, and the lack of treatment for parts that arguably could be seen
> as additional parameters to the link-path. For example the
> "::extra_content" suffix in file links, that by the parser currently
> is just a part of the path itself.

In [[file:name::extra_content]] :path is "name", and :search-option is
"extra_content". As you noted, :search-option is not valid in non-file
links, so it belongs to the path.

It seems there is some friction about this search option part. IIUC, you
want attachment links to support this link-specific syntax. This is
indeed not possible. As I commented earlier, letting libraries decide
what the parser should do is not an option. There are a few options,
though:

1. Allow every link with an explicit type (i.e., not internal links) to
   have a search option, so you can write [[docview:filename::12]] or
   [[attachment:filename::*Section 2]]. IMO this is a waste, because
   most links do not need this, and it could become confusing
   [[https:www.orgmode.org::#sec2]].

2. Provide a function in "ol.el" to do the parsing for you, so that
   every new link library doesn't have to re-invent the wheel. E.g.,
   (org-link-extract-search-options "foo::bar") => (:path
   "foo" :search-options "bar").

3. Keep that way, i.e., any link library requiring to read the search
   part can do a dumb `match-string` and, in two or three lines of code,
   extract it. IOW, since this is so trivial, why bother?

WDYT?

> Maybe clearer documentation in the function of what each part is
> /supposed/ to be, and the design principle that is applied, i.e. that
> the path is the raw path with options included can help future me and
> others who might want to understand. Thoughts on that?

There is some information in the manual, and the Org Syntax document.

> Hmm, don't really know if a link description should be regarded as
> content. I for one hadn't considered it until now when you mentioned
> it! But it preserves space in the parse tree I suppose.

This is unrelated to the size of the parse tree. 

A description may contain Org specific markup, e.g., bold, so it needs
to be parsed further. Therefore, a link with a description is not
a leaf: it has contents.

> If the docstring of the link parser would make it clear what each
> property is supposed to contain, in this case :contents-begin
> & :contents-end, I'm sure I would have been less confused about this,
> and wouldn't have had any objections.

A docstring is not a manual. I explained this in the Org Element API
documentation, IIRC.

> Ok, got it. You're saying the link interpreter for docview (in this
> case) have to be able to parse the link one step further, to be able
> to extract this ":go-to-page" information, before being able to
> operate on it. Indeed a design decision. Maybe the best one, who am
> I to say otherwise. It will make the Org link parser leaner for sure.

Sarcasm?

My opinion is the library defining the Org syntax should be a fixed
point. It is important that the output of the parser does not depend on
what libraries are currently loaded. It guarantees that if we both open
the same document, we'll get the same parse tree.

Of course, this is not exactly true at the moment. For example, we may
not have the same TODO keywords, or the same item list separator.
Nevertheless, I see no reason to diverge further from this goal.

> Not sure about location. Maybe ol.el? What I'm talking about is
> a higher order function for the :export property within
> org-link-parameters. File link exporting would then have to be
> declared using org-link-set-parameters, just as other link types are
> supposed to be declared for exporting.

See above about my suggestion of a generic tool.

>> As suggested already (I think) we could add a phase in ox.el that would
>> expand attachment links into their file link counterpart automatically.
>> This would avoid adding specific code in every export back-end. However,
>> export back-end would miss the opportunity to handle attachment links
>> specifically. This is one way or the 

RE: attachment: link type export to HTML invalid attach dir

2020-02-06 Thread Gustav Wikström
Hi again,

> -Original Message-
> From: Nicolas Goaziou 
> Sent: den 5 februari 2020 17:54
> To: Gustav Wikström 
> Cc: emacs-orgmode@gnu.org
> Subject: Re: attachment: link type export to HTML invalid attach dir
> 
> > That was kind of what I was trying to do, by allowing subtyping, so
> > that the parser would be more knowledgeable about particular types of
> > links, by allowing extended attributes. Maybe not implemented in the
> > most extensible way though I'll admit.
> 
> Just to be clear, I firmly believe Element library should be as
> low-level as possible. Ideally (we're not there yet), it should not
> depend on any other part of Org. It should be the step (largely) above
> "re-search-forward".

Ok, fair enough. I guess where I was going was a bit further than only parsing 
the characters, into also interpreting them in meaningful ways. Especially true 
for attachment links that would need to look up other parts of the tree to 
fetch extended information. I'll leave that trail of thought. 

> > My suggestion to the link parser would be to have the following properties 
> > as the catch-all properties for links:
> > - type :: Which type of link protocol applies. E.g. file, http, ftp, 
> > attachment, duckduckgo (ex. for a custom link abbreviation to search ddg)
> > - raw-path :: The path to use for the particular protocol. Would contain 
> > everything in the link following "protocol:"
> > - format :: Which format the link has. Plain, bracket or angular bracket
> 
> Barring custom link abbreviations, this is already the case.

Hmm, maybe that is so.. Except raw-path is called path (not really an issue 
though) and there is another property called raw-link. Not sure how to 
interpret the use of both path and raw-link. And there are things happening 
between parsing the actual buffer link and storing the raw-link and path 
parameters. 

In the end, what made me consider the sub-typing I've been writing about could 
very well just have been the ambiguities regarding what's what, and the lack of 
treatment for parts that arguably could be seen as additional parameters to the 
link-path. For example the "::extra_content" suffix in file links, that by the 
parser currently is just a part of the path itself. Maybe clearer documentation 
in the function of what each part is /supposed/ to be, and the design principle 
that is applied, i.e. that the path is the raw path with options included can 
help future me and others who might want to understand. Thoughts on that? 

> > - description :: The description part of the link 
> > ([[protocol:path][description]])
> 
> Description is not meaningful here. This is parsed content.
>
> > - begin, end, post-blank :: The default properties used for all elements. 
> > Not sure if contents-begin and contents-end are a part here as well.
> 
> They are when the link has contents, i.e., a description.

Hmm, don't really know if a link description should be regarded as content. I 
for one hadn't considered it until now when you mentioned it! But it preserves 
space in the parse tree I suppose. If the docstring of the link parser would 
make it clear what each property is supposed to contain, in this case 
:contents-begin & :contents-end, I'm sure I would have been less confused about 
this, and wouldn't have had any objections.

> > As you've stated previously, from a performance perspective maybe it
> > will be best to not expand the specific properties and instead let the
> > expansion of those be handled in code by the link type owner (e.g.
> > org-attach.el for attachment links).
> 
> More than performance, this is a design issue.
> 
> > Ah yes. Just briefly looking at the docview code. Docview doesn't seem
> > to use the parser when extracting information about that "go to page"
> > information from the link. Which means docview links doesn't really
> > care how the link information is encoded in the parser anyways.
> 
> Indeed. However, higher level functions, e.g., org-open-link, do care
> about it and ensure that "ol-docview.el" really processes a link.
> 
> > Maybe if docview were allowed to encode custom docview information
> > into the parsed link in the parser it and others could reused that
> > encoded information later? Docview would be able to add
> > a ":go-to-page" option, for example. Just a thought.
> 
> I'm quite sure this is a wrong way to go.

Ok, got it. You're saying the link interpreter for docview (in this case) have 
to be able to parse the link one step further, to be able to extract this 
":go-to-page" information, before being able to operate on it. Indeed a design 
decision. Maybe the best one, who am I to say

Re: attachment: link type export to HTML invalid attach dir

2020-02-05 Thread Nicolas Goaziou
Hello,

Gustav Wikström  writes:

> That was kind of what I was trying to do, by allowing subtyping, so
> that the parser would be more knowledgeable about particular types of
> links, by allowing extended attributes. Maybe not implemented in the
> most extensible way though I'll admit.

Just to be clear, I firmly believe Element library should be as
low-level as possible. Ideally (we're not there yet), it should not
depend on any other part of Org. It should be the step (largely) above
"re-search-forward".

So, extending the parser to include knowledge to some other library in
Org is not an option. Element is expected to dumbly stick to Org syntax,
without knowing what it means.

> My suggestion to the link parser would be to have the following properties as 
> the catch-all properties for links:
> - type :: Which type of link protocol applies. E.g. file, http, ftp, 
> attachment, duckduckgo (ex. for a custom link abbreviation to search ddg)
> - raw-path :: The path to use for the particular protocol. Would contain 
> everything in the link following "protocol:"
> - format :: Which format the link has. Plain, bracket or angular bracket

Barring custom link abbreviations, this is already the case.

> - description :: The description part of the link 
> ([[protocol:path][description]])

Description is not meaningful here. This is parsed content.

> - begin, end, post-blank :: The default properties used for all elements. Not 
> sure if contents-begin and contents-end are a part here as well.

They are when the link has contents, i.e., a description.

> As you've stated previously, from a performance perspective maybe it
> will be best to not expand the specific properties and instead let the
> expansion of those be handled in code by the link type owner (e.g.
> org-attach.el for attachment links).

More than performance, this is a design issue.

> Ah yes. Just briefly looking at the docview code. Docview doesn't seem
> to use the parser when extracting information about that "go to page"
> information from the link. Which means docview links doesn't really
> care how the link information is encoded in the parser anyways.

Indeed. However, higher level functions, e.g., org-open-link, do care
about it and ensure that "ol-docview.el" really processes a link.

> Maybe if docview were allowed to encode custom docview information
> into the parsed link in the parser it and others could reused that
> encoded information later? Docview would be able to add
> a ":go-to-page" option, for example. Just a thought.

I'm quite sure this is a wrong way to go.

> Most link types probably won't need custom link properties anyways.
> But could maybe let the parser know stuff by use of higher order
> functions? Or push for being important enough to be included into Org
> and allowed to be known by the parser.

Only syntax changes can go into the parser. Attachment links are regular
links, not something that requires extending Org's syntax. Note that
this is not the case of file links, which have a special syntax, e.g.,
[[./cat.jpg]].

> Ok, got it. Thanks for explaining. I'll admit it hasn't been totally
> clear to me what the best way forward is. After refreshing my
> understanding on links the conclusion I came to was that link
> abbreviations in its current form was not a fitting concept for
> attachment links. Attachments are pretty much similar to file links
> though, so even if my intention isn't to blur the meaning they will
> and should be treated very similar, since functionality is so similar.

Functionality may be familiar, but their syntax is not the same,
therefore, the differences should be handled elsewhere.

> Speaking from the perspective of attachment links, if there were
> a file link type exporter function available that could be used by
> attachment links without code duplication.

Where would live that function? ox.el is not an option since every
exporter treats file links differently.

As suggested already (I think) we could add a phase in ox.el that would
expand attachment links into their file link counterpart automatically.
This would avoid adding specific code in every export back-end. However,
export back-end would miss the opportunity to handle attachment links
specifically. This is one way or the other.

> Yes, and the function that implements that would also need the user
> argument as input. In addition to the link path.

Patch welcome. I see no objection to extending :follow link parameter in
"ol.el".

> Anyways, patch attached that makes org-element not know about
> attachments any longer, and moves most of the responsibility to
> org-attach.el. The exporters still needs to know to use the new
> function inside org-attach.el though.

Thank you. Please apply it. 

Now we can try improving the situation for exporters.

Regards,

-- 
Nicolas Goaziou



RE: attachment: link type export to HTML invalid attach dir

2020-01-25 Thread Gustav Wikström
Hi again,

> -Original Message-
> From: Nicolas Goaziou 
> Sent: den 20 januari 2020 02:25
> To: Gustav Wikström 
> Cc: emacs-orgmode@gnu.org
> Subject: Re: attachment: link type export to HTML invalid attach dir
> 
> Gustav Wikström  writes:
> 
> > Ok, noted. To my defense I was thinking more in the terms of subtyping
> > then hardcoding. Because we have multiple link types which try to
> > share an interface. But the interface isn't perfect for all different
> > kinds of links.
> 
> Then, I suggest to improve the interface.

That was kind of what I was trying to do, by allowing subtyping, so that the 
parser would be more knowledgeable about particular types of links, by allowing 
extended attributes. Maybe not implemented in the most extensible way though 
I'll admit.

My suggestion to the link parser would be to have the following properties as 
the catch-all properties for links:
- type :: Which type of link protocol applies. E.g. file, http, ftp, 
attachment, duckduckgo (ex. for a custom link abbreviation to search ddg)
- raw-path :: The path to use for the particular protocol. Would contain 
everything in the link following "protocol:"
- format :: Which format the link has. Plain, bracket or angular bracket
- description :: The description part of the link 
([[protocol:path][description]])
- begin, end, post-blank :: The default properties used for all elements. Not 
sure if contents-begin and contents-end are a part here as well.

In addition to that each protocol would be able to declare their own properties 
using a :parse function. All would probably implement a path property. Some 
would implement an options property, and some maybe would maybe declare their 
own properties.

As you've stated previously, from a performance perspective maybe it will be 
best to not expand the specific properties and instead let the expansion of 
those be handled in code by the link type owner (e.g. org-attach.el for 
attachment links). But not letting specific protocol parsers expand their own 
properties into a link makes it more difficult to implement support outside of 
emacs for the org specification, which in my opinion includes also a certain 
set of link protocols which do have their own specifications.

> > So it doesn't seem too farfetched that some of those link types would
> > benefit from additional custom properties, specific for those types.
> > =application= and =search-option= for example isn't universally
> > applicable to all link types.
> 
> As a side note, :application is on its way out, i.e., "file+emacs" stuff
> is in "org-compat.el".
> 
> Also, IIRC, "docview" links have "go to page" option, and they don't
> touch the parser.

Ah yes. Just briefly looking at the docview code. Docview doesn't seem to use 
the parser when extracting information about that "go to page" information from 
the link. Which means docview links doesn't really care how the link 
information is encoded in the parser anyways. Maybe if docview were allowed to 
encode custom docview information into the parsed link in the parser it and 
others could reused that encoded information later? Docview would be able to 
add a ":go-to-page" option, for example. Just a thought.

> > What I'm trying to argue for here is: Maybe it's not that crazy after
> > all to allow links to have additional properties in the parser based
> > on its type? (While certainly still trying to avoid it if possible!)
> 
> If new link types may not function correctly without touching the
> parser, how do you create new link types out of Org's core? This is not
> modular at all.

Most link types probably won't need custom link properties anyways. But could 
maybe let the parser know stuff by use of higher order functions? Or push for 
being important enough to be included into Org and allowed to be known by the 
parser.

> > (Off topic) I'm sorry to hear that you think I'm intentionally making
> > things fuzzy.
> 
> Not at all! My wording is terrible. When I wrote
> 
> Also, you sometimes seem to blur, on purpose, the difference between
> "attachment" and "file" links.
> 
> I meant something like:
> 
>It seems to me that your intent is to have "attachment" links
>transparently become "file" links to the user.
> 
> Hence my suggestion to use link abbreviations. There's nothing negative
> in it.

Ok, got it. Thanks for explaining. I'll admit it hasn't been totally clear to 
me what the best way forward is. After refreshing my understanding on links the 
conclusion I came to was that link abbreviations in its current form was not a 
fitting concept for attachment links. Attachments are pretty much similar to 
file links though, so even if 

Re: attachment: link type export to HTML invalid attach dir

2020-01-19 Thread Nicolas Goaziou
Gustav Wikström  writes:

> Ok, noted. To my defense I was thinking more in the terms of subtyping
> then hardcoding. Because we have multiple link types which try to
> share an interface. But the interface isn't perfect for all different
> kinds of links.

Then, I suggest to improve the interface.

> So it doesn't seem too farfetched that some of those link types would
> benefit from additional custom properties, specific for those types.
> =application= and =search-option= for example isn't universally
> applicable to all link types.

As a side note, :application is on its way out, i.e., "file+emacs" stuff
is in "org-compat.el".

Also, IIRC, "docview" links have "go to page" option, and they don't
touch the parser.

> What I'm trying to argue for here is: Maybe it's not that crazy after
> all to allow links to have additional properties in the parser based
> on its type? (While certainly still trying to avoid it if possible!)

If new link types may not function correctly without touching the
parser, how do you create new link types out of Org's core? This is not
modular at all.

> (Off topic) I'm sorry to hear that you think I'm intentionally making
> things fuzzy.

Not at all! My wording is terrible. When I wrote

Also, you sometimes seem to blur, on purpose, the difference between
"attachment" and "file" links.

I meant something like:

   It seems to me that your intent is to have "attachment" links
   transparently become "file" links to the user.

Hence my suggestion to use link abbreviations. There's nothing negative
in it.

> One can indeed use the buffer position to derive the part of the path
> that comes from the subtree. But leaving it at that puts more
> requirements on the user of the parsed link in order to use it.

And higher order functions in "org-attach" could take care of it. Note
that expanding a link in the parser means all attachment links are
always expanded, even if they are not used. There's a cost to consider.

Besides, parser focus on the contents of the buffer. I think it is out
of its scope to infer file names for abbreviation links. It's more the
job of the parts consuming the parsed data.

> There is no :raw-path available in the properties for a parsed link.
> If there were we surely wouldn't have this conversation and I'd be
> using that already! :) 

I meant :raw-link, sorry.

> I.e. I like this suggestion. I considered using the :raw-link
> property, but the way it's currently used by the parser (i.e.
> expanding abbreviations) stopped me.

Please take how link abbreviations are handled in the parser out of the
equation. I already stated this is not a good way to handle them. We
should probably expand them in a "temp-link" variable, and parse it,
while keeping original link in :raw-link (barring white spaces fixes).
That would not be perfect either, because we would still be inferring
stuff.

That's another topic, really. Let's just ignore it for now.

> So, we're discussing regarding attachment as either: an abbreviation
> or a proper separate link type? I'm not sure what other options we
> have? I personally can't categorize it as an abbreviation since that
> is handled by a separate piece of code with other specifications. I.e.
> attachment wouldn't fit well inside org-link-abbrev-alist. 

What makes you think it wouldn't fit well?

> Attachments should function in the same way as file links, with search
> options and all. But be limited to the current set of attached files.

This isn't incompatible with the above, AFAICT.

> That's basically the way it was before the patches to fix stardiviners
> issues. Except not functioning well enough. It would require quite
> some code in org-attach to replicate existing file links
> functionality. Mostly code duplication I presume.

I didn't read stardiviner issues. Would you mind summarizing them? Or,
at least could you explain what is not functioning well enough?

Anyhow, you may have missed the "let's spot the needs and improve these
tools" part. If current tools lead to code duplication, we can fix that.

> The raw-path option sounds better to my ears.

Except I was meaning :raw-link :)

> One improvement I can think of (outside of the discussion above) is to
> make it possible to pass the argument to org-link-open along to each
> separate link type specialization.

Isn't that the job of :follow when defining a new link type?

> That one has bugged me for some time now, when I've wanted to force
> opening attachment links in emacs but couldn't.

IMO, forcing users to open stuff in a specific, non-configurable, way is
a bad idea. Why would we know better than them?



RE: attachment: link type export to HTML invalid attach dir

2020-01-19 Thread Gustav Wikström
Hi Nicolas,

Thanks for your comments!

> -Original Message-
> From: Nicolas Goaziou 
> Sent: den 19 januari 2020 22:12
> To: Gustav Wikström 
> Cc: emacs-orgmode@gnu.org
> Subject: Re: attachment: link type export to HTML invalid attach dir
> 
> Hello,
> 
> Gustav Wikström  writes:
> 
> > Functionally it's in a good (and maintainable) state right now, in my
> > opinion. But I do understand that the contextual attribute added to
> > the parser may require some discussion.
> 
> This is not my main gripe here, although I'm not convinced that extra
> attribute is warranted. My concern is that you hard-code "attachment"
> type in the parser. I don't think this is the correct way to handle the
> situation.

Ok, noted. To my defense I was thinking more in the terms of subtyping then 
hardcoding. Because we have multiple link types which try to share an 
interface. But the interface isn't perfect for all different kinds of links. So 
it doesn't seem too farfetched that some of those link types would benefit from 
additional custom properties, specific for those types. =application= and 
=search-option= for example isn't universally applicable to all link types. And 
link abbreviations are completely hidden from the parser output, since even the 
raw-link is expanded. So maybe, in the quest for completion, link abbreviations 
should have custom properties as well? And allowing some kind of sub-typing 
might make the parsed output more easy to use, because the interpreter can 
infer based on type (which already exist as a link property) what extra 
properties to use.

What I'm trying to argue for here is: Maybe it's not that crazy after all to 
allow links to have additional properties in the parser based on its type? 
(While certainly still trying to avoid it if possible!)

> Some link types specifically handle files, e.g., "docview", yet do not
> need a special treatment in the parser. Note that "file" links have a good
> reason to be treated specially there, besides their obvious importance, as
> their type can be omitted. E.g., [[~/file.org]]. OTOH, I see no strong
> reason to handle "attachments" in here, since they behave like any other
> link type.
> 
> Worse, the parser is more or less the definition of the Org syntax.
> Therefore, including "attachment" in the parser is a signal meaning that
> in order to fully implement Org syntax, e.g., in another language, one
> need to support attachment links. Attachment links are undoubtedly useful,
> but they are not core, at all. So, I feel uneasy about leaking that type
> of link in the Element library.
> 
> Also, you sometimes seem to blur, on purpose, the difference between
> "attachment" and "file" links. If there should be no difference of
> treatment between them, as I already suggested, you may want to consider
> "attachment" as some functional link abbreviation. Then the "attachment"
> type doesn't really exist, much like the "bugzilla" link type from the
> manual.

(Off topic) I'm sorry to hear that you think I'm intentionally making things 
fuzzy. If it seems so, then please take it as a lack of clarity of thought or 
communication rather than intentional bad will. (/Off topic)

> In any case, we need a proper definition, a proper category too, for
> "attachment" links. Meanwhile, modifying the parser is just grasping at
> straws.
> 
> > I argue that the attachment folder is a part of the attachment link,
> > even though the information is found at a different location in the
> > document (i.e. as a property to nodes in the document hierarchy).
> > Parsing an attachment link would then be incomplete if that
> > information is discarded.
> 
> I argue that the buffer position of the attachment link and the path as
> written in the link are enough to fully expand the attachment file name.

One can indeed use the buffer position to derive the part of the path that 
comes from the subtree. But leaving it at that puts more requirements on the 
user of the parsed link in order to use it. Which takes us back to my first 
implementation of how the exporters would deal with the parsed link information 
(d70db54dbc).

> If I'm wrong, which could be, I probably didn't invest enough time in the
> Attach changes, then having the expanded form in :path and the initial
> form in :raw-path is enough.
> 
> > One option to adding an attribute could be to modify existing
> > properties by adding the attachment folder to, for example, the path
> > property. But that means to remove information about what was written
> > as path in the original link.
> 
> There is :raw-path for that purpose.

There is no :raw-path available in the properti

Re: attachment: link type export to HTML invalid attach dir

2020-01-19 Thread Nicolas Goaziou
Hello,

Gustav Wikström  writes:

> Functionally it's in a good (and maintainable) state right now, in my
> opinion. But I do understand that the contextual attribute added to
> the parser may require some discussion.

This is not my main gripe here, although I'm not convinced that extra
attribute is warranted. My concern is that you hard-code "attachment"
type in the parser. I don't think this is the correct way to handle the
situation.

Some link types specifically handle files, e.g., "docview", yet do not
need a special treatment in the parser. Note that "file" links have
a good reason to be treated specially there, besides their obvious
importance, as their type can be omitted. E.g., [[~/file.org]]. OTOH,
I see no strong reason to handle "attachments" in here, since they
behave like any other link type.

Worse, the parser is more or less the definition of the Org syntax.
Therefore, including "attachment" in the parser is a signal meaning that
in order to fully implement Org syntax, e.g., in another language, one
need to support attachment links. Attachment links are undoubtedly
useful, but they are not core, at all. So, I feel uneasy about leaking
that type of link in the Element library.

Also, you sometimes seem to blur, on purpose, the difference between
"attachment" and "file" links. If there should be no difference of
treatment between them, as I already suggested, you may want to consider
"attachment" as some functional link abbreviation. Then the "attachment"
type doesn't really exist, much like the "bugzilla" link type from the
manual.

In any case, we need a proper definition, a proper category too, for
"attachment" links. Meanwhile, modifying the parser is just grasping at
straws.

> I argue that the attachment folder is a part of the attachment link,
> even though the information is found at a different location in the
> document (i.e. as a property to nodes in the document hierarchy).
> Parsing an attachment link would then be incomplete if that
> information is discarded.

I argue that the buffer position of the attachment link and the path as
written in the link are enough to fully expand the attachment file name.

If I'm wrong, which could be, I probably didn't invest enough time in
the Attach changes, then having the expanded form in :path and the
initial form in :raw-path is enough.

> One option to adding an attribute could be to modify existing
> properties by adding the attachment folder to, for example, the path
> property. But that means to remove information about what was written
> as path in the original link.

There is :raw-path for that purpose.

> So I argue to keep path as the original path. But that means extra
> information is needed to also make it work in the filesystem. If we
> would translate an attachment link to a file link in ox.el that means
> we remove the option for exporters to decide for themselves what to do
> with the link. And I think the exporter should have that option. :)

Let's first think about what category of object an attachment link
belongs to. Then we can discuss about how to export it. 

Again, if "attachment" == "file", the exporters shouldn't treat them
differently. If "attachment" is a new link type, it should define its
own rules in its own library, namely "org-attach.el".

> Right now the ASCII exporter for example outputs attachment links as
> attachment:expanded_path instead of file:expanded_path. Since the link
> type actually is attachment. And for a solemnly textual export the
> exported information should be kept as close to source as possible. So
> either we explicitly and always say attachment-links *are* file-links
> in disguise (i.e. even change type in the parser), or we don't say
> that, and then don't say that all the way to the edge of the system.
> And let the uses of the link type decide themselves what to do. Which
> is what I propose.

As I explained above, your proposal is not crystal clear.

My gut feeling is that "attachment" links are just a regular link type,
that can be opened, and exported, like "file" links. They should live in
"org-attach.el", using the provided tools to define new link types, like
almost every other link types do. If those tools are not enough to
express all the subtleties of "attachment" links, then let's spot the
needs and improve those tools. That will benefit to every developer that
wants to implement a new link type, what creating another corner case in
the parser cannot do.


Regards,

-- 
Nicolas Goaziou



RE: attachment: link type export to HTML invalid attach dir

2020-01-18 Thread Gustav Wikström
Hi,

> -Original Message-
> From: Nicolas Goaziou 
> Sent: den 18 januari 2020 12:34
> To: Gustav Wikström 
> Cc: emacs-orgmode@gnu.org
> Subject: Re: attachment: link type export to HTML invalid attach dir
> 
> Hello,
> 
> Gustav Wikström  writes:
> 
> > Ok, so change pushed...
> 
> I'm sorry, but this is going too fast. We're discussing core design here
> (the parser), and I couldn't even answer your proposal. Let's at least
> reach an agreement on the change to make.

Yes, agreed, it was a bit fast. The final changes was done to not stop "in the 
middle" so to say, with something both you and me think is suboptimal. 
Functionally it's in a good (and maintainable) state right now, in my opinion. 
But I do understand that the contextual attribute added to the parser may 
require some discussion. If the decision is to not allow contextual attributes 
in the parser I'm prepared to revert and change again. No stress though.

Just to add a note about the trail of my thoughts regarding this... And why I 
thought the contextual attribute was a good option here:

I argue that the attachment folder is a part of the attachment link, even 
though the information is found at a different location in the document (i.e. 
as a property to nodes in the document hierarchy). Parsing an attachment link 
would then be incomplete if that information is discarded. One option to adding 
an attribute could be to modify existing properties by adding the attachment 
folder to, for example, the path property. But that means to remove information 
about what was written as path in the original link. So I argue to keep path as 
the original path. But that means extra information is needed to also make it 
work in the filesystem. If we would translate an attachment link to a file link 
in ox.el that means we remove the option for exporters to decide for themselves 
what to do with the link. And I think the exporter should have that option. :) 
Right now the ASCII exporter for example outputs attachment links as 
attachment:expanded_path instead of file:expanded_path. Since the link type 
actually is attachment. And for a solemnly textual export the exported 
information should be kept as close to source as possible. So either we 
explicitly and always say attachment-links *are* file-links in disguise (i.e. 
even change type in the parser), or we don't say that, and then don't say that 
all the way to the edge of the system. And let the uses of the link type decide 
themselves what to do. Which is what I propose. But with the addition of a bit 
of extra information contextual for the attachment links. 
 
Whoops long paragraph, sorry... I'm just trying to explain my current way of 
looking at this.

Regards
Gustav



Re: attachment: link type export to HTML invalid attach dir

2020-01-18 Thread Nicolas Goaziou
Hello,

Gustav Wikström  writes:

> Ok, so change pushed...

I'm sorry, but this is going too fast. We're discussing core design here
(the parser), and I couldn't even answer your proposal. Let's at least
reach an agreement on the change to make.

Regards,

-- 
Nicolas Goaziou



RE: attachment: link type export to HTML invalid attach dir

2020-01-17 Thread Gustav Wikström
Ok, so change pushed...

Where org-element-link-parser now sets one extra property only for attachment 
links. The exporters and org-link-open use this additional information when 
exporting and opening attachment links. Feature parity with file links should 
now be complete. Note that exporters outside of the Org mode repo need to be 
aware of the attachment link type if the path expansion is to be correct. They 
aren't translated in between into file-links. Not doing that translation is the 
proper think in my opinion. No magic hiding of the attachment link type. Who 
knows - maybe some exporters in the future need the link as is, without 
expansion!?. Making an exporter in the wild aware of attachment links can be 
done using org-element-property and the new property :attachment-path created 
by the parser only for attachment links.

Regards
Gustav

> -Original Message-
> From: Gustav Wikström
> Sent: den 17 januari 2020 19:36
> To: Nicolas Goaziou 
> Cc: emacs-orgmode@gnu.org
> Subject: RE: attachment: link type export to HTML invalid attach dir
> 
> Hi again,
> 
> > -Original Message-
> > From: Gustav Wikström
> > Sent: den 17 januari 2020 15:30
> > To: 'Nicolas Goaziou' 
> > Cc: emacs-orgmode@gnu.org
> > Subject: RE: attachment: link type export to HTML invalid attach dir
> >
> > Hi,
> >
> > [...]
> >
> > >
> > > It is true that Element library expands link abbreviations right
> > > before parsing a link, and an attachment is similar to a local link
> > abbreviation.
> > > This is not great because some information is lost in the
> > > process: interpreting the parse tree will not bring the abbreviation
> > > back, only its expanded form. Actually, `org-link-expand-abbrev' is
> > > called so that the parser knows what is the true type of the link,
> > > since abbreviations could expand to anything. OTOH, attachments can
> > > only expand to a "file" link, so the motivation for expansion
> > > doesn't
> > hold.
> >
> > Hmm, interesting... And are we sure the destructive behavior is
> > something we want to maintain? I for one would vote for the parsers
> > ability to provide information that can reconstruct the source... Is
> > it really worth the space saving in the parse tree to do that
> > destruction? I feel inclined here to add a :link and :raw-path
> > property to the output from the link parser for example. That would
> > allow those links that expand to be stored in the parse tree in both
> expanded and non-expanded form.
> >
> > Reasons against?
> 
> Hmm, I'm actually kind of going full circle here, back to think that the
> logic currently implemented is in its right place... Either that, or to
> just decorate the link in the parse-tree with some auxiliary information
> that can be specific for the type of link. For attachment links that
> auxiliary information would be attachment-path-prefix (or something
> shorter but possibly less clear). For abbreviated links possibly the
> auxiliary information would be it's unexpanded form. But I'm not sure of
> the necessity or need for that, except from allowing interpreters to
> reconstruct the original link.
> 
> >
> > [...]
> >
> 
> Regards
> Gustav


RE: attachment: link type export to HTML invalid attach dir

2020-01-17 Thread Gustav Wikström
Hi again,

> -Original Message-
> From: Gustav Wikström
> Sent: den 17 januari 2020 15:30
> To: 'Nicolas Goaziou' 
> Cc: emacs-orgmode@gnu.org
> Subject: RE: attachment: link type export to HTML invalid attach dir
> 
> Hi,
> 
> [...]
>
> >
> > It is true that Element library expands link abbreviations right
> > before parsing a link, and an attachment is similar to a local link
> abbreviation.
> > This is not great because some information is lost in the
> > process: interpreting the parse tree will not bring the abbreviation
> > back, only its expanded form. Actually, `org-link-expand-abbrev' is
> > called so that the parser knows what is the true type of the link,
> > since abbreviations could expand to anything. OTOH, attachments can
> > only expand to a "file" link, so the motivation for expansion doesn't
> hold.
> 
> Hmm, interesting... And are we sure the destructive behavior is something
> we want to maintain? I for one would vote for the parsers ability to
> provide information that can reconstruct the source... Is it really worth
> the space saving in the parse tree to do that destruction? I feel inclined
> here to add a :link and :raw-path property to the output from the link
> parser for example. That would allow those links that expand to be stored
> in the parse tree in both expanded and non-expanded form.
> 
> Reasons against?

Hmm, I'm actually kind of going full circle here, back to think that the logic 
currently implemented is in its right place... Either that, or to just decorate 
the link in the parse-tree with some auxiliary information that can be specific 
for the type of link. For attachment links that auxiliary information would be 
attachment-path-prefix (or something shorter but possibly less clear). For 
abbreviated links possibly the auxiliary information would be it's unexpanded 
form. But I'm not sure of the necessity or need for that, except from allowing 
interpreters to reconstruct the original link.

> 
> [...]
> 

Regards
Gustav


RE: attachment: link type export to HTML invalid attach dir

2020-01-17 Thread Gustav Wikström
Hi,

> -Original Message-
> From: Nicolas Goaziou 
> Sent: den 17 januari 2020 01:40
> To: Gustav Wikström 
> Cc: emacs-orgmode@gnu.org
> Subject: Re: attachment: link type export to HTML invalid attach dir
> 
> Gustav Wikström  writes:
> 
> > Hardcoding the translation of attachment-links into file-links in an
> > in-between layer (ox.el - that is somewhat complicated as well) is not
> > transparent and I think best to avoid. Even if an attachment link is
> > /very/ similar to a file link it may be best still to treat them for
> > what they are. If some export back-ends out in the wild don't work
> > with attachment-links today then so be it. But let's at least make it
> > easy to fix! So I'll try to remove the hard coding of org-attach
> > invocation and instead make the attachment-path when parsed by
> > org-element return a path that is an actual file-system path out of
> > the box. I'll see what I figure out in terms of code I suppose...!
> >
> > What do you say?
> 
> It is true that Element library expands link abbreviations right before
> parsing a link, and an attachment is similar to a local link abbreviation.
> This is not great because some information is lost in the
> process: interpreting the parse tree will not bring the abbreviation back,
> only its expanded form. Actually, `org-link-expand-abbrev' is called so
> that the parser knows what is the true type of the link, since
> abbreviations could expand to anything. OTOH, attachments can only expand
> to a "file" link, so the motivation for expansion doesn't hold.

Hmm, interesting... And are we sure the destructive behavior is something we 
want to maintain? I for one would vote for the parsers ability to provide 
information that can reconstruct the source... Is it really worth the space 
saving in the parse tree to do that destruction? I feel inclined here to add a 
:link and :raw-path property to the output from the link parser for example. 
That would allow those links that expand to be stored in the parse tree in both 
expanded and non-expanded form. 

Reasons against?  

> Besides, there's no good reason to store redundant, or even remote,
> information in the parse tree. The parser needs to be as low level as
> possible.

Yes, this is a reason against... But big enough? I humbly question that a bit. 
Also, would it be feasible to store the original "raw" form only if it differs 
from the expanded form? It may complicate the interpreters work though... They 
would have to have a bit more insight into when the original form will be 
needed. Not sure of the better design decision here. 

> Moreover, the issue encountered here is specific to export, so it makes
> sense to solve at the export level, i.e., in "ox.el". This is not trivial
> though, as it needs to be done very early, much link footnotes.
> Indeed, since the meaning of the link is position dependent, including
> parts of a document, or executing arbitrary code can mess it up. IOW
> expansion should happen in `org-export--prepare-file-contents', i.e.,
> within every included piece of text, and in `org-export-as', somewhere
> after `org-export-expand-include-keyword' call, but before Babel code
> execution.

It's actually not only specific to exporting. Opening attachment links show a 
similar issue (org-link-open). Because the current way to open files and 
attachments differ. And it shouldn't have to! That as well would be solved if 
the link parser would take care of the path expansion. One example of the 
difference today is that attachment links cannot abide to any prefix argument. 
As a user I would expect the prefix arguments that work for files to also work 
for attachments. Something I think we can fix here, by making the parser do a 
little bit more work. I'd be happy to apply a patch. 

Kind regards
Gustav



Re: attachment: link type export to HTML invalid attach dir

2020-01-16 Thread Nicolas Goaziou
Gustav Wikström  writes:

> Hardcoding the translation of attachment-links into file-links in an
> in-between layer (ox.el - that is somewhat complicated as well) is not
> transparent and I think best to avoid. Even if an attachment link is
> /very/ similar to a file link it may be best still to treat them for
> what they are. If some export back-ends out in the wild don't work
> with attachment-links today then so be it. But let's at least make it
> easy to fix! So I'll try to remove the hard coding of org-attach
> invocation and instead make the attachment-path when parsed by
> org-element return a path that is an actual file-system path out of
> the box. I'll see what I figure out in terms of code I suppose...!
>
> What do you say?

It is true that Element library expands link abbreviations right before
parsing a link, and an attachment is similar to a local link
abbreviation. This is not great because some information is lost in the
process: interpreting the parse tree will not bring the abbreviation
back, only its expanded form. Actually, `org-link-expand-abbrev' is
called so that the parser knows what is the true type of the link, since
abbreviations could expand to anything. OTOH, attachments can only
expand to a "file" link, so the motivation for expansion doesn't hold.

Besides, there's no good reason to store redundant, or even remote,
information in the parse tree. The parser needs to be as low level as
possible.

Moreover, the issue encountered here is specific to export, so it makes
sense to solve at the export level, i.e., in "ox.el". This is not
trivial though, as it needs to be done very early, much link footnotes.
Indeed, since the meaning of the link is position dependent, including
parts of a document, or executing arbitrary code can mess it up. IOW
expansion should happen in `org-export--prepare-file-contents', i.e.,
within every included piece of text, and in `org-export-as', somewhere
after `org-export-expand-include-keyword' call, but before Babel code
execution.

HTH,

Regards,



RE: attachment: link type export to HTML invalid attach dir

2020-01-16 Thread Gustav Wikström
Hi again,

> -Original Message-
> From: Gustav Wikström
> Sent: den 16 januari 2020 22:42
> To: Nicolas Goaziou 
> Cc: numbch...@gmail.com; emacs-orgmode@gnu.org
> Subject: RE: attachment: link type export to HTML invalid attach dir
> 
> Hi Nicolas,
> 
> > -Original Message-
> > From: Nicolas Goaziou 
> > Sent: den 16 januari 2020 14:18
> > To: Gustav Wikström 
> > Cc: numbch...@gmail.com; emacs-orgmode@gnu.org
> > Subject: Re: attachment: link type export to HTML invalid attach dir
> >
> > Hello,
> >
> > Gustav Wikström  writes:
> >
> > > Ah yes. Found the culprit for this issue. Hopefully the last one.
> > > The exporter doesn't actually move the point in the buffer during
> > > the export. So org-attach-expand tried to expand from the first
> > > character in the buffer. This should be fixed from a few minutes ago.
> >
> > I'm not sure hard-coding attachment links in exporters in the best way
> > forward. For example, exporters in the wild may not cope with them
> > before a long time, if ever. There is some code duplication, too.
> 
> Yes indeed, duplicated functionality for all export backends as it stands.
> 
> >
> > If attachments links are similar to file links from an export point of
> > view, then I suggest to add a phase in ox.el to expand the former into
> > the latter, before even using export back-ends. This way, there is no
> > change required in the exporters, shipped in or not.
> 
> Yeah, I do think attachment links should be treated as file links when
> exported. And I like this suggestion, although that means I probably have
> to dig into the ox.el code. Not an easy task. I suspect you'd guide me to
> adding logic inside org-export-as for this. I'll have a look starting from
> there. But wouldn't mind some further insights here!

After thinking a while I'm leaning towards thinking this should be handled 
already in the element link parser and interpreter. Need a bit more metadata 
for that though, to be able to deconstruct and reconstruct the link properly 
while still providing the correct paths.

Hardcoding the translation of attachment-links into file-links in an in-between 
layer (ox.el - that is somewhat complicated as well) is not transparent and I 
think best to avoid. Even if an attachment link is /very/ similar to a file 
link it may be best still to treat them for what they are. If some export 
back-ends out in the wild don't work with attachment-links today then so be it. 
But let's at least make it easy to fix! So I'll try to remove the hard coding 
of org-attach invocation and instead make the attachment-path when parsed by 
org-element return a path that is an actual file-system path out of the box. 
I'll see what I figure out in terms of code I suppose...! 

What do you say? 

> 
> Regards
> Gustav
> 
> >
> > Regards,
> >
> > --
> > Nicolas Goaziou


RE: attachment: link type export to HTML invalid attach dir

2020-01-16 Thread Gustav Wikström
Hi Nicolas,

> -Original Message-
> From: Nicolas Goaziou 
> Sent: den 16 januari 2020 14:18
> To: Gustav Wikström 
> Cc: numbch...@gmail.com; emacs-orgmode@gnu.org
> Subject: Re: attachment: link type export to HTML invalid attach dir
> 
> Hello,
> 
> Gustav Wikström  writes:
> 
> > Ah yes. Found the culprit for this issue. Hopefully the last one. The
> > exporter doesn't actually move the point in the buffer during the
> > export. So org-attach-expand tried to expand from the first character
> > in the buffer. This should be fixed from a few minutes ago.
> 
> I'm not sure hard-coding attachment links in exporters in the best way
> forward. For example, exporters in the wild may not cope with them before
> a long time, if ever. There is some code duplication, too.

Yes indeed, duplicated functionality for all export backends as it stands.

> 
> If attachments links are similar to file links from an export point of
> view, then I suggest to add a phase in ox.el to expand the former into the
> latter, before even using export back-ends. This way, there is no change
> required in the exporters, shipped in or not.

Yeah, I do think attachment links should be treated as file links when 
exported. And I like this suggestion, although that means I probably have to 
dig into the ox.el code. Not an easy task. I suspect you'd guide me to adding 
logic inside org-export-as for this. I'll have a look starting from there. But 
wouldn't mind some further insights here!

Regards
Gustav

> 
> Regards,
> 
> --
> Nicolas Goaziou


Re: attachment: link type export to HTML invalid attach dir

2020-01-16 Thread Nicolas Goaziou
Hello,

Gustav Wikström  writes:

> Ah yes. Found the culprit for this issue. Hopefully the last one. The
> exporter doesn't actually move the point in the buffer during the
> export. So org-attach-expand tried to expand from the first character
> in the buffer. This should be fixed from a few minutes ago.

I'm not sure hard-coding attachment links in exporters in the best way
forward. For example, exporters in the wild may not cope with them
before a long time, if ever. There is some code duplication, too.

If attachments links are similar to file links from an export point of
view, then I suggest to add a phase in ox.el to expand the former into
the latter, before even using export back-ends. This way, there is no
change required in the exporters, shipped in or not.

Regards,

-- 
Nicolas Goaziou



Re: attachment: link type export to HTML invalid attach dir

2020-01-16 Thread stardiviner


Gustav Wikström  writes:

> Hi,
>
>> -Original Message-
>> From: stardiviner 
>> Sent: den 15 januari 2020 06:53
>> To: Gustav Wikström 
>> Cc: numbch...@gmail.com; emacs-orgmode@gnu.org
>> Subject: Re: attachment: link type export to HTML invalid attach dir
>>
>> [...]
>>
>> I checked out the latest commit "0ac6a9e1f", The ~~ tag is solved.
>> But the ~(org-attach-dir)~ still not work. The exported ~~
>> inline image still does not contains the attach directory.
>

> Ah yes. Found the culprit for this issue. Hopefully the last one. The exporter
doesn't actually move the point in the buffer during the export. So
org-attach-expand tried to expand from the first character in the buffer. This
should be fixed from a few minutes ago.

Aha, wow!!! thanks for great patch. It worked now. Thanks really. :)

>
> Crossing fingers for this to work properly now!
>
> Regards
> Gustav
>  

-- 
[ stardiviner ]
   I try to make every word tell the meaning what I want to express.

   Blog: https://stardiviner.github.io/
   IRC(freenode): stardiviner, Matrix: stardiviner
   GPG: F09F650D7D674819892591401B5DF1C95AE89AC3
  



RE: attachment: link type export to HTML invalid attach dir

2020-01-15 Thread Gustav Wikström
Hi,

> -Original Message-
> From: stardiviner 
> Sent: den 15 januari 2020 06:53
> To: Gustav Wikström 
> Cc: numbch...@gmail.com; emacs-orgmode@gnu.org
> Subject: Re: attachment: link type export to HTML invalid attach dir
>
> [...]
>
> I checked out the latest commit "0ac6a9e1f", The ~~ tag is solved.
> But the ~(org-attach-dir)~ still not work. The exported ~~
> inline image still does not contains the attach directory.

Ah yes. Found the culprit for this issue. Hopefully the last one. The exporter 
doesn't actually move the point in the buffer during the export. So 
org-attach-expand tried to expand from the first character in the buffer. This 
should be fixed from a few minutes ago.

Crossing fingers for this to work properly now!

Regards
Gustav
 


Re: attachment: link type export to HTML invalid attach dir

2020-01-14 Thread stardiviner


Gustav Wikström  writes:

> Hi again,
>
>> -Original Message-
>> From: Emacs-orgmode  On
>> Behalf Of stardiviner
>> Sent: den 14 januari 2020 06:04
>> To: Gustav Wikström 
>> Cc: emacs-orgmode@gnu.org
>> Subject: Re: attachment: link type export to HTML invalid attach dir
>> 
>> [...]
>>
>> Many thanks for really quick patching.
>> 
>> I tested out the new patch, still does not work.
>> 
>> #+begin_src org
>> ,** Strings are not Lists, but Anyway…
>>:PROPERTIES:
>>:ID:   2fd354f3-ac7a-499d-9fe4-a76626bbdb38
>>:END:
>> 
>> In Calva Paredit, strings are treated in much the same way as lists are.
>> Here’s an example showing Slurp and Barf, Forward/Backward List, and Grow
>> Selection.
>> 
>> [[attachment:string-as-list.gif]]
>> 
>> #+end_src
>> 
>> The upper org content is exported as this (HTML page):
>> 
>> #+begin_example
>> Strings are not Lists, but Anyway…
>> 
>> In Calva Paredit, strings are treated in much the same way as lists are.
>> Here’s an example showing Slurp and Barf, Forward/Backward List, and Grow
>> Selection.
>> 
>> file:///home/stardiviner/Org/Wiki/Computer
>> Technology/Programming/Emacs/Data/Emacs Packages/string-as-list.gif
>> #+end_example
>> 
>> You can see:
>> 
>> 1. the link still does not contains the attach directory from ~(org-
>> attach-dir)~.
>> 2. image links are not exported as inline image displayed with ~> src="...">~.
>
> Ah, you're right. I missed a few things. Should be fixed now in master. Care 
> to verify from your end again?
>
> Thanks
> Gustav

I checked out the latest commit "0ac6a9e1f", The ~~ tag is solved. But the
~(org-attach-dir)~ still not work. The exported ~~ inline image 
still
does not contains the attach directory.

-- 
[ stardiviner ]
   I try to make every word tell the meaning what I want to express.

   Blog: https://stardiviner.github.io/
   IRC(freenode): stardiviner, Matrix: stardiviner
   GPG: F09F650D7D674819892591401B5DF1C95AE89AC3
  



RE: attachment: link type export to HTML invalid attach dir

2020-01-14 Thread Gustav Wikström
Hi again,

> -Original Message-
> From: Emacs-orgmode  On
> Behalf Of stardiviner
> Sent: den 14 januari 2020 06:04
> To: Gustav Wikström 
> Cc: emacs-orgmode@gnu.org
> Subject: Re: attachment: link type export to HTML invalid attach dir
> 
> [...]
>
> Many thanks for really quick patching.
> 
> I tested out the new patch, still does not work.
> 
> #+begin_src org
> ,** Strings are not Lists, but Anyway…
>:PROPERTIES:
>:ID:   2fd354f3-ac7a-499d-9fe4-a76626bbdb38
>:END:
> 
> In Calva Paredit, strings are treated in much the same way as lists are.
> Here’s an example showing Slurp and Barf, Forward/Backward List, and Grow
> Selection.
> 
> [[attachment:string-as-list.gif]]
> 
> #+end_src
> 
> The upper org content is exported as this (HTML page):
> 
> #+begin_example
> Strings are not Lists, but Anyway…
> 
> In Calva Paredit, strings are treated in much the same way as lists are.
> Here’s an example showing Slurp and Barf, Forward/Backward List, and Grow
> Selection.
> 
> file:///home/stardiviner/Org/Wiki/Computer
> Technology/Programming/Emacs/Data/Emacs Packages/string-as-list.gif
> #+end_example
> 
> You can see:
> 
> 1. the link still does not contains the attach directory from ~(org-
> attach-dir)~.
> 2. image links are not exported as inline image displayed with ~ src="...">~.

Ah, you're right. I missed a few things. Should be fixed now in master. Care to 
verify from your end again?

Thanks
Gustav


Re: attachment: link type export to HTML invalid attach dir

2020-01-13 Thread stardiviner


Gustav Wikström  writes:

> Hi!
>
> Thanks for reporting this!
>
> The code is updated on the master branch to make the exporters aware of how 
> to deal with attachment links. Commit d70db54db for the curious. Basically, 
> attachment links are expanded into file-links by the exporters now, before 
> further processing into links in the respective markup language.
>
> Regards
> Gustav

Many thanks for really quick patching.

I tested out the new patch, still does not work.

#+begin_src org
,** Strings are not Lists, but Anyway…
   :PROPERTIES:
   :ID:   2fd354f3-ac7a-499d-9fe4-a76626bbdb38
   :END:

In Calva Paredit, strings are treated in much the same way as lists are. Here’s
an example showing Slurp and Barf, Forward/Backward List, and Grow Selection.

[[attachment:string-as-list.gif]]

#+end_src

The upper org content is exported as this (HTML page):

#+begin_example
Strings are not Lists, but Anyway…

In Calva Paredit, strings are treated in much the same way as lists are. Here’s 
an example showing Slurp and Barf, Forward/Backward List, and Grow Selection.

file:///home/stardiviner/Org/Wiki/Computer 
Technology/Programming/Emacs/Data/Emacs Packages/string-as-list.gif
#+end_example

You can see:

1. the link still does not contains the attach directory from 
~(org-attach-dir)~.
2. image links are not exported as inline image displayed with ~~.

-- 
[ stardiviner ]
   I try to make every word tell the meaning what I want to express.

   Blog: https://stardiviner.github.io/
   IRC(freenode): stardiviner, Matrix: stardiviner
   GPG: F09F650D7D674819892591401B5DF1C95AE89AC3
  



Re: attachment: link type export to HTML invalid attach dir

2020-01-13 Thread Gustav Wikström
Hi!

Thanks for reporting this!

The code is updated on the master branch to make the exporters aware of how to 
deal with attachment links. Commit d70db54db for the curious. Basically, 
attachment links are expanded into file-links by the exporters now, before 
further processing into links in the respective markup language.

Regards
Gustav


From: Emacs-orgmode  on behalf of 
stardiviner 
Sent: Monday, January 13, 2020 13:24
To: emacs-orgmode@gnu.org
Subject: Re: attachment: link type export to HTML invalid attach dir


When I export with =[C-c C-e h o]=, I found attached inline image links are not
displayed in HTML page.

Here is Org content:

#+begin_src org
,** Strings are not Lists, but Anyway…
   :PROPERTIES:
   :ID:   2fd354f3-ac7a-499d-9fe4-a76626bbdb38
   :END:

In Calva Paredit, strings are treated in much the same way as lists are. Here’s
an example showing Slurp and Barf, Forward/Backward List, and Grow Selection.

[[attachment:string-as-list.gif]]

#+end_src

The email attachment contains the screenshot of HTML page.

I used Edebug on functions, track down to the error function ~(org-attach-dir)~,
it returns ~nil~ when in exporting to HTML, but when I evaluate 
~(org-attach-dir)~
interactively under the headline, it works correctly.

--
[ stardiviner ]
   I try to make every word tell the meaning what I want to express.

   Blog: https://stardiviner.github.io/
   IRC(freenode): stardiviner, Matrix: stardiviner
   GPG: F09F650D7D674819892591401B5DF1C95AE89AC3




Re: attachment: link type export to HTML invalid attach dir

2020-01-13 Thread stardiviner

When I export with =[C-c C-e h o]=, I found attached inline image links are not
displayed in HTML page.

Here is Org content:

#+begin_src org
,** Strings are not Lists, but Anyway…
   :PROPERTIES:
   :ID:   2fd354f3-ac7a-499d-9fe4-a76626bbdb38
   :END:

In Calva Paredit, strings are treated in much the same way as lists are. Here’s
an example showing Slurp and Barf, Forward/Backward List, and Grow Selection.

[[attachment:string-as-list.gif]]

#+end_src

The email attachment contains the screenshot of HTML page.

I used Edebug on functions, track down to the error function ~(org-attach-dir)~,
it returns ~nil~ when in exporting to HTML, but when I evaluate 
~(org-attach-dir)~
interactively under the headline, it works correctly.

-- 
[ stardiviner ]
   I try to make every word tell the meaning what I want to express.

   Blog: https://stardiviner.github.io/
   IRC(freenode): stardiviner, Matrix: stardiviner
   GPG: F09F650D7D674819892591401B5DF1C95AE89AC3