Re: [NTG-context] attachments working again (issue with /EmbeddedFiles)
On 2/14/20 3:43 AM, Rik Kabel wrote: > [...] > The further I look into bookmark, annotation, and attachment handling by > a number of viewers, the more confusing things appear. Hi Rik, this is why following the prescription (the PDF spec) is better in order to check the standard than gathering information in different viewers. > And Acrobat still shows attachments twice under some circumstances, > with the helpful flyover for one entry giving the location as the > associated page number, and for the other giving it as the > attachments tab. That practice is explictly discouraged by the PDF Association, citing the behaviour of some viewers (but not naming Acrobat). See https://www.pdfa.org/wp-content/uploads/2018/10/PDF20_AN002-AF.pdf#search=usability. > What is more troublesome on the ConTeXt side is the validation issue > with the appearance dictionary for visible attachments. This is my (educated?) guess: appearance dictionaries are required to print annotation icons. This isn’t especially directed to attachments, but to all annotation types (excepting popups and links). > The difference between validators when looking at the hidden attachment > in my example hinges on the handling of the attachment mime type. I have > filed a defect report with VeraPDF, which appears to be rejecting valid > mime type that includes a charset clause. If this is a bug, they should fix it. They may have missed a valid MIME type. Pablo -- http://www.ousia.tk ___ If your question is of interest to others as well, please add an entry to the Wiki! maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context webpage : http://www.pragma-ade.nl / http://context.aanhet.net archive : https://bitbucket.org/phg/context-mirror/commits/ wiki : http://contextgarden.net ___
Re: [NTG-context] attachments working again (issue with /EmbeddedFiles)
On 2/13/2020 03:56, Hans Hagen wrote: On 2/13/2020 12:45 AM, Rik Kabel wrote: Perhaps there is something in the difference between annotations and attachments that results in different treatments by the various viewers. it has always been that messy; there's also the annotation symbol treatment that can differ; it doesn't help that acrobat changed behaviour a couple of times i think this is why the whole attachment handling was as it is: some comprimise that kind of made them show up ok on the average; but validatin spoiled this and now we're stick with these incompatible viewer behaviour .. not much we can do about it i fear (For instance, i remember that at some point acrobat showed attachments twice in some lists. Often these features ended up in specs without viewer support yet so probably not well tested for interferences either.) Hans The further I look into bookmark, annotation, and attachment handling by a number of viewers, the more confusing things appear. And Acrobat still shows attachments twice under some circumstances, with the helpful flyover for one entry giving the location as the associated page number, and for the other giving it as the attachments tab. What is more troublesome on the ConTeXt side is the validation issue with the appearance dictionary for visible attachments. The difference between validators when looking at the hidden attachment in my example hinges on the handling of the attachment mime type. I have filed a defect report with VeraPDF, which appears to be rejecting valid mime type that includes a charset clause. -- Rik ___ If your question is of interest to others as well, please add an entry to the Wiki! maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context webpage : http://www.pragma-ade.nl / http://context.aanhet.net archive : https://bitbucket.org/phg/context-mirror/commits/ wiki : http://contextgarden.net ___
Re: [NTG-context] attachments working again (issue with /EmbeddedFiles)
On 2/13/2020 12:45 AM, Rik Kabel wrote: Perhaps there is something in the difference between annotations and attachments that results in different treatments by the various viewers. it has always been that messy; there's also the annotation symbol treatment that can differ; it doesn't help that acrobat changed behaviour a couple of times i think this is why the whole attachment handling was as it is: some comprimise that kind of made them show up ok on the average; but validatin spoiled this and now we're stick with these incompatible viewer behaviour .. not much we can do about it i fear (For instance, i remember that at some point acrobat showed attachments twice in some lists. Often these features ended up in specs without viewer support yet so probably not well tested for interferences either.) Hans - Hans Hagen | PRAGMA ADE Ridderstraat 27 | 8061 GH Hasselt | The Netherlands tel: 038 477 53 69 | www.pragma-ade.nl | www.pragma-pod.nl - ___ If your question is of interest to others as well, please add an entry to the Wiki! maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context webpage : http://www.pragma-ade.nl / http://context.aanhet.net archive : https://bitbucket.org/phg/context-mirror/commits/ wiki : http://contextgarden.net ___
Re: [NTG-context] attachments working again (issue with /EmbeddedFiles)
Following up on my viewer observations, I downloaded Evince for Windows version 2.32 and see something that might be meaningful. Evince has a side-panel with a drop-down for selecting Thumbnails, Index, Attachments, Layers, or Annotations. Those that are not present in the document are greyed out. For the document with the pushpin (which Evince does display) the available choices are Thumbnails and Annotations. The Annotations selection shows the attached file on a tree under Page 1. For the document with the hidden attachment, both the Attachments and the Annotations choices are available. When Annotations is selected, the message "/Document contains no annotations/" is shown. When Attachments is selected, the attachment is properly listed. Perhaps there is something in the difference between annotations and attachments that results in different treatments by the various viewers. -- Rik ___ If your question is of interest to others as well, please add an entry to the Wiki! maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context webpage : http://www.pragma-ade.nl / http://context.aanhet.net archive : https://bitbucket.org/phg/context-mirror/commits/ wiki : http://contextgarden.net ___
Re: [NTG-context] attachments working again (issue with /EmbeddedFiles)
On 2/12/2020 14:20, Pablo Rodriguez wrote:
On 2/8/20 10:13 PM, Rik Kabel wrote:
Not quite ready yet. When the hidden method is used, it passes the
online validator[1], but fails VeraPDF[2] for clause 6.8-1. When
visible, it fails the online validator stating that the appearance
dictionary doesn't contain an entry, and fails VeraPDF for clauses 6.8-1
and 6.3.3-1.
Hi Rik,
as for 6.8-1, did you set MIME type of the attached file (such as in
"type={application/pdf}")?
Oddly, the bookmarks do not appear either way under MkIV, and they do
not appear when the attachment is visible under LMTX. The bookmarks do
appear with a hidden attachment under LMTX.
As far as I understand, only LMTX will receive regular updates now. (But
Hans knows better 😅.)
I get both visible attachments and bookmarks in LMTX (latest and
previous latest [“latest” is the new beta]).
Could you provide a minimal sample that shows the issue?
Pablo
--
http://www.ousia.tk
Hello Pablo (and everyone else),
Here is my test file. You can use "context --mode=hidden file.tex" to
hide the pushpin, or leave out the mode to expose it.
%% Test file for pdf3a compliance.
%%
%% So far, this fails when the attachment is included. Other than the
%%handling of attachments, ConTeXt does a good job with the various
%%PDF/A variants.
%%
%% Testing done locally with the VeraPDF validator (most recently with
%%version 1.14.8) and online with the 3-Heights validator found at
%%https://www.pdf-online.com/osa/validate.aspx (current 2020-01-20).
%%
%% This is not a minimal example.
\definemode [hidden][keep]
\setuplanguage [en]
\setuppapersize [A6,landscape]
\setuppagenumbering [location=]
\setuptagging[state=start]
\setupinteraction[state=start]
%setupbackend[format={pdf/a-3u},
\setupbackend[format={pdf/a-3a},
level=0,
profile={default_gray.icc},
intent={ISO coated v2 300\letterpercent\space (ECI)}]
\setupstructure [state=start,
method=auto]
\setupcolors [pagecolormodel=auto,
start=yes,
rgb=no,
cmyk=no]
\setupbodyfont [palatino,18pt]
\starttext
This document was produced using version \contextversion\ of
\ConTeXt\ \doifelse{\texenginename}{luametatex}{LMTX}{\contextmark}
with the version \texengineversion\ of \texenginename.
\attachment[file=./\jobname.tex,
method={\doifmodeelse{hidden}{hidden}{normal}},
type={text/plain;charset=UTF-8}]
The source code for this document is attached as a plain text file. It
should show up in the pdf bookmarks for the document\doifmodeelse
{hidden}{}{ and through the icon at the start of this paragraph}.
\stoptext
I have tested with LMTX 2020.02.11 16:41.
*Mime type*
* You can see that the mime type is set in the ConTeXt file and it
appears in the resulting pdf, although not knowing enough about the
pdf file structure I do not know if it appears correctly.
*Validation*
* With the pin hidden, VeraPDF 1.14.8 complains about two occurrences
of clause 6.8 test 1 failure. With the pin exposed, it complains
about one occurrence of clause 6.8 test 1 failure and one occurrence
of clause 6.3.3 test 1 failure (appearance dictionary).
* With the pin hidden, pdf-online.com declares the valid. With the
pin exposed, pdf-online.com fails it because "[t]he appearance
dictionary doesn't contain an entry." They are aware of the
difference between their report and that of VeraPDF.
*Viewing*
* Acrobat Reader DC shows the pushpin, changes the pointer, and
displays flyover text when appropriate and shows the bookmark for
both variants.
* SumatraPDF shows the pushpin, changes the pointer, and displays
flyover text when appropriate. It does not show the bookmark when
the pushpin is visible, but does show it when it is hidden.
* MuPDF does not show the pushpin, instead leaving empty space, when
it should be visible. It does not show bookmarks or any other
indication that there is an attachment for either variant. (MuPDF
can show bookmarks for some other documents.)
* Firefox 73 does not show the pushpin when it should be visible, but
does change the pointer and show the attachment name as a flyover.
Firefox 73 does show the bookmark for both variants.
* Chrome 80 does not show the pushpin when it should be visible, but
does show the attachment name as a flyover. Chrome 80 does not show
the bookmark for either variant. (Chrome 80 does show bookmarks for
some other PDF document.)
* The three viewers listed here that do not show the pushpin do
reserve the appropriate space – changing the symbol to the
Paperclip demonstrates t
Re: [NTG-context] attachments working again (issue with /EmbeddedFiles)
On 2/8/20 10:13 PM, Rik Kabel wrote:
>
>
> Not quite ready yet. When the hidden method is used, it passes the
> online validator[1], but fails VeraPDF[2] for clause 6.8-1. When
> visible, it fails the online validator stating that the appearance
> dictionary doesn't contain an entry, and fails VeraPDF for clauses 6.8-1
> and 6.3.3-1.
Hi Rik,
as for 6.8-1, did you set MIME type of the attached file (such as in
"type={application/pdf}")?
> Oddly, the bookmarks do not appear either way under MkIV, and they do
> not appear when the attachment is visible under LMTX. The bookmarks do
> appear with a hidden attachment under LMTX.
As far as I understand, only LMTX will receive regular updates now. (But
Hans knows better 😅.)
I get both visible attachments and bookmarks in LMTX (latest and
previous latest [“latest” is the new beta]).
Could you provide a minimal sample that shows the issue?
Pablo
--
http://www.ousia.tk
___
If your question is of interest to others as well, please add an entry to the
Wiki!
maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context
webpage : http://www.pragma-ade.nl / http://context.aanhet.net
archive : https://bitbucket.org/phg/context-mirror/commits/
wiki : http://contextgarden.net
___
Re: [NTG-context] attachments working again (issue with /EmbeddedFiles)
On 2/8/2020 12:32, Pablo Rodriguez wrote: On 2/8/20 1:45 PM, Peter Rolf wrote: [...] I attached the current code, so you and Rik are able to test it if you want (creating a new format is required). Best contact me off-list if you run into new problems. Hi Peter, the new version is fine for me. From what I see, /F is now set to 4 (print). Many thanks for your help, Pablo -- http://www.ousia.tk Not quite ready yet. When the hidden method is used, it passes the online validator[1], but fails VeraPDF[2] for clause 6.8-1. When visible, it fails the online validator stating that the appearance dictionary doesn't contain an entry, and fails VeraPDF for clauses 6.8-1 and 6.3.3-1. Oddly, the bookmarks do not appear either way under MkIV, and they do not appear when the attachment is visible under LMTX. The bookmarks do appear with a hidden attachment under LMTX. The results are generally better than before this updated lpdf-wid.lua. [1] https://www.pdf-online.com/osa/validate.aspx [2] Version 1.14.8, which I believe is the current version -- Rik ___ If your question is of interest to others as well, please add an entry to the Wiki! maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context webpage : http://www.pragma-ade.nl / http://context.aanhet.net archive : https://bitbucket.org/phg/context-mirror/commits/ wiki : http://contextgarden.net ___
Re: [NTG-context] attachments working again (issue with /EmbeddedFiles)
On 2/8/20 1:45 PM, Peter Rolf wrote: > [...] > > I attached the current code, so you and Rik are able to test it if you > want (creating a new format is required). Best contact me off-list if > you run into new problems. Hi Peter, the new version is fine for me. From what I see, /F is now set to 4 (print). Many thanks for your help, Pablo -- http://www.ousia.tk ___ If your question is of interest to others as well, please add an entry to the Wiki! maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context webpage : http://www.pragma-ade.nl / http://context.aanhet.net archive : https://bitbucket.org/phg/context-mirror/commits/ wiki : http://contextgarden.net ___
Re: [NTG-context] attachments working again (issue with /EmbeddedFiles)
Am 07.02.2020 um 22:37 schrieb Hans Hagen:
> On 2/7/2020 9:55 PM, Pablo Rodriguez wrote:
>> On 2/7/20 9:19 PM, Rik Kabel wrote:
>>>
>>> On 2/7/2020 14:46, Pablo Rodriguez wrote:
many thanks for having fixed the issues with attachments (in latest
beta
from 2020.02.07 18:36). I haven’t tested attachments with PDF/A-3a.
>>> PDF/A-3a attachments still fail validation with the same issues.
>>
>> Hi Rik,
>>
>> it has way less issues, at least using verapdf-1.5.18 and the following
>> sample:
>>
>> \setuptagging[state=start]
>>
>> \setupbodyfont[30pt]
>>
>> \setupbackend
>> [format=PDF/A-3a,
>> intent=sRGB IEC61966-2.1,
>> profile={sRGB.icc,default_gray.icc},
>> level=0]
>>
>> \setupcolors[pagecolormodel=auto]
>>
>> \setupinteraction[state=start]
>> \starttext
>> \startTEXpage[offset=1em]
>> an attachment\attachment[file=xml-mkiv.pdf,
>> type={application/pdf}]
>> \stopTEXpage
>> \stoptext
>>
>> With "method=hidden" (no attachment annotation) it has no issue.
>>
>> Both issues are related to annotations in general:
>>
>> - If the flag annotation is present (/F key), it should be set to print
>> and disable everything else.
>> - Annotations need an appearance dictionary (unless their /Rect is one
>> and the same point, or /Popup or /Link annotations).
>>
>> Actual value of the annotation flag key is "/F null". I wonder whether
>> this is a bug (for having wanted to avoid the presence of /F at all).
>>
>> Otherwise, I don’t think that setting all embedded file annotations to
>> printable is a good default.
>>
>> Just in case it helps to the discussion,
> Peter is stepwise looking into all these issues but we decided to also
> see how that checking and standards evolve in cases where it's a
> confusing mess. And these 'appearance dicts' are an example of a mess.
> On the one hand there's predefined appearances and on the other hand
> enforced renderings which gives some chicken-egg issue. It smells a lot
> like bugs became features (standards) or 'acrobat behaviour' made the
> standard or ... (like the zero rect thing, which, given t e plenty of
> flags there are and verbosity there is in pdfm is pretty weird and
> actually can make viewers bark. Irr the current approach we follow is
> kind of a compromis.
>
> Now, of course we can play safe and *always* have some fake appearance
> (we could basically choose whatever funny shape we like as over time
> people will interpret hard codes symbols for attachments differently)
> and drop support for standardized renderings (that could adapt over time).
>
> So ... no changes etc till Peter gives his blessing as he is testing all
> this in the framework we've set up for it.
>
> Hans
>
As Hans stated, we are working on it and two of the three validation
problems are already solved (needs more testing though). The only
remaining problem is this
https://github.com/veraPDF/veraPDF-validation-profiles/wiki/PDFA-Parts-2-and-3-rules#rule-633-1
I attached the current code, so you and Rik are able to test it if you
want (creating a new format is required). Best contact me off-list if
you run into new problems.
Peter
lpdf-wid.7z
Description: Binary data
___
If your question is of interest to others as well, please add an entry to the
Wiki!
maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context
webpage : http://www.pragma-ade.nl / http://context.aanhet.net
archive : https://bitbucket.org/phg/context-mirror/commits/
wiki : http://contextgarden.net
___
Re: [NTG-context] attachments working again (issue with /EmbeddedFiles)
On 2/7/20 10:37 PM, Hans Hagen wrote: > Peter is stepwise looking into all these issues but we decided to also > see how that checking and standards evolve in cases where it's a > confusing mess. And these 'appearance dicts' are an example of a mess. > On the one hand there's predefined appearances and on the other hand > enforced renderings which gives some chicken-egg issue. It smells a lot > like bugs became features (standards) or 'acrobat behaviour' made the > standard or ... (like the zero rect thing, which, given t e plenty of > flags there are and verbosity there is in pdfm is pretty weird and > actually can make viewers bark. Irr the current approach we follow is > kind of a compromise. Just a comment on one of the issues. Maybe the simplest way to avoid the issues with the annotation flag is not including it at all. Once annotations have appearance dictionaries, Acrobat will print those ones with an annotation flag set to print. Pablo -- http://www.ousia.tk ___ If your question is of interest to others as well, please add an entry to the Wiki! maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context webpage : http://www.pragma-ade.nl / http://context.aanhet.net archive : https://bitbucket.org/phg/context-mirror/commits/ wiki : http://contextgarden.net ___
Re: [NTG-context] attachments working again (issue with /EmbeddedFiles)
On 2/7/2020 9:55 PM, Pablo Rodriguez wrote:
On 2/7/20 9:19 PM, Rik Kabel wrote:
On 2/7/2020 14:46, Pablo Rodriguez wrote:
many thanks for having fixed the issues with attachments (in latest beta
from 2020.02.07 18:36). I haven’t tested attachments with PDF/A-3a.
PDF/A-3a attachments still fail validation with the same issues.
Hi Rik,
it has way less issues, at least using verapdf-1.5.18 and the following
sample:
\setuptagging[state=start]
\setupbodyfont[30pt]
\setupbackend
[format=PDF/A-3a,
intent=sRGB IEC61966-2.1,
profile={sRGB.icc,default_gray.icc},
level=0]
\setupcolors[pagecolormodel=auto]
\setupinteraction[state=start]
\starttext
\startTEXpage[offset=1em]
an attachment\attachment[file=xml-mkiv.pdf,
type={application/pdf}]
\stopTEXpage
\stoptext
With "method=hidden" (no attachment annotation) it has no issue.
Both issues are related to annotations in general:
- If the flag annotation is present (/F key), it should be set to print
and disable everything else.
- Annotations need an appearance dictionary (unless their /Rect is one
and the same point, or /Popup or /Link annotations).
Actual value of the annotation flag key is "/F null". I wonder whether
this is a bug (for having wanted to avoid the presence of /F at all).
Otherwise, I don’t think that setting all embedded file annotations to
printable is a good default.
Just in case it helps to the discussion,
Peter is stepwise looking into all these issues but we decided to also
see how that checking and standards evolve in cases where it's a
confusing mess. And these 'appearance dicts' are an example of a mess.
On the one hand there's predefined appearances and on the other hand
enforced renderings which gives some chicken-egg issue. It smells a lot
like bugs became features (standards) or 'acrobat behaviour' made the
standard or ... (like the zero rect thing, which, given t e plenty of
flags there are and verbosity there is in pdfm is pretty weird and
actually can make viewers bark. Irr the current approach we follow is
kind of a compromis.
Now, of course we can play safe and *always* have some fake appearance
(we could basically choose whatever funny shape we like as over time
people will interpret hard codes symbols for attachments differently)
and drop support for standardized renderings (that could adapt over
time).
So ... no changes etc till Peter gives his blessing as he is testing all
this in the framework we've set up for it.
Hans
-
Hans Hagen | PRAGMA ADE
Ridderstraat 27 | 8061 GH Hasselt | The Netherlands
tel: 038 477 53 69 | www.pragma-ade.nl | www.pragma-pod.nl
-
___
If your question is of interest to others as well, please add an entry to the
Wiki!
maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context
webpage : http://www.pragma-ade.nl / http://context.aanhet.net
archive : https://bitbucket.org/phg/context-mirror/commits/
wiki : http://contextgarden.net
___
Re: [NTG-context] attachments working again (issue with /EmbeddedFiles)
On 2/7/20 9:19 PM, Rik Kabel wrote:
>
> On 2/7/2020 14:46, Pablo Rodriguez wrote:
>> many thanks for having fixed the issues with attachments (in latest beta
>> from 2020.02.07 18:36). I haven’t tested attachments with PDF/A-3a.
>>
> PDF/A-3a attachments still fail validation with the same issues.
Hi Rik,
it has way less issues, at least using verapdf-1.5.18 and the following
sample:
\setuptagging[state=start]
\setupbodyfont[30pt]
\setupbackend
[format=PDF/A-3a,
intent=sRGB IEC61966-2.1,
profile={sRGB.icc,default_gray.icc},
level=0]
\setupcolors[pagecolormodel=auto]
\setupinteraction[state=start]
\starttext
\startTEXpage[offset=1em]
an attachment\attachment[file=xml-mkiv.pdf,
type={application/pdf}]
\stopTEXpage
\stoptext
With "method=hidden" (no attachment annotation) it has no issue.
Both issues are related to annotations in general:
- If the flag annotation is present (/F key), it should be set to print
and disable everything else.
- Annotations need an appearance dictionary (unless their /Rect is one
and the same point, or /Popup or /Link annotations).
Actual value of the annotation flag key is "/F null". I wonder whether
this is a bug (for having wanted to avoid the presence of /F at all).
Otherwise, I don’t think that setting all embedded file annotations to
printable is a good default.
Just in case it helps to the discussion,
Pablo
--
http://www.ousia.tk
___
If your question is of interest to others as well, please add an entry to the
Wiki!
maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context
webpage : http://www.pragma-ade.nl / http://context.aanhet.net
archive : https://bitbucket.org/phg/context-mirror/commits/
wiki : http://contextgarden.net
___
Re: [NTG-context] attachments working again (issue with /EmbeddedFiles)
On 2/7/2020 14:46, Pablo Rodriguez wrote: many thanks for having fixed the issues with attachments (in latest beta from 2020.02.07 18:36). I haven’t tested attachments with PDF/A-3a. PDF/A-3a attachments still fail validation with the same issues. -- Rik ___ If your question is of interest to others as well, please add an entry to the Wiki! maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context webpage : http://www.pragma-ade.nl / http://context.aanhet.net archive : https://bitbucket.org/phg/context-mirror/commits/ wiki : http://contextgarden.net ___
