Re: insert listing so it can be copy/pasted from pdf

[Dr Eberhard Lisse]

Would something like this work?

http://mirror.ufs.ac.za/ctan/macros/latex/contrib/attachfile/attachfile.pdf

el


On 06/05/2019 04:25, Joel Kulesza wrote:
> On Sun, May 5, 2019 at 2:43 PM Bob Alvarez <4...@outlook.com> wrote:
> 
>>
>>  I want to insert text code listings into Lyx  so readers can copy and
>> paste from the pdf document into their plain text editor software.
>>
> 
> Bob,
> 
> This doesn't directly address your need, but to permit readers to obtain
> content from PDFs, the approach I've taken is to use the `navigator` (
> https://ctan.org/pkg/navigator?lang=en) package. What I will typically do
> is include the listing and then also use `navigator` to embed the full
> document as an attachment to the PDF.  I've begun to routinely take this
> approach because of ongoing difficulty providing easily copied text within
> a LaTeX-generated PDF.
> 
> Hope this helps,
> Joel
> 

Re: insert listing so it can be copy/pasted from pdf

[Bob Alvarez]

Here is an interesting discussion on the topic of this thread
embed-non-pdf-files-e-g-bibtex-into-pdf-with-hyperlink-in-the-pdf

The post lists 9 criteria for a useful embedded file that I mostly agree with. 
Then LateX packages are evaluated against the criteria by testing with several 
pdf readers as I did with Joel's pdf file. The packages are: embedfile, 
navigator, attachfile, and some combinations. None satisfied all the critera 
and this lead to writing a new package, intopdf. Unfortunately the new package 
was not tested.

I am still interested in how to get any of these working with Lyx.

Re: insert listing so it can be copy/pasted from pdf

[Bob Alvarez]

It looks like this did not get on the mailing list so I am re-sending


>Unless you will go through a publisher that will prevent it, I would still 
>encourage you to look at the approach I suggested. It is demonstrated here: 
>https://laws.lanl.gov/vhosts/mcnp.lanl.gov/pdf_files/la-ur-19-20291.pdf

Thanks for the example. I used it to test the readers on my Win7 desktop 
computer and was able to extract the downloads with: Adobe Acrobat Reader DC, 
Pdf-Xchange Viewer (free v2.5 2012), and Sumatra 2.3.2.

I like this approach. I doubt that I could use this for a paper with a 
traditional journal but who knows?? I also post papers to ResearchGate etc and 
there it would work since you just have to post a pdf. I don't know whether it 
would work on arXiv where they want tex files but it is also worth a shot.

So how do I implement this with Lyx? You mentioned to use the "Navigator' 
package. I looked at the documentation and AFAIK it allows including links to 
stuff but it is not clear how to add an attachment.

I would appreciate help in getting attachments working from a plain-vanilla 
install of Lyx.

Bob

Re: insert listing so it can be copy/pasted from pdf

[Joel Kulesza]

> On May 8, 2019, at 17:13, Bob Alvarez <4...@outlook.com> wrote:
> 
> > So I expect this to be a problem of your PDF Viewer or clipboard or OS.
> 
> I agree. As I mentioned in a previous post on this thread, putting the code 
> in the pdf is not reliable enough to be used. So I have decided not to 
> attempt to do it. I will put the code in a supplementary information file for 
> the paper and post it online. 
> 

Bob,

Unless you will go through a publisher that will prevent it, I would still 
encourage you to look at the approach I suggested. It is demonstrated here: 
https://laws.lanl.gov/vhosts/mcnp.lanl.gov/pdf_files/la-ur-19-20291.pdf

See Appendix A.

- Joel

Re: insert listing so it can be copy/pasted from pdf

[Bob Alvarez]

> So I expect this to be a problem of your PDF Viewer or clipboard or OS.

I agree. As I mentioned in a previous post on this thread, putting the code in 
the pdf is not reliable enough to be used. So I have decided not to attempt to 
do it. I will put the code in a supplementary information file for the paper 
and post it online.

Thanks for your help.

Bob

Re: insert listing so it can be copy/pasted from pdf

[Guenter Milde]

On 2019-05-06, Bob Alvarez wrote:

> [-- Type: text/plain, Encoding: quoted-printable --]


>>Which fonts does your PDF use?
> Here is what Pdf-Xchange viewer reported:

> CMSY9 (Embedded Subset)
> Type: Type 1 Encoding: Built-in Object Number: 5

> LMRomanlO-Regular (Embedded Subset)
> Type: Type 1 (CID) Encoding: Identity-H Object Number: 6

> LMRoman9- Regular (Embedded Subset)
> Type: Type 1 (CID) Encoding: Identity-H Object Number: 4

This looks good. (I get CM Roman instead of LM Roman with your
example but this should not have any influence on - and *.)

>>Is this with non-TeX fonts or with TeX fonts?
> Either one gives same result

>>Which output format?
> I tried several, luaTex and pdfLatex. All gave same problem.

I tested again with your example as well as the printable characters from 
Unicode Blocks Basic Latin and Latin-1 Supplement. The result when copying
back into text is:

 x−y
 
and

30 ! ” # $ % & ’
40 ( ) * + , − . / 0 1
50 2 3 4 5 6 7 8 9 : ;
60 < = > ? @ A B C D E
70 F G H I J K L M N O
80 P Q R S T U V W X Y
90 Z [ \ ] ^ _ ‘ a b c
100 d e f g h i j k l m
110 n o p q r s t u v w
120 x y z { | } ~

160
¡ ¢ £ ¤ ¥ ¦ §  ̈ ©
170 a « ¬ ®  ̄ ° ± 2 3
180  ́ μ ¶ ·  ̧ 1 o » 1⁄4 1⁄2
190 3⁄4 ¿ À Á Â Ã Ä Å Æ Ç
200 È É Ê Ë Ì Í Î Ï Ð Ñ
210 Ò Ó Ô Õ Ö × Ø Ù Ú Û
220 Ü Ý Þ ß à á â ã ä å
230 æ ç è é ê ë ì í î ï
240 ð ñ ò ó ô õ ö ÷ ø ù
250 ú û ü ý þ ÿ


So I expect this to be a problem of your PDF Viewer or clipboard or OS.

Could you try with another viewer?

With Postscript instead of PDF?

On a different computer?

Send the PDF as pm?

Günter

Re: insert listing so it can be copy/pasted from pdf

[Bob Alvarez]

>Which fonts does your PDF use?
Here is what Pdf-Xchange viewer reported:

CMSY9 (Embedded Subset)
Type: Type 1 Encoding: Built-in Object Number: 5

LMRomanlO-Regular (Embedded Subset)
Type: Type 1 (CID) Encoding: Identity-H Object Number: 6

LMRoman9- Regular (Embedded Subset)
Type: Type 1 (CID) Encoding: Identity-H Object Number: 4

>Is this with non-TeX fonts or with TeX fonts?
Either one gives same result

>Which output format?
I tried several, luaTex and pdfLatex. All gave same problem.

Re: insert listing so it can be copy/pasted from pdf

[Guenter Milde]

On 2019-05-06, Bob Alvarez wrote:

> [-- Type: text/plain, Encoding: quoted-printable --]

> Guenter and Joel

> I appreciate your help but I am coming to the conclusion that the pdf
> file format is not well enough defined to be able to reliably embed
> software listings that the reader can copy/paste into a file and then
> be able to compile the results.  I will have to continue as I have done
> in the past to include a listing in the pdf but also to provide a link
> to a supplementary online zip file with the source code in text files.

> Bob

> p.s.
> Guenter, here are the answers to some of the questions in your response:

>>Does it show up as hyphen or as question mark in the PDF?
> It is displayed as a '-' in the pdf but copy and paste of the 3 characters in 
> the listing results in x?y with '?' being 3F hex according to my editor.

OK this points to an encoding issue.

>>Which fonts does your PDF use?

> I do not know how to answer that question. How do I find out what fonts
> it uses?

In the "evince" PDF viewer, there is a menu entry "Properties" with a tab
"Fonts". Something similar should be possible with other viewers, too.

>>What is the result if your listing contains: ...

> Here is what I get if I enter your list into the listing, create a pdf
> then copy and paste the text in the listing.

> 30 ! " # $ % & '
> 40 ( ) ? + , ? . / 0 1
> 50 2 3 4 5 6 7 8 9 : ;
> 60 < = > ? @ A B C D E
> 70 F G H I J K L M N O
> 80 P Q R S T U V W X Y
> 90 Z [ \ ] ^ _ ` a b c
> 100 d e f g h i j k l m
> 110 n o p q r s t u v w
> 120 x y z { | } ~
> 160 ½¢¿¤¥¦?¨ c ?
> 170
> a ?¬­
> R ?¯°±²³
> 180 ´µ¶· ¹ o ?¼½
> 190 ¾¾ÀÁÂÃÄÅÆÇ
> 200 ÈÉÊËÌÍÎÏÐÑ
> 210 ÒÓÔÕÖ×ØÙÚÛ
> 220 ÜÝÞÿàáâãäå
> 230 æçèéêëìíîï
> 240 ðñòóôõö÷øù
> 250 úûüýþ¸

This looks not good: besides the "-", also "*" is mapped to "?".
And non-ASCII is even worse.

Is this with non-TeX fonts or with TeX fonts?
Which output format?

Günter

Re: insert listing so it can be copy/pasted from pdf

[Bob Alvarez]

Guenter and Joel

I appreciate your help but I am coming to the conclusion that the pdf file 
format is not well enough defined to be able to reliably embed software 
listings that the reader can copy/paste into a file and then be able to compile 
the results.  I will have to continue as I have done in the past to include a 
listing in the pdf but also to provide a link to a supplementary online zip 
file with the source code in text files.

Bob

p.s.
Guenter, here are the answers to some of the questions in your response:

>Does it show up as hyphen or as question mark in the PDF?
It is displayed as a '-' in the pdf but copy and paste of the 3 characters in 
the listing results in x?y with '?' being 3F hex according to my editor.

>Which fonts does your PDF use?
I do not know how to answer that question. How do I find out what fonts it uses?

>What is the result if your listing contains: ...
Here is what I get if I enter your list into the listing, create a pdf then 
copy and paste the text in the listing.

30 ! " # $ % & '
40 ( ) ? + , ? . / 0 1
50 2 3 4 5 6 7 8 9 : ;
60 < = > ? @ A B C D E
70 F G H I J K L M N O
80 P Q R S T U V W X Y
90 Z [ \ ] ^ _ ` a b c
100 d e f g h i j k l m
110 n o p q r s t u v w
120 x y z { | } ~
160 ½¢¿¤¥¦?¨ c ?
170
a ?¬­
R ?¯°±²³
180 ´µ¶· ¹ o ?¼½
190 ¾¾ÀÁÂÃÄÅÆÇ
200 ÈÉÊËÌÍÎÏÐÑ
210 ÒÓÔÕÖ×ØÙÚÛ
220 ÜÝÞÿàáâãäå
230 æçèéêëìíîï
240 ðñòóôõö÷øù
250 úûüýþ¸

Re: insert listing so it can be copy/pasted from pdf

[Guenter Milde]

On 2019-05-06, Bob Alvarez wrote:

>>>Does using "non-TeX" fonts in Document>Settings>Fonts help?(This
>>>allows all Unicode characters in listings and uses Unicode-encoded
>>>fonts in the output.)

> There was no change. The '-' character on the clipboard is still
> encoded as 3f hex.

Does it show up as hyphen or as question mark in the PDF?

Which fonts does your PDF use?

Maybe it is a lookalike character in the source?
Which character code do you get when copying from the LyX-GUI?

Are there similar problems with other characters in listings?
What is the result if your listing contains:

   30 ! " # $ % & '
   40 ( ) * + , - . / 0 1
   50 2 3 4 5 6 7 8 9 : ;
   60 < = > ? @ A B C D E
   70 F G H I J K L M N O
   80 P Q R S T U V W X Y
   90 Z [ \ ] ^ _ ` a b c
  100 d e f g h i j k l m
  110 n o p q r s t u v w
  120 x y z { | } ~

  160   ¡ ¢ £ ¤ ¥ ¦ § ¨ ©
  170 ª « ¬ ­ ® ¯ ° ± ² ³
  180 ´ µ ¶ · ¸ ¹ º » ¼ ½
  190 ¾ ¿ À Á Â Ã Ä Å Æ Ç
  200 È É Ê Ë Ì Í Î Ï Ð Ñ
  210 Ò Ó Ô Õ Ö × Ø Ù Ú Û
  220 Ü Ý Þ ß à á â ã ä å
  230 æ ç è é ê ë ì í î ï
  240 ð ñ ò ó ô õ ö ÷ ø ù
  250 ú û ü ý þ ÿ

> p.s. I copied the text at the bottom of my email and pasted into a text
> editor then saved as example2.lyx. When I tried to open it with my Lyx
> 2.1.3 I get a pop-up saying

> ...example2.lyx is not a readable LyX document.

> where ... is the path on my computer

Here, I was able to save the example from my text editor to a file, open it
in LyX and compile: 

Drag-and-drop from the PDF results in:

  x−y
  
With the - beeing the ASCII Character (45, 0x2D) 002D   HYPHEN-MINUS

Günter

Re: insert listing so it can be copy/pasted from pdf

[Joel Kulesza]

On Sun, May 5, 2019 at 2:43 PM Bob Alvarez <4...@outlook.com> wrote:

>
>  I want to insert text code listings into Lyx  so readers can copy and
> paste from the pdf document into their plain text editor software.
>

Bob,

This doesn't directly address your need, but to permit readers to obtain
content from PDFs, the approach I've taken is to use the `navigator` (
https://ctan.org/pkg/navigator?lang=en) package. What I will typically do
is include the listing and then also use `navigator` to embed the full
document as an attachment to the PDF.  I've begun to routinely take this
approach because of ongoing difficulty providing easily copied text within
a LaTeX-generated PDF.

Hope this helps,
Joel

Re: insert listing so it can be copy/pasted from pdf

[Bob Alvarez]

>>Does using "non-TeX" fonts in Document>Settings>Fonts help?(This allows all 
>>Unicode characters in listings and uses Unicode-encoded fonts in the output.)

There was no change. The '-' character on the clipboard is still encoded as 3f 
hex.

Bob

p.s. I copied the text at the bottom of my email and pasted into a text editor 
then saved as example2.lyx. When I tried to open it with my Lyx 2.1.3 I get a 
pop-up saying

...example2.lyx is not a readable LyX document.

where ... is the path on my computer

Re: insert listing so it can be copy/pasted from pdf

[Guenter Milde]

On 2019-05-05, Bob Alvarez wrote:

> I want to insert text code listings into Lyx so readers can copy and
> paste from the pdf document into their plain text editor software.
>
> To replicate my problem use the example lyx file belo. Export as pdf
> (LuaTex). In the pdf select the listing text 'x-y', copy onto the
> clipboard, then paste into a text editor.
>
> My editor complains that it cannot represent some of the characters.
> The '-' character in the listing is encoded as 3f hex instead of 2d hex
> as it would be with ASCII.

This is really strange, as 2f is a question mark.

> As you can see in the example, I tried fooling around with the listings
> parameters by setting extendedchars=false but that did not help.
>
> Any suggestions as to how to fix this?

The document uses TeX fonts with LuaTeX:

> \use_non_tex_fonts false

Does using "non-TeX" fonts  in Document>Settings>Fonts help?
(This allows all Unicode characters in listings and uses Unicode-encoded
fonts in the output.)

Günter

Re: insert listing so it can be copy/pasted from pdf

[Bob Alvarez]

Whoops--I inadvertently sent this before I finished. Here is the complete 
question ...

I want to insert text code listings into Lyx  so readers can copy and paste 
from the pdf document into their plain text editor software.

To replicate my problem use the example lyx file belo. Export as pdf (LuaTex). 
In the pdf select the listing text 'x-y', copy onto the clipboard, then paste 
into a text editor.

My editor complains that it cannot represent some of the characters. The '-' 
character in the listing is encoded as 3f hex instead of 2d hex as it would be 
with ASCII.

As you can see in the example, I tried fooling around with the listings 
parameters by setting extendedchars=false
but that did not help.

Any suggestions as to how to fix this?

Bob

 example.lyx--
#LyX 2.1 created this file. For more info see http://www.lyx.org/
\lyxformat 474
\begin_document
\begin_header
\textclass article
\use_default_options true
\maintain_unincluded_children false
\language english
\language_package default
\inputencoding auto
\fontencoding global
\font_roman default
\font_sans default
\font_typewriter default
\font_math auto
\font_default_family default
\use_non_tex_fonts false
\font_sc false
\font_osf false
\font_sf_scale 100
\font_tt_scale 100
\graphics default
\default_output_format default
\output_sync 0
\bibtex_command default
\index_command default
\paperfontsize default
\spacing single
\use_hyperref false
\papersize default
\use_geometry false
\use_package amsmath 1
\use_package amssymb 1
\use_package cancel 1
\use_package esint 1
\use_package mathdots 1
\use_package mathtools 1
\use_package mhchem 1
\use_package stackrel 1
\use_package stmaryrd 1
\use_package undertilde 1
\cite_engine basic
\cite_engine_type default
\biblio_style plain
\use_bibtopic false
\use_indices false
\paperorientation portrait
\suppress_date false
\justification true
\use_refstyle 1
\index Index
\shortcut idx
\color #008000
\end_index
\secnumdepth 3
\tocdepth 3
\paragraph_separation indent
\paragraph_indentation default
\quotes_language english
\papercolumns 1
\papersides 1
\paperpagestyle default
\listings_params "extendedchars=false"
\tracking_changes false
\output_changes false
\html_math_output 0
\html_css_as_file 0
\html_be_strict false
\end_header

\begin_body

\begin_layout Standard
\begin_inset Float figure
wide false
sideways false
status open

\begin_layout Plain Layout
\begin_inset listings
lstparams "basicstyle={\small},showstringspaces=false"
inline false
status open

\begin_layout Plain Layout

x-y
\end_layout

\end_inset


\end_layout

\end_inset


\end_layout

\end_body
\end_document