Re: Latest lilypond git master creating PDF 1.4 files instead of 1.7
Hello David
No that does not work, you need to compile from git. You might notice that
your generated pdf file does not in fact have transparency. This is because
the gs version bundled with the LP webpage download is 9.21, which is very old
(2016ish). For this transparency thing to work we need at least 9.53 I think.
As I said, pdftops/poppler is now intelligent enough to recognize that there
all alpha channels are actually fully visible and thus ignores transparency,
while pdf2ps/gs just assumes that there is transparency by some flag.
I have appended a pdf compiled against current git with gs 9.53.3 from debian
sid, so you can see the actual result with transparency. Also I’ve included
what pdftops makes of this.
So basically pdftops rasters with a resolution of 300 dpi, while gs/ps2writer
rasters by default with 720 dpi. So to get the same quality you’d have to
invoke pdftops with -r 720. But then the produced files are basically the same
size.
That being said, pdftops has some additional poppler advantages, so it is no
bad thing to use it (it generally handles fonts better for example). But I
guess one should increase the -r parameter, because 300 dpi is not really
sufficient for high quality printing.
But then, you probably should not convert your scores to ps anyways.
Cheers,
Valentin
alpha.pdf
Description: Adobe PDF document
\absolute {
\override NoteHead.color = #(rgb-color 0.13 0.73 0.13 0.07)
\override Flag.color = #(rgb-color 0.86 0 0.53 0)
\override Stem.color = #(rgb-color 1 1 1 0)
\override Beam.color = #(rgb-color 0 0.53 0.53 0.86)
\override Rest.color = #(0 0.2 0.93 1)
g'8 \noBeam aes'
r4 b'16 f' g'8
\once \hide Accidental a'
\once \override NoteHead.stencil =
#ly:text-interface::print
\once \override NoteHead.text =
\markup \with-color #(rgb-color 1 0 0 0.1) \lower #0.5 "text"
e'
\override NoteHead.color = #(rgb-color 0.13 0.73 0.13 0.93)
\override Flag.color = #(rgb-color 0.86 0 0.53 1)
\override Stem.color = #(rgb-color 1 1 1 1)
\override Beam.color = #(rgb-color 0 0.53 0.53 0.13)
\override Rest.color = #(0 0.2 0.93 0)
g'8 \noBeam aes'
r4 b'16 f' g'8
\once \hide Accidental a'
\once \override NoteHead.stencil =
#ly:text-interface::print
\once \override NoteHead.text =
\markup \with-color #(rgb-color 1 0 0 0.9) \lower #0.5 "text"
e'
}
alpha-pdftops.pdf
Description: Adobe PDF document
signature.asc
Description: This is a digitally signed message part.
Re: Latest lilypond git master creating PDF 1.4 files instead of 1.7
Hello David, I'm simply using a colored markup with alpha channel. Such as \markup\with-color #(rgb-color 1 0 0 0.5) "a" Cheers, Valentin 13.09.2021 06:59:02 David Wright : > On Sat 11 Sep 2021 at 21:35:00 (+0200), Valentin Petzel wrote: >> From what I see pdftops appears to be intelligent enough to rasterize the >> file >> only if it does actually contain transparency. If it does it uses a much less >> smooth rasterization that is probably not fit for printing, so be careful >> with >> this! See the attached picture for a comparisation, the left one is poppler >> (pdftops), the right one gs (pdf2ps). > > How do you produce an original PDF for carrying out this test? > > Cheers, > David.
Re: Latest lilypond git master creating PDF 1.4 files instead of 1.7
On Sat 11 Sep 2021 at 19:11:36 (-0700), H. S. Teoh wrote: > I discovered that converting the pdf to ps and back again significantly > reduces the file size. So it's kinda my poor-man's way of compressing > pdfs. Of course, this only works if your pdf doesn't contain any > features incompatible with ps, which in my case is true. And now that I > know that pdftops does a proper job with the conversion even when there > is transparency, it's a sufficiently-acceptable solution to this issue. Running any of these conversions on test-good.pdf will result in a significantly smaller file size simply because all the textedit information is stripped out. I ran these processes on a 28pp production score (2.21.80, no textedit), and the file sizes are: 407323 requiem6.pdf → pdf2ps → 1774527 requiem6-2.ps → ps2pdf → 548498 (134.7%) requiem6-2.pdf 407323 requiem6.pdf → (DAK's suggested method) → ps2pdf → 409954 (100.6%) requiem6-ps2pdf.pdf 407323 requiem6.pdf → pdftops → 2219804 requiem6-to.ps → ps2pdf → 371264 (91.1%) requiem6-to.pdf The loss in quality caused by pdf2ps (only) is as before. Cheers, David.
Re: Latest lilypond git master creating PDF 1.4 files instead of 1.7
On Sat 11 Sep 2021 at 21:35:00 (+0200), Valentin Petzel wrote: > From what I see pdftops appears to be intelligent enough to rasterize the > file > only if it does actually contain transparency. If it does it uses a much less > smooth rasterization that is probably not fit for printing, so be careful > with > this! See the attached picture for a comparisation, the left one is poppler > (pdftops), the right one gs (pdf2ps). How do you produce an original PDF for carrying out this test? Cheers, David.
Re: Latest lilypond git master creating PDF 1.4 files instead of 1.7
"H. S. Teoh" writes: > On Sat, Sep 11, 2021 at 09:35:00PM +0200, Valentin Petzel wrote: >> From what I see pdftops appears to be intelligent enough to rasterize >> the file only if it does actually contain transparency. If it does it >> uses a much less smooth rasterization that is probably not fit for >> printing, so be careful with this! See the attached picture for a >> comparisation, the left one is poppler (pdftops), the right one gs >> (pdf2ps). > > I experimented with pstops, and it does seem to produce a non-rasterized > PS file, which can then be converted back to PDF without transparency, > thus sidestepping this issue. I'll use that as a workaround for now. > :-D > > > [...] >> But anyway. Usually there should not be any reason to convert from pdf >> to ps, apart from printing or doing ps-style manipulations. But ps is >> only a subset of pdf, so the conversion in that direction cannot >> always be done in an acceptable manner. > > I discovered that converting the pdf to ps and back again significantly > reduces the file size. So it's kinda my poor-man's way of compressing > pdfs. Of course, this only works if your pdf doesn't contain any > features incompatible with ps, which in my case is true. And now that I > know that pdftops does a proper job with the conversion even when there > is transparency, it's a sufficiently-acceptable solution to this issue. It may be worth noting that somewhat counterintuitively running ps2pdf on a PDF file tends to work fine, producing a PDF file again in a single step without an intermediate ps file. -- David Kastrup
Re: Latest lilypond git master creating PDF 1.4 files instead of 1.7
On Sat, Sep 11, 2021 at 09:35:00PM +0200, Valentin Petzel wrote: > From what I see pdftops appears to be intelligent enough to rasterize > the file only if it does actually contain transparency. If it does it > uses a much less smooth rasterization that is probably not fit for > printing, so be careful with this! See the attached picture for a > comparisation, the left one is poppler (pdftops), the right one gs > (pdf2ps). I experimented with pstops, and it does seem to produce a non-rasterized PS file, which can then be converted back to PDF without transparency, thus sidestepping this issue. I'll use that as a workaround for now. :-D [...] > But anyway. Usually there should not be any reason to convert from pdf > to ps, apart from printing or doing ps-style manipulations. But ps is > only a subset of pdf, so the conversion in that direction cannot > always be done in an acceptable manner. I discovered that converting the pdf to ps and back again significantly reduces the file size. So it's kinda my poor-man's way of compressing pdfs. Of course, this only works if your pdf doesn't contain any features incompatible with ps, which in my case is true. And now that I know that pdftops does a proper job with the conversion even when there is transparency, it's a sufficiently-acceptable solution to this issue. > Also I have tested gv on the pdf-files, but I’m not experiencing these > delays. [...] I wonder if it has to do with the amount of available RAM, as my PC is 10+ years old, so on a more modern machine the difference would be unnoticeable. T -- If it tastes good, it's probably bad for you.
Re: Latest lilypond git master creating PDF 1.4 files instead of 1.7
From what I see pdftops appears to be intelligent enough to rasterize the file only if it does actually contain transparency. If it does it uses a much less smooth rasterization that is probably not fit for printing, so be careful with this! See the attached picture for a comparisation, the left one is poppler (pdftops), the right one gs (pdf2ps). Thus poppler will produce much smaller files that are easier to render (and if there is no transparency poppler will not do any rasterisation), but it rasterizes to a quite bad resolution (though this can probably be adjusted, but that removes the advantage). But anyway. Usually there should not be any reason to convert from pdf to ps, apart from printing or doing ps-style manipulations. But ps is only a subset of pdf, so the conversion in that direction cannot always be done in an acceptable manner. Also I have tested gv on the pdf-files, but I’m not experiencing these delays. Cheers, Valentin signature.asc Description: This is a digitally signed message part.
Re: Latest lilypond git master creating PDF 1.4 files instead of 1.7
On Sat, Sep 11, 2021 at 12:04:11PM +0200, Valentin Petzel wrote: > Okay. Basically the problem is that conversion from PDF to PS get’s > really complicated at some point, and results in really large PS > files. > > I’ve tracked down the responsible commit to be > cf2ae39e9d93c2dcdc50afd7d0389c314898273c, which implements > transparency for PDF. Now this seems plausible, as transparency is NOT > part of the Postscript specification, and Adobe and GS have just added > (very volatile) methods to the PS specification for transparency in a > created PDF file. [...] Is there an option for turning off transparency in Lilypond? I don't need transparency, so it would be nice to sidestep this issue in my environment. T -- Obviously, some things aren't very obvious.
Re: Latest lilypond git master creating PDF 1.4 files instead of 1.7
On Fri 10 Sep 2021 at 13:52:04 (-0700), H. S. Teoh wrote: > On Fri, Sep 10, 2021 at 09:52:09PM +0200, Jean Abou Samra wrote: > > Le 10/09/2021 à 19:09, H. S. Teoh a écrit : > > > OK, I attached a contrived .ly file that produces 2 pages of music > > > (the effect is most noticeable when there's more than 1 page), and > > > two .pdf's, one compiled by the official Debian lilypond package > > > (2.22.1), and one by my local lilypond build (latest git master). > > > > > > The second file (test-bad.pdf) is noticeably slower to view on a PDF > > > viewer; this is especially noticeable when flipping between pages. > > > Whereas the first one (test-good.pdf) renders quickly and pages > > > quickly. > > > > What is your PDF reader? With all of Evince, > > Okular and Firefox's built-in PDF reader on my > > Ubuntu system, both files render fast enough and > > I cannot see a noticeable difference. > [...] > > I'm using GV, which shows noticeably slowdown. I tested again with > Evince; the slowdown is still there but only in a more complex score. > With the contrived score I posted it's not really noticeable. Also, > Evince appears to cache generated pages at startup: with a more complex > score there's a noticeable delay during startup, but thereafter about > 2-3 pages are cached, and the delay only happens when you scroll to an > uncached page. With 2.22.1, there is no such slowdown. > > Since PDF reader performance seems an unreliable measure of this issue, > here's another one: running pdf2ps and then ps2pdf on the 2.22.1 score > produces a compressed score (well, sometimes it doesn't compress well > but the resulting file size is comparable to the original). But on git > master, both pdf2ps and ps2pdf take a very long time to run, and produce > a .ps and second .pdf about 10x the size of the original file. > > Something about the .pdf produced by (my build of) git master causes > these tools to behave an in unusual way, and I want to find out why, and > whether it's something caused by lilypond itself, or by some build > misconfiguration on my part. I've had a comment in my .bashrc file for years reminding me never to use pdf2ps, but pdftops instead (comes with poppler). I think pdf2ps might be suitable for the final rendering step in, say, CUPS. I did run your test on test-good.pdf, using Debian buster: 109532 test-good.pdf →pdf2ps→ 230847 good2.ps →ps2pdf→ 28144 good2.pdf 109532 test-good.pdf →pdftops→ 100040 goodto.ps →ps2pdf→ 19427 goodto.pdf 2000% screenshots attached. Cheers, David.
Re: Latest lilypond git master creating PDF 1.4 files instead of 1.7,Re: Latest lilypond git master creating PDF 1.4 files instead of 1.7
> Okay. Basically the problem is that conversion from PDF to PS get’s > really complicated at some point, and results in really large PS > files. > > I’ve tracked down the responsible commit to be > cf2ae39e9d93c2dcdc50afd7d0389c314898273c, which implements > transparency for PDF. Now this seems plausible, as transparency is > NOT part of the Postscript specification, and Adobe and GS have just > added (very volatile) methods to the PS specification for > transparency in a created PDF file. > > So when we have a PDF with transparency we cannot really convert it > into proper Postscript. Thus such a conversion appears to rasterize > the entire page, resulting in a very large ps file (of lower optical > quality) that is hard to render. > > But this probably means that your PDF viewer does not handle PDF > properly but instead converts the PDF to PS and displays that. That > is nothing Lilypond can influence, apart from maybe handling > transparency in a more intelligent way such that transparency is > only used if it is called for. But the best way would probably be to > use a proper PDF viewer. Maybe this deservers a comment in the manual. And maybe it deserves an issue in the bug tracker, too. Werner
Re: Latest lilypond git master creating PDF 1.4 files instead of 1.7
Okay. Basically the problem is that conversion from PDF to PS get’s really complicated at some point, and results in really large PS files. I’ve tracked down the responsible commit to be cf2ae39e9d93c2dcdc50afd7d0389c314898273c, which implements transparency for PDF. Now this seems plausible, as transparency is NOT part of the Postscript specification, and Adobe and GS have just added (very volatile) methods to the PS specification for transparency in a created PDF file. So when we have a PDF with transparency we cannot really convert it into proper Postscript. Thus such a conversion appears to rasterize the entire page, resulting in a very large ps file (of lower optical quality) that is hard to render. But this probably means that your PDF viewer does not handle PDF properly but instead converts the PDF to PS and displays that. That is nothing Lilypond can influence, apart from maybe handling transparency in a more intelligent way such that transparency is only used if it is called for. But the best way would probably be to use a proper PDF viewer. Cheers, Valentin signature.asc Description: This is a digitally signed message part.
Re: Latest lilypond git master creating PDF 1.4 files instead of 1.7
On Fri, Sep 10, 2021 at 09:52:09PM +0200, Jean Abou Samra wrote: > Le 10/09/2021 à 19:09, H. S. Teoh a écrit : > > OK, I attached a contrived .ly file that produces 2 pages of music > > (the effect is most noticeable when there's more than 1 page), and > > two .pdf's, one compiled by the official Debian lilypond package > > (2.22.1), and one by my local lilypond build (latest git master). > > > > The second file (test-bad.pdf) is noticeably slower to view on a PDF > > viewer; this is especially noticeable when flipping between pages. > > Whereas the first one (test-good.pdf) renders quickly and pages > > quickly. > > What is your PDF reader? With all of Evince, > Okular and Firefox's built-in PDF reader on my > Ubuntu system, both files render fast enough and > I cannot see a noticeable difference. [...] I'm using GV, which shows noticeably slowdown. I tested again with Evince; the slowdown is still there but only in a more complex score. With the contrived score I posted it's not really noticeable. Also, Evince appears to cache generated pages at startup: with a more complex score there's a noticeable delay during startup, but thereafter about 2-3 pages are cached, and the delay only happens when you scroll to an uncached page. With 2.22.1, there is no such slowdown. Since PDF reader performance seems an unreliable measure of this issue, here's another one: running pdf2ps and then ps2pdf on the 2.22.1 score produces a compressed score (well, sometimes it doesn't compress well but the resulting file size is comparable to the original). But on git master, both pdf2ps and ps2pdf take a very long time to run, and produce a .ps and second .pdf about 10x the size of the original file. Something about the .pdf produced by (my build of) git master causes these tools to behave an in unusual way, and I want to find out why, and whether it's something caused by lilypond itself, or by some build misconfiguration on my part. T -- It is of the new things that men tire --- of fashions and proposals and improvements and change. It is the old things that startle and intoxicate. It is the old things that are young. -- G.K. Chesterton
Re: Latest lilypond git master creating PDF 1.4 files instead of 1.7
Le 10/09/2021 à 19:09, H. S. Teoh a écrit : OK, I attached a contrived .ly file that produces 2 pages of music (the effect is most noticeable when there's more than 1 page), and two .pdf's, one compiled by the official Debian lilypond package (2.22.1), and one by my local lilypond build (latest git master). The second file (test-bad.pdf) is noticeably slower to view on a PDF viewer; this is especially noticeable when flipping between pages. Whereas the first one (test-good.pdf) renders quickly and pages quickly. What is your PDF reader? With all of Evince, Okular and Firefox's built-in PDF reader on my Ubuntu system, both files render fast enough and I cannot see a noticeable difference. Thanks, Jean
Re: Latest lilypond git master creating PDF 1.4 files instead of 1.7
Is it a problem with all documents, or only with specific ones? Can you give us some example file where you get the problem (also maybe two pdf files compiled with both versions)? signature.asc Description: This is a digitally signed message part.
Re: Latest lilypond git master creating PDF 1.4 files instead of 1.7
On Fri, Sep 10, 2021 at 02:22:50PM +0200, Jonas Hahnfeld wrote: > Am Freitag, dem 10.09.2021 um 10:46 +0200 schrieb Valentin Petzel: [...] > > This is nothing new and an arbitrary decision by Lilypond, simply > > because XeTeX defaults to 1.5 and thus produces error when embedding > > a 1.7 PDF. > > > > But this means that whatever problem you have should not come frome > > the PDF CompatibilityLevel, as this did never change (and anyway a > > lower version should only limit the amount of functionality but not > > create a need for conversion). > > It did intermittently change during the last development cycle after I > mistakenly removed "CompatibilityLevel 1.4" and recent Ghostscript > versions defaulted to 1.7. The old behavior was restored soon after to > avoid the warnings from XeTeX mentioned above. If there is an > advantage in using a newer compatibility level, it would be great if > you could open an issue on GitLab with more details. [...] Hmm, upon further investigation, CompatibilityLevel isn't really the problem after all. Lilypond 2.22.1 also produces files with compatibility level 1.4, but it does not exhibit the same slowness. What else has changed since 2.22.1? Or could it possibly be some other environmental issue like fonts / PDF libraries? T -- Leather is waterproof. Ever see a cow with an umbrella?
Re: Latest lilypond git master creating PDF 1.4 files instead of 1.7
Am Freitag, dem 10.09.2021 um 10:46 +0200 schrieb Valentin Petzel: > Hello, > > This is nothing new and an arbitrary decision by Lilypond, simply because > XeTeX defaults to 1.5 and thus produces error when embedding a 1.7 PDF. > > But this means that whatever problem you have should not come frome the PDF > CompatibilityLevel, as this did never change (and anyway a lower version > should only limit the amount of functionality but not create a need for > conversion). It did intermittently change during the last development cycle after I mistakenly removed "CompatibilityLevel 1.4" and recent Ghostscript versions defaulted to 1.7. The old behavior was restored soon after to avoid the warnings from XeTeX mentioned above. If there is an advantage in using a newer compatibility level, it would be great if you could open an issue on GitLab with more details. Thanks, Jonas signature.asc Description: This is a digitally signed message part
Re: Latest lilypond git master creating PDF 1.4 files instead of 1.7
Hello, This is nothing new and an arbitrary decision by Lilypond, simply because XeTeX defaults to 1.5 and thus produces error when embedding a 1.7 PDF. But this means that whatever problem you have should not come frome the PDF CompatibilityLevel, as this did never change (and anyway a lower version should only limit the amount of functionality but not create a need for conversion). If you really want Lilypond to output 1.7 pdf files then change in scm/ backend-library.scm in the function postscript->pdf the line "/ CompatibilityLevel 1.4” to "/CompatibilityLevel 1.7”. Still, I don’t think that this is your problem. Cheers, Valentin signature.asc Description: This is a digitally signed message part.
Re: Latest lilypond git master creating PDF 1.4 files instead of 1.7
Oh, is this why Adobe Reader has started putting up a box saying “preparing output” when I open a LilyPond pdf? Paul Sent from my iPad > On 10 Sep 2021, at 01:42, H. S. Teoh wrote: > > Hi all, > > For a long time I have been using a local build of lilypond cloned from > git, IIRC around 2.21.0 or thereabouts. Today, on a whim, I decided to > update lilypond to the latest git master. After recompiling everything, > lilypond started producing version 1.4 PDF files instead of version 1.7 > like it used to. The resulting file is very slow to load or view with > my local PDF reader, which makes it very difficult to work with. > > How do I coax lilypond to produce PDF 1.7 files again? I'm not 100% > sure it's lilypond's fault; it could be something else in my environment > that changed, perhaps one of lilypond's dependencies? But I'm at a > complete loss as to where to start looking. Any pointers would be > greatly appreciated. > > Thanks! > > > T > > -- > Right now I'm having amnesia and deja vu at the same time. I think I've > forgotten this before. >
Latest lilypond git master creating PDF 1.4 files instead of 1.7
Hi all, For a long time I have been using a local build of lilypond cloned from git, IIRC around 2.21.0 or thereabouts. Today, on a whim, I decided to update lilypond to the latest git master. After recompiling everything, lilypond started producing version 1.4 PDF files instead of version 1.7 like it used to. The resulting file is very slow to load or view with my local PDF reader, which makes it very difficult to work with. How do I coax lilypond to produce PDF 1.7 files again? I'm not 100% sure it's lilypond's fault; it could be something else in my environment that changed, perhaps one of lilypond's dependencies? But I'm at a complete loss as to where to start looking. Any pointers would be greatly appreciated. Thanks! T -- Right now I'm having amnesia and deja vu at the same time. I think I've forgotten this before.
