Re: LyX terrible converting images
I have used LyX in -dbg mode and checked the conversion process. It does not seem that LyX itself is doing things badly, but it calls convert for every image on the current screen and convert seems to be doing a bad job, taking up all memory and filling the swap area as well. Wouldn't 'nice' be useful in this situation? Sure. But not LyX's business. The user is free to write his own shell scrip t wrapper for convert/gs/whatever. Total freedom to set the nice value as desired. Oh, I agree... this was intended as a tip for Joao... (or people in general who suffer from this problem) Hm... I had to leave for 2 days, now I am catching up the news. I think that adding nice to the conversion process is a good idea that can be done in a user-by-user basis, but I believe that one of the major problems of convert is its memory consumption. Adding nice would reduce the CPU consumption, but after a while memory will again fill up with imaegs being coverted. Thta is why I believe that making the conversion process a serial one would speed up things much more: memory is used for one image at a time and thus fills up only in extreme cases. No swap, no disk delays, etc... joao
Re: LyX terrible converting images
--- Joao B. Oliveira [EMAIL PROTECTED] wrote: Thta is why I believe that making the conversion process a serial one would speed up things much more: memory is used for one image at a time and thus fills up only in extreme cases. No swap, no disk delays, etc... I think it is better to simply have a function to add preview picture: create a preview picture and put it aloneside the original one. User specifies the resolution and other attributes of the preview pictures. Only reconvert when the original has a timestamp more recent than the preview picture. No delay, no disk race, no memory comsumption ... Why not? Max
Re: LyX terrible converting images
On Thu, 20 Feb 2003, Joao B. Oliveira wrote: I think that adding nice to the conversion process is a good idea that can be done in a user-by-user basis, but I believe that one of the major problems of convert is its memory consumption. Adding nice would reduce the CPU consumption, but after a while memory will again fill up with imaegs being coverted. Thta is why I believe that making the conversion process a serial one would speed up things much more: memory is used for one image at a time and thus fills up only in extreme cases. No swap, no disk delays, etc... You can always make your own converter command that causes the conversions to run in series... it could be done like this for instance: You need a script, serialConvertServer.sh, that continuously reads a file and if the file contains a non-empty line, it executes that line. (The line should be a conversion command). Then change the conversion rule to be: \convert png eps serialConvert.sh $$i $$o where serialConvert.sh adds new assignements to a queue-file. The script could also start serialConvertServer.sh if it isn't running. Then the script should wait until the output exists, and when it exists it returns. Perhaps there's even a ready made unix command that does these things? /Christian -- Christian Ridderström, +46-8-790 91 37 http://www.md.kth.se/~chr Mechatronics lab, Dept. of Machine Designhttp://www.md.kth.se
Re: LyX terrible converting images
I have used LyX in -dbg mode and checked the conversion process. It does not seem that LyX itself is doing things badly, but it calls convert for every image on the current screen and convert seems to be doing a bad job, taking up all memory and filling the swap area as well. Wouldn't 'nice' be useful in this situation? Sure. But not LyX's business. The user is free to write his own shell scrip t wrapper for convert/gs/whatever. Total freedom to set the nice value as desired. Oh, I agree... this was intended as a tip for Joao... (or people in general who suffer from this problem) Hm... I had to leave for 2 days, now I am catching up the news. I think that adding nice to the conversion process is a good idea that can be done in a user-by-user basis, but I believe that one of the major problems of convert is its memory consumption. Adding nice would reduce the CPU consumption, but after a while memory will again fill up with imaegs being coverted. Thta is why I believe that making the conversion process a serial one would speed up things much more: memory is used for one image at a time and thus fills up only in extreme cases. No swap, no disk delays, etc... joao
Re: LyX terrible converting images
--- Joao B. Oliveira [EMAIL PROTECTED] wrote: Thta is why I believe that making the conversion process a serial one would speed up things much more: memory is used for one image at a time and thus fills up only in extreme cases. No swap, no disk delays, etc... I think it is better to simply have a function to add preview picture: create a preview picture and put it aloneside the original one. User specifies the resolution and other attributes of the preview pictures. Only reconvert when the original has a timestamp more recent than the preview picture. No delay, no disk race, no memory comsumption ... Why not? Max
Re: LyX terrible converting images
On Thu, 20 Feb 2003, Joao B. Oliveira wrote: I think that adding nice to the conversion process is a good idea that can be done in a user-by-user basis, but I believe that one of the major problems of convert is its memory consumption. Adding nice would reduce the CPU consumption, but after a while memory will again fill up with imaegs being coverted. Thta is why I believe that making the conversion process a serial one would speed up things much more: memory is used for one image at a time and thus fills up only in extreme cases. No swap, no disk delays, etc... You can always make your own converter command that causes the conversions to run in series... it could be done like this for instance: You need a script, serialConvertServer.sh, that continuously reads a file and if the file contains a non-empty line, it executes that line. (The line should be a conversion command). Then change the conversion rule to be: \convert png eps serialConvert.sh $$i $$o where serialConvert.sh adds new assignements to a queue-file. The script could also start serialConvertServer.sh if it isn't running. Then the script should wait until the output exists, and when it exists it returns. Perhaps there's even a ready made unix command that does these things? /Christian -- Christian Ridderström, +46-8-790 91 37 http://www.md.kth.se/~chr Mechatronics lab, Dept. of Machine Designhttp://www.md.kth.se
Re: LyX terrible converting images
> > > > > I have used LyX in -dbg mode and checked the conversion process. It > > > > > does not seem that LyX itself is doing things badly, but it calls > > > > > convert for every image on the current screen and convert seems to > > > > > be doing a bad job, taking up all memory and filling the swap area > > > > > as well. > > > > > > Wouldn't 'nice' be useful in this situation? > > > > Sure. But not LyX's business. The user is free to write his own shell scrip > t > > wrapper for convert/gs/whatever. Total freedom to set the nice value as > > desired. > > Oh, I agree... this was intended as a tip for Joao... (or people in > general who suffer from this problem) Hm... I had to leave for 2 days, now I am catching up the news. I think that adding nice to the conversion process is a good idea that can be done in a user-by-user basis, but I believe that one of the major problems of convert is its memory consumption. Adding nice would reduce the CPU consumption, but after a while memory will again fill up with imaegs being coverted. Thta is why I believe that making the conversion process a serial one would speed up things much more: memory is used for one image at a time and thus fills up only in extreme cases. No swap, no disk delays, etc... joao
Re: LyX terrible converting images
--- "Joao B. Oliveira" <[EMAIL PROTECTED]> wrote: > Thta is why I believe that making the conversion process a serial > one would speed up things much more: memory is used for one image > at a time and thus fills up only in extreme cases. No swap, no disk > delays, etc... I think it is better to simply have a function to add preview picture: create a preview picture and put it aloneside the original one. User specifies the resolution and other attributes of the preview pictures. Only reconvert when the original has a timestamp more recent than the preview picture. No delay, no disk race, no memory comsumption ... Why not? Max
Re: LyX terrible converting images
On Thu, 20 Feb 2003, Joao B. Oliveira wrote: > I think that adding nice to the conversion process is a good idea > that can be done in a user-by-user basis, but I believe that one > of the major problems of convert is its memory consumption. Adding > nice would reduce the CPU consumption, but after a while memory > will again fill up with imaegs being coverted. > > Thta is why I believe that making the conversion process a serial > one would speed up things much more: memory is used for one image > at a time and thus fills up only in extreme cases. No swap, no disk > delays, etc... You can always make your own converter command that causes the conversions to run in series... it could be done like this for instance: You need a script, serialConvertServer.sh, that continuously reads a file and if the file contains a non-empty line, it executes that line. (The line should be a conversion command). Then change the conversion rule to be: \convert png eps "serialConvert.sh $$i $$o" where serialConvert.sh adds new assignements to a queue-file. The script could also start serialConvertServer.sh if it isn't running. Then the script should wait until the output exists, and when it exists it returns. Perhaps there's even a ready made unix command that does these things? /Christian -- Christian Ridderström, +46-8-790 91 37 http://www.md.kth.se/~chr Mechatronics lab, Dept. of Machine Designhttp://www.md.kth.se
Re: LyX terrible converting images
Andre Poenitz wrote: Have you tried to run xfig's exported .eps files through 'eps2eps'? It makes pretty compact .eps... Much more complicated idea would be to produce Metapost sources instead of xfig -- interposing of Bezier curves over set of points is very simple in MP. However, of course, it is very complicated and substantial step. Matej -- Matej Cepl, Finger: 89EF 4BC6 288A BF43 1BAB 25C3 E09F EF25 D964 84AC 138 Highland Ave. #10, Somerville, Ma 02143, (617) 623-1488 In those days spirits were brave, the stakes were high, men were real men, women were real women and small furry creatures from Alpha Centauri were real small furry creatures from Alpha Centauri. -- Douglas Adams
Re: LyX terrible converting images
Andre Poenitz wrote: Have you tried to run xfig's exported .eps files through 'eps2eps'? It makes pretty compact .eps... Much more complicated idea would be to produce Metapost sources instead of xfig -- interposing of Bezier curves over set of points is very simple in MP. However, of course, it is very complicated and substantial step. Matej -- Matej Cepl, Finger: 89EF 4BC6 288A BF43 1BAB 25C3 E09F EF25 D964 84AC 138 Highland Ave. #10, Somerville, Ma 02143, (617) 623-1488 In those days spirits were brave, the stakes were high, men were real men, women were real women and small furry creatures from Alpha Centauri were real small furry creatures from Alpha Centauri. -- Douglas Adams
Re: LyX terrible converting images
Andre Poenitz wrote: > Have you tried to run xfig's exported .eps files through 'eps2eps'? > It makes pretty compact .eps... Much more complicated idea would be to produce Metapost sources instead of xfig -- interposing of Bezier curves over set of points is very simple in MP. However, of course, it is very complicated and substantial step. Matej -- Matej Cepl, Finger: 89EF 4BC6 288A BF43 1BAB 25C3 E09F EF25 D964 84AC 138 Highland Ave. #10, Somerville, Ma 02143, (617) 623-1488 In those days spirits were brave, the stakes were high, men were real men, women were real women and small furry creatures from Alpha Centauri were real small furry creatures from Alpha Centauri. -- Douglas Adams
Re: LyX terrible converting images
Joao B. Oliveira wrote: Hi everyone (specially the folks from the devel team), Hello, Joao. Some history first: I keep LyX running on a Computer Science Department, with some 20-30 students constantly using LyX on several machines. Everything was fine, until LyX 1.2.0 came along. The problem was, the handling of figures was changed in that version, and from that version onwards the students that work on documents with a larger number of figures (or larger figures) were horrified to see that their documents were taking like 10 minutes (sometimes much longer) to be opened, and the convert processes would take over the machine. Simply opening a document and scrolling down to the editing point would create so many conversions and take up so much memory that using the system is impossible. In fact, situation is so bad that I have already given up LyX myself, at least for documents that have more than a few images. The image handling code in LyX 1.2 is indeed nasty. Things should be much better in LyX 1.3. In LyX 1.3 the conversion process is triggered only for images that have been on screen for 2 seconds, so you should now be able to scroll without any conversion occurring. Our machines are neither small nor slow, and LyX (currently 1.3.0) simply makes them stop with the conversion process. I am surprised at this as I would expect only one or two images to be converted at a time (since only one or two images will be visible on the screen at any one time). So, I went after documentation or details but did not find anything that I could use to accelerate things. I really need to understand what is going on, and maybe someone can answer these questions (and ADD THEM to the current documentation): That would be welcome. -- First of all, how does the conversion/display process really works? Specially, how EPS files are handled? (I understand that they are always converted to ppm for displaying? Why ppm?) If you run $ lyx -dbg graphics you'll see that the conversion is handled by an external program. You can specify the program you use by defining formats and converters between formats. See the Edit-Preferences dialog. By default, we use ImageMagick's 'convert' program, run through a script convertDefault.sh. I too find that 'convert' is very slow. (But this shouldn't really matter. See above.) I attach a modified convertDefault.sh that uses ghostscript to convert eps files to either ppm or png format. Personally, I would use it as the basis for a my_ps2ppm or my_ps2png converter... As for the final format, well we chose ppm because both the xforms and Qt frontends can load it natively. If you are using the Qt frontned, then I suggest that you define a converter ps - png. -- In previous versions EPS files could be displayed rather quickly. Why was their processing slower from 1.2.0 on? Because the code was rewritten from scratch to be more flexible. The old code had its own bugs and was unmaintainable. -- Is there a way of avoiding the conversion process and handling EPS files directly? Possibly. It depends on the image loading abilities of the underlying GUI toolkit. The xforms library most certainly CANNOT handle arbitrary PostScript files reliably, so native loading of PostScript files is disabled in the xforms frontend. I don't know about the abilities of the Qt loader. Try modifying the loadableFormats method in src/frontends/qt2/QLImage.C. -- Is there a way of accelerating the conversion process (if it cannot be avoided)? Are there shortcuts to be used? Try using a converter other than ImageMagick's convert. See attached. -- When several images are in the same figure, they *all* get converted at the same time, filling up memory and swap space. At the very least, can this be changed so that we get one conversion at a time, to preserver swap space (and time, as a consequence?) -- If EPS files are large (I have some with more or less 1.5 Mb) their conversion and displaying takes all memory and a 64Mb computer simply crashes. With more images of the same size, larger machines crash as well. Was this foreseen? Of course not. Feel free to add you expertise to the development effort. I would appreciate any help, as I want to keep LyX running but currently I also have to explain many things to both management and disappointed students. Let us know if any of this information helps... -- Angus convertDefault.sh Description: application/shellscript
Re: LyX terrible converting images
Joao == Joao B Oliveira [EMAIL PROTECTED] writes: Joao -- If EPS files are large (I have some with more or less 1.5 Mb) Joao their conversion and displaying takes all memory and a 64Mb Joao computer simply crashes. With more images of the same size, Joao larger machines crash as well. Was this foreseen? The question that comes to mind is: why do you have eps files of 1.5Mb? Where do they come from? Postscript is really not the right format for this kind of images... JMarc
Re: LyX terrible converting images
Jean-Marc Lasgouttes wrote: Joao -- If EPS files are large (I have some with more or less 1.5 Mb) Joao their conversion and displaying takes all memory and a 64Mb Joao computer simply crashes. With more images of the same size, Joao larger machines crash as well. Was this foreseen? The question that comes to mind is: why do you have eps files of 1.5Mb? Where do they come from? Postscript is really not the right format for this kind of images... I have got similar beasts from R in some cases, but then it was possible to use .png images directly, which were much smaller. Matej -- Matej Cepl, Finger: 89EF 4BC6 288A BF43 1BAB 25C3 E09F EF25 D964 84AC 138 Highland Ave. #10, Somerville, Ma 02143, (617) 623-1488 The function of the expert is not to be more right than other people, but to be wrong for more sophisticated reasons. -- Dr. David Butler, British psephologist
Re: LyX terrible converting images
hi everyone, Some history first: I keep LyX running on a Computer Science... The image handling code in LyX 1.2 is indeed nasty. Things should be much better in LyX 1.3. In LyX 1.3 the conversion process is triggered only for images that have been on screen for 2 seconds, so you should now be able to scroll without any conversion occurring. Well, first of all, thanks for the prompt answer to all these questions... one cannot stop wondering at the patience that the development team puts into answering questions. I have to admit, as we just installed LyX 1.3.0, we are still discovering the points that are causing us problems. For example, the problem about a long time on loading a document was reported for the previous version, not (yet) for 1.3.0. Our machines are neither small nor slow, and LyX (currently 1.3.0) simply makes them stop with the conversion process. I am surprised at this as I would expect only one or two images to be converted at a time (since only one or two images will be visible on the screen at any one time). Well, not really. It is rather common (at least here) to put several smaller images in a figure, to draw comparisons like In Fig 6a you see this, and in 6b we clearly see that, but 6c clearly shows Sometimes we put four or six smaller EPS imagens in a figure. (And it does not look that bad :) -- First of all, how does the conversion/display process really works? Specially, how EPS files are handled? (I understand that they are always converted to ppm for displaying? Why ppm?) By default, we use ImageMagick's 'convert' program, run through a script convertDefault.sh. I too find that 'convert' is very slow. (But this shouldn't really matter. See above.) I have used LyX in -dbg mode and checked the conversion process. It does not seem that LyX itself is doing things badly, but it calls convert for every image on the current screen and convert seems to be doing a bad job, taking up all memory and filling the swap area as well. I attach a modified convertDefault.sh that uses ghostscript to convert eps files to either ppm or png format. Personally, I would use it as the basis for a my_ps2ppm or my_ps2png converter... Thank you! As some initial tests are showing, using gs directly seems to be significantly more efficient than convert. Maybe this was the reason for most of our problems all along. I'll invest on that in the next days. To begin, comparing the shell script you sent (that is, gs) with the traditional convert, regarding time and memory consumption... -- Is there a way of accelerating the conversion process (if it cannot be avoided)? Are there shortcuts to be used? Try using a converter other than ImageMagick's convert. See attached. Ok. I think gs will be a resonable bet at the moment. -- If EPS files are large (I have some with more or less 1.5 Mb) their conversion and displaying takes all memory and a 64Mb computer simply crashes. With more images of the same size, larger machines crash as well. Was this foreseen? Of course not. Feel free to add you expertise to the development effort. Sorry. It was a bad choice of words. What I meant was, it was surprising that convert dies in a quite ungraceful way. I would appreciate any help, as I want to keep LyX running but currently I also have to explain many things to both management and disappointed students. Let us know if any of this information helps... Surely it helps, the path is getting clearer now. I suppose that trying other ways to avoid convert would be an interesting option, and I intend to spread the word around here. Of course, Jean-Marc's question about how to get 1.5 Mb EPS images is a valid one. The answer: we have a few programs that create images directly in XFig format. From that they can be edited and/or converted into whatever we want. Unfortunately they contain many rectangles and squares represented as polylines in XFig, and when a polyline is exported it produces a surprisingly large PS code. In fact, there is room for improvement in the tools, trying to avoid polylines or changing them to simpler structures. I'll test that too, as it is possible that 4 straight lines have a much shorter description produced by XFig than a polyline. Well, I have to thank again all people that contribute with suggestions to solve the problem... this is the kind of thing that makes LyX so amazing! thank you all, j. b. oliveira
Re: LyX terrible converting images
Joao B. Oliveira wrote: Well, not really. It is rather common (at least here) to put several smaller images in a figure, to draw comparisons like In Fig 6a you see this, and in 6b we clearly see that, but 6c clearly shows Sometimes we put four or six smaller EPS imagens in a figure. (And it does not look that bad :) -- First of all, how does the conversion/display process really works? Specially, how EPS files are handled? (I understand that they are always converted to ppm for displaying? Why ppm?) By default, we use ImageMagick's 'convert' program, run through a script convertDefault.sh. I too find that 'convert' is very slow. (But this shouldn't really matter. See above.) I have used LyX in -dbg mode and checked the conversion process. It does not seem that LyX itself is doing things badly, but it calls convert for every image on the current screen and convert seems to be doing a bad job, taking up all memory and filling the swap area as well. Sure, at the moment shell scripts are generated for each single conversion process. These conversions therefore execute in parallel. One possible improvement would be to gather all images on the screen into a list and send the list to a shell script for conversion and the conversions would execute in series. Another possible improvement, tied to this, would be to enable communication between LyX and the conversion script so that individual images in this list of many images could be loaded by LyX as they become available. At the moment, LyX only receives notification that the shell script has finished, which it uses as a flag to try and load the .png or .ppm file. Finally, if we enable LyX to use pipes to communicate with the shell script, Dekel's suggestion of loading the image from stdout rather than from a file written to disk becomes possible also. I attach a modified convertDefault.sh that uses ghostscript to convert eps files to either ppm or png format. Personally, I would use it as the basis for a my_ps2ppm or my_ps2png converter... Thank you! As some initial tests are showing, using gs directly seems to be significantly more efficient than convert. Maybe this was the reason for most of our problems all along. I'll invest on that in the next days. To begin, comparing the shell script you sent (that is, gs) with the traditional convert, regarding time and memory consumption... Note that sometimes gs refuses to convert postscript files to bitmap format. However, it will happily 'fix' such problem files if you run them through eps2eps and thereafter has no problems at all. Personally, I prefer using individual converters of the form \converter eps png sh /home/angus/bin/my_ps2png $$i $$o rather than a catch-all like convertDefault.sh. You could easily add such converters to the /usr/share/lyx/scripts directory together with converter definitions in the global /usr/share/lyx/lyxrc.defaults file and your users will all benefit. -- Angus
Re: LyX terrible converting images
On Tue, 18 Feb 2003, Angus Leeming wrote: Joao B. Oliveira wrote: I have used LyX in -dbg mode and checked the conversion process. It does not seem that LyX itself is doing things badly, but it calls convert for every image on the current screen and convert seems to be doing a bad job, taking up all memory and filling the swap area as well. Wouldn't 'nice' be useful in this situation? /Christian -- Christian Ridderström http://www.md.kth.se/~chr
Re: LyX terrible converting images
On Tuesday 18 February 2003 8:24 pm, Christian Ridderström wrote: On Tue, 18 Feb 2003, Angus Leeming wrote: Joao B. Oliveira wrote: I have used LyX in -dbg mode and checked the conversion process. It does not seem that LyX itself is doing things badly, but it calls convert for every image on the current screen and convert seems to be doing a bad job, taking up all memory and filling the swap area as well. Wouldn't 'nice' be useful in this situation? Sure. But not LyX's business. The user is free to write his own shell script wrapper for convert/gs/whatever. Total freedom to set the nice value as desired. Angus
Re: LyX terrible converting images
On Tue, 18 Feb 2003, Angus Leeming wrote: On Tuesday 18 February 2003 8:24 pm, Christian Ridderström wrote: On Tue, 18 Feb 2003, Angus Leeming wrote: Joao B. Oliveira wrote: I have used LyX in -dbg mode and checked the conversion process. It does not seem that LyX itself is doing things badly, but it calls convert for every image on the current screen and convert seems to be doing a bad job, taking up all memory and filling the swap area as well. Wouldn't 'nice' be useful in this situation? Sure. But not LyX's business. The user is free to write his own shell script wrapper for convert/gs/whatever. Total freedom to set the nice value as desired. Oh, I agree... this was intended as a tip for Joao... (or people in general who suffer from this problem) It's funny though, this mail came directly to me (twice actually ?!?), but I'm guessing that you also sent it to the user's list? /C PS. I'm reading the list via a news. DS -- Christian Ridderström, +46-8-790 91 37 http://www.md.kth.se/~chr Mechatronics lab, Dept. of Machine Designhttp://www.md.kth.se
Re: LyX terrible converting images
On Tue, Feb 18, 2003 at 12:36:35PM +, Angus Leeming wrote: -- First of all, how does the conversion/display process really works? Specially, how EPS files are handled? (I understand that they are always converted to ppm for displaying? Why ppm?) If you run $ lyx -dbg graphics you'll see that the conversion is handled by an external program. You can specify the program you use by defining formats and converters between formats. See the Edit-Preferences dialog. By default, we use ImageMagick's 'convert' program, run through a script convertDefault.sh. I too find that 'convert' is very slow. (But this shouldn't really matter. See above.) I attach a modified convertDefault.sh that uses ghostscript to convert eps files to either ppm or png format. Personally, I would use it as the basis for a my_ps2ppm or my_ps2png converter... This is still bad. We should modify convertDefault.sh to work as a filter, namely 'convertDefault.sh file.eps' should write its output to stdout and not to the disk. In LyX, the necessary modification is to create a pipe when calling to convertDefault.sh. This will be much faster than the current method.
Re: LyX terrible converting images
On Tue, Feb 18, 2003 at 04:19:20PM -0300, Joao B. Oliveira wrote: Of course, Jean-Marc's question about how to get 1.5 Mb EPS images is a valid one. The answer: we have a few programs that create images directly in XFig format. From that they can be edited and/or converted into whatever we want. Unfortunately they contain many rectangles and squares represented as polylines in XFig, and when a polyline is exported it produces a surprisingly large PS code. Have you tried to run xfig's exported .eps files through 'eps2eps'? It makes pretty compact .eps... Andre' -- Those who desire to give up Freedom in order to gain Security, will not have, nor do they deserve, either one. (T. Jefferson)
Re: LyX terrible converting images
On Tue, Feb 18, 2003 at 07:47:22PM +0200, Dekel Tsur wrote: We should modify convertDefault.sh to work as a filter, namely 'convertDefault.sh file.eps' should write its output to stdout and not to the disk. Seconded. And possibly read from stdin, at least if no file argument was given... Andre' -- Those who desire to give up Freedom in order to gain Security, will not have, nor do they deserve, either one. (T. Jefferson)
Re: LyX terrible converting images
Joao B. Oliveira wrote: Hi everyone (specially the folks from the devel team), Hello, Joao. Some history first: I keep LyX running on a Computer Science Department, with some 20-30 students constantly using LyX on several machines. Everything was fine, until LyX 1.2.0 came along. The problem was, the handling of figures was changed in that version, and from that version onwards the students that work on documents with a larger number of figures (or larger figures) were horrified to see that their documents were taking like 10 minutes (sometimes much longer) to be opened, and the convert processes would take over the machine. Simply opening a document and scrolling down to the editing point would create so many conversions and take up so much memory that using the system is impossible. In fact, situation is so bad that I have already given up LyX myself, at least for documents that have more than a few images. The image handling code in LyX 1.2 is indeed nasty. Things should be much better in LyX 1.3. In LyX 1.3 the conversion process is triggered only for images that have been on screen for 2 seconds, so you should now be able to scroll without any conversion occurring. Our machines are neither small nor slow, and LyX (currently 1.3.0) simply makes them stop with the conversion process. I am surprised at this as I would expect only one or two images to be converted at a time (since only one or two images will be visible on the screen at any one time). So, I went after documentation or details but did not find anything that I could use to accelerate things. I really need to understand what is going on, and maybe someone can answer these questions (and ADD THEM to the current documentation): That would be welcome. -- First of all, how does the conversion/display process really works? Specially, how EPS files are handled? (I understand that they are always converted to ppm for displaying? Why ppm?) If you run $ lyx -dbg graphics you'll see that the conversion is handled by an external program. You can specify the program you use by defining formats and converters between formats. See the Edit-Preferences dialog. By default, we use ImageMagick's 'convert' program, run through a script convertDefault.sh. I too find that 'convert' is very slow. (But this shouldn't really matter. See above.) I attach a modified convertDefault.sh that uses ghostscript to convert eps files to either ppm or png format. Personally, I would use it as the basis for a my_ps2ppm or my_ps2png converter... As for the final format, well we chose ppm because both the xforms and Qt frontends can load it natively. If you are using the Qt frontned, then I suggest that you define a converter ps - png. -- In previous versions EPS files could be displayed rather quickly. Why was their processing slower from 1.2.0 on? Because the code was rewritten from scratch to be more flexible. The old code had its own bugs and was unmaintainable. -- Is there a way of avoiding the conversion process and handling EPS files directly? Possibly. It depends on the image loading abilities of the underlying GUI toolkit. The xforms library most certainly CANNOT handle arbitrary PostScript files reliably, so native loading of PostScript files is disabled in the xforms frontend. I don't know about the abilities of the Qt loader. Try modifying the loadableFormats method in src/frontends/qt2/QLImage.C. -- Is there a way of accelerating the conversion process (if it cannot be avoided)? Are there shortcuts to be used? Try using a converter other than ImageMagick's convert. See attached. -- When several images are in the same figure, they *all* get converted at the same time, filling up memory and swap space. At the very least, can this be changed so that we get one conversion at a time, to preserver swap space (and time, as a consequence?) -- If EPS files are large (I have some with more or less 1.5 Mb) their conversion and displaying takes all memory and a 64Mb computer simply crashes. With more images of the same size, larger machines crash as well. Was this foreseen? Of course not. Feel free to add you expertise to the development effort. I would appreciate any help, as I want to keep LyX running but currently I also have to explain many things to both management and disappointed students. Let us know if any of this information helps... -- Angus convertDefault.sh Description: application/shellscript
Re: LyX terrible converting images
Joao == Joao B Oliveira [EMAIL PROTECTED] writes: Joao -- If EPS files are large (I have some with more or less 1.5 Mb) Joao their conversion and displaying takes all memory and a 64Mb Joao computer simply crashes. With more images of the same size, Joao larger machines crash as well. Was this foreseen? The question that comes to mind is: why do you have eps files of 1.5Mb? Where do they come from? Postscript is really not the right format for this kind of images... JMarc
Re: LyX terrible converting images
Jean-Marc Lasgouttes wrote: Joao -- If EPS files are large (I have some with more or less 1.5 Mb) Joao their conversion and displaying takes all memory and a 64Mb Joao computer simply crashes. With more images of the same size, Joao larger machines crash as well. Was this foreseen? The question that comes to mind is: why do you have eps files of 1.5Mb? Where do they come from? Postscript is really not the right format for this kind of images... I have got similar beasts from R in some cases, but then it was possible to use .png images directly, which were much smaller. Matej -- Matej Cepl, Finger: 89EF 4BC6 288A BF43 1BAB 25C3 E09F EF25 D964 84AC 138 Highland Ave. #10, Somerville, Ma 02143, (617) 623-1488 The function of the expert is not to be more right than other people, but to be wrong for more sophisticated reasons. -- Dr. David Butler, British psephologist
Re: LyX terrible converting images
hi everyone, Some history first: I keep LyX running on a Computer Science... The image handling code in LyX 1.2 is indeed nasty. Things should be much better in LyX 1.3. In LyX 1.3 the conversion process is triggered only for images that have been on screen for 2 seconds, so you should now be able to scroll without any conversion occurring. Well, first of all, thanks for the prompt answer to all these questions... one cannot stop wondering at the patience that the development team puts into answering questions. I have to admit, as we just installed LyX 1.3.0, we are still discovering the points that are causing us problems. For example, the problem about a long time on loading a document was reported for the previous version, not (yet) for 1.3.0. Our machines are neither small nor slow, and LyX (currently 1.3.0) simply makes them stop with the conversion process. I am surprised at this as I would expect only one or two images to be converted at a time (since only one or two images will be visible on the screen at any one time). Well, not really. It is rather common (at least here) to put several smaller images in a figure, to draw comparisons like In Fig 6a you see this, and in 6b we clearly see that, but 6c clearly shows Sometimes we put four or six smaller EPS imagens in a figure. (And it does not look that bad :) -- First of all, how does the conversion/display process really works? Specially, how EPS files are handled? (I understand that they are always converted to ppm for displaying? Why ppm?) By default, we use ImageMagick's 'convert' program, run through a script convertDefault.sh. I too find that 'convert' is very slow. (But this shouldn't really matter. See above.) I have used LyX in -dbg mode and checked the conversion process. It does not seem that LyX itself is doing things badly, but it calls convert for every image on the current screen and convert seems to be doing a bad job, taking up all memory and filling the swap area as well. I attach a modified convertDefault.sh that uses ghostscript to convert eps files to either ppm or png format. Personally, I would use it as the basis for a my_ps2ppm or my_ps2png converter... Thank you! As some initial tests are showing, using gs directly seems to be significantly more efficient than convert. Maybe this was the reason for most of our problems all along. I'll invest on that in the next days. To begin, comparing the shell script you sent (that is, gs) with the traditional convert, regarding time and memory consumption... -- Is there a way of accelerating the conversion process (if it cannot be avoided)? Are there shortcuts to be used? Try using a converter other than ImageMagick's convert. See attached. Ok. I think gs will be a resonable bet at the moment. -- If EPS files are large (I have some with more or less 1.5 Mb) their conversion and displaying takes all memory and a 64Mb computer simply crashes. With more images of the same size, larger machines crash as well. Was this foreseen? Of course not. Feel free to add you expertise to the development effort. Sorry. It was a bad choice of words. What I meant was, it was surprising that convert dies in a quite ungraceful way. I would appreciate any help, as I want to keep LyX running but currently I also have to explain many things to both management and disappointed students. Let us know if any of this information helps... Surely it helps, the path is getting clearer now. I suppose that trying other ways to avoid convert would be an interesting option, and I intend to spread the word around here. Of course, Jean-Marc's question about how to get 1.5 Mb EPS images is a valid one. The answer: we have a few programs that create images directly in XFig format. From that they can be edited and/or converted into whatever we want. Unfortunately they contain many rectangles and squares represented as polylines in XFig, and when a polyline is exported it produces a surprisingly large PS code. In fact, there is room for improvement in the tools, trying to avoid polylines or changing them to simpler structures. I'll test that too, as it is possible that 4 straight lines have a much shorter description produced by XFig than a polyline. Well, I have to thank again all people that contribute with suggestions to solve the problem... this is the kind of thing that makes LyX so amazing! thank you all, j. b. oliveira
Re: LyX terrible converting images
Joao B. Oliveira wrote: Well, not really. It is rather common (at least here) to put several smaller images in a figure, to draw comparisons like In Fig 6a you see this, and in 6b we clearly see that, but 6c clearly shows Sometimes we put four or six smaller EPS imagens in a figure. (And it does not look that bad :) -- First of all, how does the conversion/display process really works? Specially, how EPS files are handled? (I understand that they are always converted to ppm for displaying? Why ppm?) By default, we use ImageMagick's 'convert' program, run through a script convertDefault.sh. I too find that 'convert' is very slow. (But this shouldn't really matter. See above.) I have used LyX in -dbg mode and checked the conversion process. It does not seem that LyX itself is doing things badly, but it calls convert for every image on the current screen and convert seems to be doing a bad job, taking up all memory and filling the swap area as well. Sure, at the moment shell scripts are generated for each single conversion process. These conversions therefore execute in parallel. One possible improvement would be to gather all images on the screen into a list and send the list to a shell script for conversion and the conversions would execute in series. Another possible improvement, tied to this, would be to enable communication between LyX and the conversion script so that individual images in this list of many images could be loaded by LyX as they become available. At the moment, LyX only receives notification that the shell script has finished, which it uses as a flag to try and load the .png or .ppm file. Finally, if we enable LyX to use pipes to communicate with the shell script, Dekel's suggestion of loading the image from stdout rather than from a file written to disk becomes possible also. I attach a modified convertDefault.sh that uses ghostscript to convert eps files to either ppm or png format. Personally, I would use it as the basis for a my_ps2ppm or my_ps2png converter... Thank you! As some initial tests are showing, using gs directly seems to be significantly more efficient than convert. Maybe this was the reason for most of our problems all along. I'll invest on that in the next days. To begin, comparing the shell script you sent (that is, gs) with the traditional convert, regarding time and memory consumption... Note that sometimes gs refuses to convert postscript files to bitmap format. However, it will happily 'fix' such problem files if you run them through eps2eps and thereafter has no problems at all. Personally, I prefer using individual converters of the form \converter eps png sh /home/angus/bin/my_ps2png $$i $$o rather than a catch-all like convertDefault.sh. You could easily add such converters to the /usr/share/lyx/scripts directory together with converter definitions in the global /usr/share/lyx/lyxrc.defaults file and your users will all benefit. -- Angus
Re: LyX terrible converting images
On Tue, 18 Feb 2003, Angus Leeming wrote: Joao B. Oliveira wrote: I have used LyX in -dbg mode and checked the conversion process. It does not seem that LyX itself is doing things badly, but it calls convert for every image on the current screen and convert seems to be doing a bad job, taking up all memory and filling the swap area as well. Wouldn't 'nice' be useful in this situation? /Christian -- Christian Ridderström http://www.md.kth.se/~chr
Re: LyX terrible converting images
On Tuesday 18 February 2003 8:24 pm, Christian Ridderström wrote: On Tue, 18 Feb 2003, Angus Leeming wrote: Joao B. Oliveira wrote: I have used LyX in -dbg mode and checked the conversion process. It does not seem that LyX itself is doing things badly, but it calls convert for every image on the current screen and convert seems to be doing a bad job, taking up all memory and filling the swap area as well. Wouldn't 'nice' be useful in this situation? Sure. But not LyX's business. The user is free to write his own shell script wrapper for convert/gs/whatever. Total freedom to set the nice value as desired. Angus
Re: LyX terrible converting images
On Tue, 18 Feb 2003, Angus Leeming wrote: On Tuesday 18 February 2003 8:24 pm, Christian Ridderström wrote: On Tue, 18 Feb 2003, Angus Leeming wrote: Joao B. Oliveira wrote: I have used LyX in -dbg mode and checked the conversion process. It does not seem that LyX itself is doing things badly, but it calls convert for every image on the current screen and convert seems to be doing a bad job, taking up all memory and filling the swap area as well. Wouldn't 'nice' be useful in this situation? Sure. But not LyX's business. The user is free to write his own shell script wrapper for convert/gs/whatever. Total freedom to set the nice value as desired. Oh, I agree... this was intended as a tip for Joao... (or people in general who suffer from this problem) It's funny though, this mail came directly to me (twice actually ?!?), but I'm guessing that you also sent it to the user's list? /C PS. I'm reading the list via a news. DS -- Christian Ridderström, +46-8-790 91 37 http://www.md.kth.se/~chr Mechatronics lab, Dept. of Machine Designhttp://www.md.kth.se
Re: LyX terrible converting images
On Tue, Feb 18, 2003 at 12:36:35PM +, Angus Leeming wrote: -- First of all, how does the conversion/display process really works? Specially, how EPS files are handled? (I understand that they are always converted to ppm for displaying? Why ppm?) If you run $ lyx -dbg graphics you'll see that the conversion is handled by an external program. You can specify the program you use by defining formats and converters between formats. See the Edit-Preferences dialog. By default, we use ImageMagick's 'convert' program, run through a script convertDefault.sh. I too find that 'convert' is very slow. (But this shouldn't really matter. See above.) I attach a modified convertDefault.sh that uses ghostscript to convert eps files to either ppm or png format. Personally, I would use it as the basis for a my_ps2ppm or my_ps2png converter... This is still bad. We should modify convertDefault.sh to work as a filter, namely 'convertDefault.sh file.eps' should write its output to stdout and not to the disk. In LyX, the necessary modification is to create a pipe when calling to convertDefault.sh. This will be much faster than the current method.
Re: LyX terrible converting images
On Tue, Feb 18, 2003 at 04:19:20PM -0300, Joao B. Oliveira wrote: Of course, Jean-Marc's question about how to get 1.5 Mb EPS images is a valid one. The answer: we have a few programs that create images directly in XFig format. From that they can be edited and/or converted into whatever we want. Unfortunately they contain many rectangles and squares represented as polylines in XFig, and when a polyline is exported it produces a surprisingly large PS code. Have you tried to run xfig's exported .eps files through 'eps2eps'? It makes pretty compact .eps... Andre' -- Those who desire to give up Freedom in order to gain Security, will not have, nor do they deserve, either one. (T. Jefferson)
Re: LyX terrible converting images
On Tue, Feb 18, 2003 at 07:47:22PM +0200, Dekel Tsur wrote: We should modify convertDefault.sh to work as a filter, namely 'convertDefault.sh file.eps' should write its output to stdout and not to the disk. Seconded. And possibly read from stdin, at least if no file argument was given... Andre' -- Those who desire to give up Freedom in order to gain Security, will not have, nor do they deserve, either one. (T. Jefferson)
Re: LyX terrible converting images
Joao B. Oliveira wrote: > > Hi everyone (specially the folks from the devel team), Hello, Joao. > Some history first: I keep LyX running on a Computer Science > Department, with some 20-30 students constantly using LyX on several > machines. Everything was fine, until LyX 1.2.0 came along. The > problem was, the handling of figures was changed in that version, > and from that version onwards the students that work on documents > with a larger number of figures (or larger figures) were horrified > to see that their documents were taking like 10 minutes (sometimes > much longer) to be opened, and the convert processes would take > over the machine. Simply opening a document and scrolling down > to the editing point would create so many conversions and take > up so much memory that using the system is impossible. In fact, > situation is so bad that I have already given up LyX myself, at > least for documents that have more than a few images. The image handling code in LyX 1.2 is indeed nasty. Things should be much better in LyX 1.3. In LyX 1.3 the conversion process is triggered only for images that have been on screen for 2 seconds, so you should now be able to scroll without any conversion occurring. > Our machines are neither small nor slow, and LyX (currently 1.3.0) > simply makes them stop with the conversion process. I am surprised at this as I would expect only one or two images to be converted at a time (since only one or two images will be visible on the screen at any one time). > So, I went after documentation or details but did not find anything > that I could use to accelerate things. I really need to understand > what is going on, and maybe someone can answer these questions (and > ADD THEM to the current documentation): That would be welcome. > -- First of all, how does the conversion/display process really > works? Specially, how EPS files are handled? (I understand that > they are always converted to ppm for displaying? Why ppm?) If you run $ lyx -dbg graphics you'll see that the conversion is handled by an external program. You can specify the program you use by defining formats and converters between formats. See the Edit->Preferences dialog. By default, we use ImageMagick's 'convert' program, run through a script convertDefault.sh. I too find that 'convert' is very slow. (But this shouldn't really matter. See above.) I attach a modified convertDefault.sh that uses ghostscript to convert eps files to either ppm or png format. Personally, I would use it as the basis for a my_ps2ppm or my_ps2png converter... As for the final format, well we chose ppm because both the xforms and Qt frontends can load it natively. If you are using the Qt frontned, then I suggest that you define a converter ps -> png. > -- In previous versions EPS files could be displayed rather > quickly. Why was their processing slower from 1.2.0 on? Because the code was rewritten from scratch to be more flexible. The old code had its own bugs and was unmaintainable. > -- Is there a way of avoiding the conversion process and handling > EPS files directly? Possibly. It depends on the image loading abilities of the underlying GUI toolkit. The xforms library most certainly CANNOT handle arbitrary PostScript files reliably, so native loading of PostScript files is disabled in the xforms frontend. I don't know about the abilities of the Qt loader. Try modifying the loadableFormats method in src/frontends/qt2/QLImage.C. > -- Is there a way of accelerating the conversion process (if it > cannot be avoided)? Are there shortcuts to be used? Try using a converter other than ImageMagick's convert. See attached. > -- When several images are in the same figure, they *all* get > converted at the same time, filling up memory and swap space. > At the very least, can this be changed so that we get one conversion > at a time, to preserver swap space (and time, as a consequence?) > > -- If EPS files are large (I have some with more or less 1.5 Mb) > their conversion and displaying takes all memory and a 64Mb computer > simply crashes. With more images of the same size, larger machines > crash as well. Was this foreseen? Of course not. Feel free to add you expertise to the development effort. > I would appreciate any help, as I want to keep LyX running but > currently I also have to explain many things to both management > and disappointed students. Let us know if any of this information helps... -- Angus convertDefault.sh Description: application/shellscript
Re: LyX terrible converting images
> "Joao" == Joao B Oliveira <[EMAIL PROTECTED]> writes: Joao> -- If EPS files are large (I have some with more or less 1.5 Mb) Joao> their conversion and displaying takes all memory and a 64Mb Joao> computer simply crashes. With more images of the same size, Joao> larger machines crash as well. Was this foreseen? The question that comes to mind is: why do you have eps files of 1.5Mb? Where do they come from? Postscript is really not the right format for this kind of images... JMarc
Re: LyX terrible converting images
Jean-Marc Lasgouttes wrote: > Joao> -- If EPS files are large (I have some with more or less 1.5 Mb) > Joao> their conversion and displaying takes all memory and a 64Mb > Joao> computer simply crashes. With more images of the same size, > Joao> larger machines crash as well. Was this foreseen? > > The question that comes to mind is: why do you have eps files of > 1.5Mb? Where do they come from? Postscript is really not the right > format for this kind of images... I have got similar beasts from R in some cases, but then it was possible to use .png images directly, which were much smaller. Matej -- Matej Cepl, Finger: 89EF 4BC6 288A BF43 1BAB 25C3 E09F EF25 D964 84AC 138 Highland Ave. #10, Somerville, Ma 02143, (617) 623-1488 The function of the expert is not to be more right than other people, but to be wrong for more sophisticated reasons. -- Dr. David Butler, British psephologist
Re: LyX terrible converting images
hi everyone, > > Some history first: I keep LyX running on a Computer Science... > > The image handling code in LyX 1.2 is indeed nasty. Things should be much > better in LyX 1.3. In LyX 1.3 the conversion process is triggered only for > images that have been on screen for 2 seconds, so you should now be able to > scroll without any conversion occurring. Well, first of all, thanks for the prompt answer to all these questions... one cannot stop wondering at the patience that the development team puts into answering questions. I have to admit, as we just installed LyX 1.3.0, we are still discovering the points that are causing us problems. For example, the problem about a long time on loading a document was reported for the previous version, not (yet) for 1.3.0. > > Our machines are neither small nor slow, and LyX (currently 1.3.0) > > simply makes them stop with the conversion process. > > I am surprised at this as I would expect only one or two images to be > converted at a time (since only one or two images will be visible on the > screen at any one time). Well, not really. It is rather common (at least here) to put several smaller images in a figure, to draw comparisons like "In Fig 6a you see this, and in 6b we clearly see that, but 6c clearly shows...". Sometimes we put four or six smaller EPS imagens in a figure. (And it does not look that bad :) > > -- First of all, how does the conversion/display process really > > works? Specially, how EPS files are handled? (I understand that > > they are always converted to ppm for displaying? Why ppm?) > > By default, we use ImageMagick's 'convert' program, run through a script > convertDefault.sh. I too find that 'convert' is very slow. (But this > shouldn't really matter. See above.) I have used LyX in -dbg mode and checked the conversion process. It does not seem that LyX itself is doing things badly, but it calls convert for every image on the current screen and convert seems to be doing a bad job, taking up all memory and filling the swap area as well. > I attach a modified convertDefault.sh that uses ghostscript to convert eps > files to either ppm or png format. Personally, I would use it as the basis > for a my_ps2ppm or my_ps2png converter... Thank you! As some initial tests are showing, using gs directly seems to be significantly more efficient than convert. Maybe this was the reason for most of our problems all along. I'll invest on that in the next days. To begin, comparing the shell script you sent (that is, gs) with the traditional convert, regarding time and memory consumption... > > -- Is there a way of accelerating the conversion process (if it > > cannot be avoided)? Are there shortcuts to be used? > > Try using a converter other than ImageMagick's convert. See attached. Ok. I think gs will be a resonable bet at the moment. > > -- If EPS files are large (I have some with more or less 1.5 Mb) > > their conversion and displaying takes all memory and a 64Mb computer > > simply crashes. With more images of the same size, larger machines > > crash as well. Was this foreseen? > > Of course not. Feel free to add you expertise to the development effort. Sorry. It was a bad choice of words. What I meant was, it was surprising that convert dies in a quite ungraceful way. > > I would appreciate any help, as I want to keep LyX running but > > currently I also have to explain many things to both management > > and disappointed students. > > Let us know if any of this information helps... Surely it helps, the path is getting clearer now. I suppose that trying other ways to avoid convert would be an interesting option, and I intend to spread the word around here. Of course, Jean-Marc's question about how to get 1.5 Mb EPS images is a valid one. The answer: we have a few programs that create images directly in XFig format. From that they can be edited and/or converted into whatever we want. Unfortunately they contain many rectangles and squares represented as "polylines" in XFig, and when a polyline is exported it produces a surprisingly large PS code. In fact, there is room for improvement in the tools, trying to avoid polylines or changing them to simpler structures. I'll test that too, as it is possible that 4 straight lines have a much shorter description produced by XFig than a polyline. Well, I have to thank again all people that contribute with suggestions to solve the problem... this is the kind of thing that makes LyX so amazing! thank you all, j. b. oliveira
Re: LyX terrible converting images
Joao B. Oliveira wrote: > Well, not really. It is rather common (at least here) to put > several smaller images in a figure, to draw comparisons like "In > Fig 6a you see this, and in 6b we clearly see that, but 6c clearly > shows...". Sometimes we put four or six smaller EPS imagens in a > figure. (And it does not look that bad :) > >> > -- First of all, how does the conversion/display process really >> > works? Specially, how EPS files are handled? (I understand that >> > they are always converted to ppm for displaying? Why ppm?) >> >> By default, we use ImageMagick's 'convert' program, run through a script >> convertDefault.sh. I too find that 'convert' is very slow. (But this >> shouldn't really matter. See above.) > > I have used LyX in -dbg mode and checked the conversion process. It > does not seem that LyX itself is doing things badly, but it calls > convert for every image on the current screen and convert seems to > be doing a bad job, taking up all memory and filling the swap area > as well. Sure, at the moment shell scripts are generated for each single conversion process. These conversions therefore execute in parallel. One possible improvement would be to gather all images on the screen into a list and send the list to a shell script for conversion and the conversions would execute in series. Another possible improvement, tied to this, would be to enable communication between LyX and the conversion script so that individual images in this list of many images could be loaded by LyX as they become available. At the moment, LyX only receives notification that the shell script has finished, which it uses as a flag to try and load the .png or .ppm file. Finally, if we enable LyX to use pipes to communicate with the shell script, Dekel's suggestion of loading the image from stdout rather than from a file written to disk becomes possible also. >> I attach a modified convertDefault.sh that uses ghostscript to convert >> eps files to either ppm or png format. Personally, I would use it as the >> basis for a my_ps2ppm or my_ps2png converter... > > Thank you! As some initial tests are showing, using gs directly > seems to be significantly more efficient than convert. Maybe this > was the reason for most of our problems all along. I'll invest on > that in the next days. To begin, comparing the shell script you > sent (that is, gs) with the traditional convert, regarding time > and memory consumption... Note that sometimes gs refuses to convert postscript files to bitmap format. However, it will happily 'fix' such problem files if you run them through eps2eps and thereafter has no problems at all. Personally, I prefer using individual converters of the form \converter "eps" "png" "sh /home/angus/bin/my_ps2png $$i $$o" "" rather than a catch-all like convertDefault.sh. You could easily add such converters to the /usr/share/lyx/scripts directory together with converter definitions in the global /usr/share/lyx/lyxrc.defaults file and your users will all benefit. -- Angus
Re: LyX terrible converting images
On Tue, 18 Feb 2003, Angus Leeming wrote: > Joao B. Oliveira wrote: > > I have used LyX in -dbg mode and checked the conversion process. It > > does not seem that LyX itself is doing things badly, but it calls > > convert for every image on the current screen and convert seems to > > be doing a bad job, taking up all memory and filling the swap area > > as well. Wouldn't 'nice' be useful in this situation? /Christian -- Christian Ridderström http://www.md.kth.se/~chr
Re: LyX terrible converting images
On Tuesday 18 February 2003 8:24 pm, Christian Ridderström wrote: > On Tue, 18 Feb 2003, Angus Leeming wrote: > > Joao B. Oliveira wrote: > > > I have used LyX in -dbg mode and checked the conversion process. It > > > does not seem that LyX itself is doing things badly, but it calls > > > convert for every image on the current screen and convert seems to > > > be doing a bad job, taking up all memory and filling the swap area > > > as well. > > Wouldn't 'nice' be useful in this situation? Sure. But not LyX's business. The user is free to write his own shell script wrapper for convert/gs/whatever. Total freedom to set the nice value as desired. Angus
Re: LyX terrible converting images
On Tue, 18 Feb 2003, Angus Leeming wrote: > On Tuesday 18 February 2003 8:24 pm, Christian Ridderström wrote: > > On Tue, 18 Feb 2003, Angus Leeming wrote: > > > Joao B. Oliveira wrote: > > > > I have used LyX in -dbg mode and checked the conversion process. It > > > > does not seem that LyX itself is doing things badly, but it calls > > > > convert for every image on the current screen and convert seems to > > > > be doing a bad job, taking up all memory and filling the swap area > > > > as well. > > > > Wouldn't 'nice' be useful in this situation? > > Sure. But not LyX's business. The user is free to write his own shell script > wrapper for convert/gs/whatever. Total freedom to set the nice value as > desired. Oh, I agree... this was intended as a tip for Joao... (or people in general who suffer from this problem) It's funny though, this mail came directly to me (twice actually ?!?), but I'm guessing that you also sent it to the user's list? /C PS. I'm reading the list via a news. DS -- Christian Ridderström, +46-8-790 91 37 http://www.md.kth.se/~chr Mechatronics lab, Dept. of Machine Designhttp://www.md.kth.se
Re: LyX terrible converting images
On Tue, Feb 18, 2003 at 12:36:35PM +, Angus Leeming wrote: > > -- First of all, how does the conversion/display process really > > works? Specially, how EPS files are handled? (I understand that > > they are always converted to ppm for displaying? Why ppm?) > > If you run > $ lyx -dbg graphics > you'll see that the conversion is handled by an external program. You can > specify the program you use by defining formats and converters between > formats. See the Edit->Preferences dialog. > > By default, we use ImageMagick's 'convert' program, run through a script > convertDefault.sh. I too find that 'convert' is very slow. (But this > shouldn't really matter. See above.) > > I attach a modified convertDefault.sh that uses ghostscript to convert eps > files to either ppm or png format. Personally, I would use it as the basis > for a my_ps2ppm or my_ps2png converter... This is still bad. We should modify convertDefault.sh to work as a filter, namely 'convertDefault.sh file.eps' should write its output to stdout and not to the disk. In LyX, the necessary modification is to create a pipe when calling to convertDefault.sh. This will be much faster than the current method.
Re: LyX terrible converting images
On Tue, Feb 18, 2003 at 04:19:20PM -0300, Joao B. Oliveira wrote: > Of course, Jean-Marc's question about how to get 1.5 Mb EPS images > is a valid one. The answer: we have a few programs that create > images directly in XFig format. From that they can be edited and/or > converted into whatever we want. Unfortunately they contain many > rectangles and squares represented as "polylines" in XFig, and when > a polyline is exported it produces a surprisingly large PS code. Have you tried to run xfig's exported .eps files through 'eps2eps'? It makes pretty compact .eps... Andre' -- Those who desire to give up Freedom in order to gain Security, will not have, nor do they deserve, either one. (T. Jefferson)
Re: LyX terrible converting images
On Tue, Feb 18, 2003 at 07:47:22PM +0200, Dekel Tsur wrote: > We should modify convertDefault.sh to work as a filter, namely > 'convertDefault.sh file.eps' should write its output to stdout and not to > the disk. Seconded. And possibly read from stdin, at least if no file argument was given... Andre' -- Those who desire to give up Freedom in order to gain Security, will not have, nor do they deserve, either one. (T. Jefferson)
