Re: [tdf-discuss] Huge problems with big documents
Hi Georgio, Giorgio Migliaccio wrote (07-02-13 09:07) We’re using LibreOffice (since v2.x up to 4.0 beta) inside our product LetterSketch, which was started 7 years ago. [...] Good to read from you, after we've met in Gouda, last year! [...] Is there any kind of suggestion you can give us to overcome this problem/limitation, since this is going to be a major showstopper for our project. We know that there already have been considerable memory usage improvements in LibreOffice after the making free ... Looking at your use case, I would seriously suggest you to get in contact with a developer knowing 'that code area', or with a company sponsoring some of those guys (or girls). We have various active in TDF. It's one of the many advantages of LibreOffice / TDF: that you can choose a route at your choise, to get improvements in. And especially when there is a interesting, big, business case, that is both healthy for your product and the LibreOffice project. Kind regards, Cor -- - Cor - http://nl.libreoffice.org - www.librelex.org -- Unsubscribe instructions: E-mail to discuss+h...@documentfoundation.org Problems? http://www.libreoffice.org/get-help/mailing-lists/how-to-unsubscribe/ Posting guidelines + more: http://wiki.documentfoundation.org/Netiquette List archive: http://listarchives.documentfoundation.org/www/discuss/ All messages sent to this list will be publicly archived and cannot be deleted
RE: [tdf-discuss] Huge problems with big documents
Our customers are just normal non-technical end-users which are used to work with MS Word for creating documents. We provide them with a solution where they an continue to work with something like they already know, LibreOffice replacing MS Word, and extend it with all kind of automation possibilities. Any other offering wouldn't be accepted by our customers, because it would mean a major training is needed for hundreds to thousands of employees, I'm sure I don't have to tell you how much money that would cost, right? You just have to learn to live with the fact that not every person on this planet has sufficient technical background to start working with applications like Lyx, how powerful such applications even might be. Conditions, loops and variables are just point and click stuff in LetterSketch, so that most non-technical people can build such templates. And believe it or not, we are just one of a couple of hundred(thousands?) companies worldwide doing this, with competitors like HP, IBM and DOC1 being market leaders, and such products are being used by any (really *any*) major bank, insurance company, or whatever company in the world for customer or internal communication... that's just a fact We, and our customers, use XML for transferring data. The data in the XML however can come from databases, Salesforce.com, Excel files, web forms, Navision, and then some. Some of them might have a DB behind them, but most of the the large banks or insurance companies won't give external parties direct access to their DB's...you must surely know that. Using Sax XML processing, complex documents can be generated in milliseconds, making it possible to generate millions of documents in 24 hours. Not too bad I would say. The 113 pages is just the template from which the generations will start. This can result in a 5 page document or even in a 10.000 page document, depending on the data and business rules. Regards, Giorgio -- Unsubscribe instructions: E-mail to discuss+h...@documentfoundation.org Problems? http://www.libreoffice.org/get-help/mailing-lists/how-to-unsubscribe/ Posting guidelines + more: http://wiki.documentfoundation.org/Netiquette List archive: http://listarchives.documentfoundation.org/www/discuss/ All messages sent to this list will be publicly archived and cannot be deleted
Re: [tdf-discuss] Huge problems with big documents
> LetterSketch is a document authoring tool with which authors can > create documents and extend it with conditions, loops, variables, sub > documents and other ornaments. Sounds like *the* use case where LyX/LaTeX is beyond competition. Because it has all this already built-in. > Finally this template gets compiled to an internal format and finally > it can be generated on a server in high volume by feeding it data in > XML format. Data is best fed from a database (LyX/LaTeX has "native" solutions for this as well). XML was never made for data and it's hopelessly inefficient (both in terms of volume as well as required processing bandwidth) as a format for large amounts of data. > At the customer we created a template/document containing 113 pages, > built up with plain text, various objects like comments and frames, > and dozens of sub-documents and where it takes some 10-20 minutes to > just open the template. 113 pages is not at all a "big" document. Unless that's just the "root" document. > Please find attached the concerning (main) document. This list doesn't allow attachments. Sincerely, Wolfgang -- Unsubscribe instructions: E-mail to discuss+h...@documentfoundation.org Problems? http://www.libreoffice.org/get-help/mailing-lists/how-to-unsubscribe/ Posting guidelines + more: http://wiki.documentfoundation.org/Netiquette List archive: http://listarchives.documentfoundation.org/www/discuss/ All messages sent to this list will be publicly archived and cannot be deleted
Re: [tdf-discuss] Huge problems with big documents
Hi, Bug filed: https://bugs.freedesktop.org/show_bug.cgi?id=60418 Yours Florian Am 07.02.2013 10:58, schrieb Giorgio Migliaccio: Hi, Please find attached the document (one of many alike). With kind regards, Giorgio Migliaccio Product Development Manager Telephone : +32 3 450 89 84 Mobile : +32 486 92 43 41 Email : giorgio.migliac...@lettergen.be LetterGen Groene Hofstraat 13, 2850 Boom, Belgium www.lettergen.com -Original Message- From: Florian Reisinger [mailto:reisi...@gmail.com] Sent: donderdag 7 februari 2013 10:48 To: Giorgio Migliaccio Cc: discuss@documentfoundation.org Subject: Re: [tdf-discuss] Huge problems with big documents Hi, The document is not attached. Please send me the document to me as PM... Yours, Florian Am 07.02.2013 09:07, schrieb Giorgio Migliaccio: Hi all, We’re using LibreOffice (since v2.x up to 4.0 beta) inside our product LetterSketch, which was started 7 years ago. LetterSketch is a document authoring tool with which authors can create documents and extend it with conditions, loops, variables, sub documents and other ornaments. Finally this template gets compiled to an internal format and finally it can be generated on a server in high volume by feeding it data in XML format. Now we got our biggest implementation up to date, a major European bank. And they need to create pretty complex document structures, containing multiple sub-documents. And here we reached the limits of the OpenDocument format, or at least the LibreOffice/OpenOffice internal object representation. At the customer we created a template/document containing 113 pages, built up with plain text, various objects like comments and frames, and dozens of sub-documents and where it takes some 10-20 minutes to just open the template. We finally pinpointed the problem. When we just take the top-level odt file, without resolving any of the sub-documents, LibreOffice needs 1 minute at 100% CPU(intel i5 CPU) and 500 MB of memory (!!!) for this document alone. So when we also start resolving the sub-documents, LibreOffice goes up to 1.8 GB of memory and then just crashes or disappears or freezes, there seems to be some kind of invisible ceiling. Closing the document takes another 1-2 minutes and some additional memory is needed! When saving the main document to the Word (.doc) format and then opening it in Word, MS Word only needs some 30 MB of memory to visualize this SAME document. Please find attached the concerning (main) document. Is there any kind of suggestion you can give us to overcome this problem/limitation, since this is going to be a major showstopper for our project. Thank you very much for your suggestions. With kind regards, Giorgio Migliaccio -- Unsubscribe instructions: E-mail to discuss+h...@documentfoundation.org Problems? http://www.libreoffice.org/get-help/mailing-lists/how-to-unsubscribe/ Posting guidelines + more: http://wiki.documentfoundation.org/Netiquette List archive: http://listarchives.documentfoundation.org/www/discuss/ All messages sent to this list will be publicly archived and cannot be deleted
Re: [tdf-discuss] Huge problems with big documents
On 2/7/13 9:07 AM, Giorgio Migliaccio wrote: > Please find attached the concerning (main) document. Mailing lists do not accept attachments. You should file a bug on Bugzilla, where you can describe the issue and attach the document. -- Italo Vignoli - italo.vign...@gmail.com mob +39.348.5653829 - VoIP 5316...@messagenet.it skype italovignoli - gtalk italo.vign...@gmail.com -- Unsubscribe instructions: E-mail to discuss+h...@documentfoundation.org Problems? http://www.libreoffice.org/get-help/mailing-lists/how-to-unsubscribe/ Posting guidelines + more: http://wiki.documentfoundation.org/Netiquette List archive: http://listarchives.documentfoundation.org/www/discuss/ All messages sent to this list will be publicly archived and cannot be deleted
Re: [tdf-discuss] Huge problems with big documents
Hi, The document is not attached. Please send me the document to me as PM... Yours, Florian Am 07.02.2013 09:07, schrieb Giorgio Migliaccio: Hi all, We’re using LibreOffice (since v2.x up to 4.0 beta) inside our product LetterSketch, which was started 7 years ago. LetterSketch is a document authoring tool with which authors can create documents and extend it with conditions, loops, variables, sub documents and other ornaments. Finally this template gets compiled to an internal format and finally it can be generated on a server in high volume by feeding it data in XML format. Now we got our biggest implementation up to date, a major European bank. And they need to create pretty complex document structures, containing multiple sub-documents. And here we reached the limits of the OpenDocument format, or at least the LibreOffice/OpenOffice internal object representation. At the customer we created a template/document containing 113 pages, built up with plain text, various objects like comments and frames, and dozens of sub-documents and where it takes some 10-20 minutes to just open the template. We finally pinpointed the problem. When we just take the top-level odt file, without resolving any of the sub-documents, LibreOffice needs 1 minute at 100% CPU(intel i5 CPU) and 500 MB of memory (!!!) for this document alone. So when we also start resolving the sub-documents, LibreOffice goes up to 1.8 GB of memory and then just crashes or disappears or freezes, there seems to be some kind of invisible ceiling. Closing the document takes another 1-2 minutes and some additional memory is needed! When saving the main document to the Word (.doc) format and then opening it in Word, MS Word only needs some 30 MB of memory to visualize this SAME document. Please find attached the concerning (main) document. Is there any kind of suggestion you can give us to overcome this problem/limitation, since this is going to be a major showstopper for our project. Thank you very much for your suggestions. With kind regards, Giorgio Migliaccio -- Unsubscribe instructions: E-mail to discuss+h...@documentfoundation.org Problems? http://www.libreoffice.org/get-help/mailing-lists/how-to-unsubscribe/ Posting guidelines + more: http://wiki.documentfoundation.org/Netiquette List archive: http://listarchives.documentfoundation.org/www/discuss/ All messages sent to this list will be publicly archived and cannot be deleted
