[libreoffice-users] Re: Intentionally crashing LibreOffice when frozen/LibreOffice will not start.
Dear Tom -- Thanks for your attention to my problem. I believe the problem with using Calc as a viewer is that I have 3.6 million lines of data. But I am getting more comfortable with using R itself to answer these questions. I think part of my problem is that it is the nature of of legal training to try to andticipate every possible problem. But when the number of possible problems exceeds a certain level, my brain runs out of working memory even if my computer still has RAM. Also, I set ambitious goals for allowing people to run my software on ordinary laptops a year or three old. I have to say that since I have scaled those back and got 24 gig of RAM instead, many problems seem more manageable, even without a database. Peace, Andrew On Fri, Apr 18, 2014 at 6:39 AM, TomD [via Document Foundation Mail Archive] ml-node+s969070n4105676...@n3.nabble.com wrote: Hi :) Ahh, Base seems to be better when the data is held externally. Postgresql should be an excellent choice for holding the data and then 'just' get Base to connect to it. I would have thought Calc would then be really good at displaying the data?! So i'm not sure what the problem is now. Hopefully someone better with databases might be more helpful! Errr, most people on this mailing list seem to be better with databases than me! Good luck and regards from Tom :) On 17 April 2014 22:56, andrewH [hidden email]http://user/SendEmail.jtp?type=nodenode=4105676i=0 wrote: Thanks Tom! Gnumeric is a great product and I have used it before. I was hoping to use Calc in this case because I am trying to learn to use the LO database, Base, as a stand-alone or a front end for PostgreSQL. But I find it very hard to define, use, and even just to import a file into a database if I can not first look at it and determine how missing variables are coded, which fields are character and which are numeric, etc. Some census products are really good at giving users this kind of metadata, others not so much. (The Economic Census metadata is hard enough to read and understand (would you put material intended to explain something to the public in pipe-delimited text?) that I have written to them asking for meta-meta data). So I wanted to use Calc for exploration and base to do the heavy lifting, in the hope that things might be easier if I stayed withing one document family. Now I am thinking that some other approach will be easier. Warmest regards, Andrew On Sat, Apr 5, 2014 at 2:09 AM, TomD [via Document Foundation Mail Archive] [hidden email] http://user/SendEmail.jtp?type=nodenode=4105676i=1 wrote: Hi :) Try Gnumeric; http://www.gnumeric.org/download.html It's a dedicated spreadsheet program with a tiny footprint that uses minimal resources, so it's faster, lighter and more robust than Excel or Calc. Many people find Gnumeric to be better than Calc or Excel for serious or hefty spreadsheets and/or for handling many more spreadsheets in a shorter time-frame. It can be installed alongside LibreOffice and/or MS Office. It uses the same format as LibreOffice natively so most spreadsheets can be bounced between the 2 programs quite happily. Part of the advantage of LibreOffice is that it fits well into a wider eco-system and co-operates well with a wider range of programs and suites allowing you to tailor individual machines to specific use-cases and yet still retain the ability to share files between different machines and different people using different OSes and programs. Regards from Tom ) On 5 April 2014 06:43, Andrew Douglas Pitonyak [hidden email] http://user/SendEmail.jtp?type=nodenode=4104190i=0 wrote: On 04/04/2014 07:44 PM, andrewH wrote: I am working with a data set that keeps causing my LibreOffice to freeze. I am pretty sure that this is only because it is big. It is a pipe-separated text from the US Economic Census imported into Calc, about 30 columns and around a million rows. (The actual data set is bigger, but Calc quits at a million-odd. The complete file is about 0.8 gig.) I suspect but can not prove that this is related to file handling somehow, e.g. breaking down during auto-saving. The first time I saved the data as a Calc file it took nearly an hour with the soffice.bin *32 process running at 25 percent of CPU time and using about 825 meg of memory the entire time. (Not sure why this is showing up as a 32-bit version). And when Calc freezes, all the LibreOffice programs freeze. So I can't just switch to another file and noodle away while waiting. It looks to me as though Calc cannot handle more than 1048576 rows of data, Do you have more rows than that? If yes, then I think that you cannot open the file. If you have less than that, and, if you
[libreoffice-users] Re: Intentionally crashing LibreOffice when frozen/LibreOffice will not start.
Thanks for the offer, Andrew! The original file is definitely bigger than the allowable limit. However, when I originally tried to import it, it opened up the first third of the file and allowed me to save it, and reopen it. However, when I tried to save it a second time, it crashed and the file was corrupted. I am pretty sure the problem is not that I am running out of memory. I have 24 gig on a 64-bit machine. I think the program is just buggy when you try to run it at its absolute outside capacity limit. So I have figured out how to open my file in R and save subsets small enough to open more reliably. I wish the program checked for file size and offered a sensible list of options when it is too big, like opening the first N rows, or opening n rows after skipping m rows, or opening a taking a p percent sample of rows. But perhaps that is too much to hope for in a spreadsheet. And thanks again for your thought and attention. Warmest regards, Andrew On Fri, Apr 4, 2014 at 10:44 PM, Andrew Douglas Pitonyak [via Document Foundation Mail Archive] ml-node+s969070n410416...@n3.nabble.com wrote: On 04/04/2014 07:44 PM, andrewH wrote: I am working with a data set that keeps causing my LibreOffice to freeze. I am pretty sure that this is only because it is big. It is a pipe-separated text from the US Economic Census imported into Calc, about 30 columns and around a million rows. (The actual data set is bigger, but Calc quits at a million-odd. The complete file is about 0.8 gig.) I suspect but can not prove that this is related to file handling somehow, e.g. breaking down during auto-saving. The first time I saved the data as a Calc file it took nearly an hour with the soffice.bin *32 process running at 25 percent of CPU time and using about 825 meg of memory the entire time. (Not sure why this is showing up as a 32-bit version). And when Calc freezes, all the LibreOffice programs freeze. So I can't just switch to another file and noodle away while waiting. It looks to me as though Calc cannot handle more than 1048576 rows of data, Do you have more rows than that? If yes, then I think that you cannot open the file. If you have less than that, and, if you think that you are simply running out of memory if you can figure out how to get the data to me, I can run a test on a 64 bit version (running on Linux). My machine has 32 GB of RAM... -- Andrew Pitonyak My Macro Document: http://www.pitonyak.org/AndrewMacro.odt Info: http://www.pitonyak.org/oo.php -- To unsubscribe e-mail to: [hidden email]http://user/SendEmail.jtp?type=nodenode=4104166i=0 Problems? http://www.libreoffice.org/get-help/mailing-lists/how-to-unsubscribe/ Posting guidelines + more: http://wiki.documentfoundation.org/Netiquette List archive: http://listarchives.libreoffice.org/global/users/ All messages sent to this list will be publicly archived and cannot be deleted -- If you reply to this email, your message will be added to the discussion below: http://nabble.documentfoundation.org/Intentionally-crashing-LibreOffice-when-frozen-LibreOffice-will-not-start-tp4104156p4104166.html To unsubscribe from Intentionally crashing LibreOffice when frozen/LibreOffice will not start., click herehttp://nabble.documentfoundation.org/template/NamlServlet.jtp?macro=unsubscribe_by_codenode=4104156code=YWhvZXJuZXJAcnByb2dyZXNzLm9yZ3w0MTA0MTU2fC0yMDQ3NjI1NDM5 . NAMLhttp://nabble.documentfoundation.org/template/NamlServlet.jtp?macro=macro_viewerid=instant_html%21nabble%3Aemail.namlbase=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.template.NodeNamespacebreadcrumbs=notify_subscribers%21nabble%3Aemail.naml-instant_emails%21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml -- J. Andrew Hoerner Director, Sustainable Economics Program Redefining Progress (510) 507-4820 -- View this message in context: http://nabble.documentfoundation.org/Intentionally-crashing-LibreOffice-when-frozen-LibreOffice-will-not-start-tp4104156p4105609.html Sent from the Users mailing list archive at Nabble.com. -- To unsubscribe e-mail to: users+unsubscr...@global.libreoffice.org Problems? http://www.libreoffice.org/get-help/mailing-lists/how-to-unsubscribe/ Posting guidelines + more: http://wiki.documentfoundation.org/Netiquette List archive: http://listarchives.libreoffice.org/global/users/ All messages sent to this list will be publicly archived and cannot be deleted
[libreoffice-users] Re: Intentionally crashing LibreOffice when frozen/LibreOffice will not start.
Thanks Tom! Gnumeric is a great product and I have used it before. I was hoping to use Calc in this case because I am trying to learn to use the LO database, Base, as a stand-alone or a front end for PostgreSQL. But I find it very hard to define, use, and even just to import a file into a database if I can not first look at it and determine how missing variables are coded, which fields are character and which are numeric, etc. Some census products are really good at giving users this kind of metadata, others not so much. (The Economic Census metadata is hard enough to read and understand (would you put material intended to explain something to the public in pipe-delimited text?) that I have written to them asking for meta-meta data). So I wanted to use Calc for exploration and base to do the heavy lifting, in the hope that things might be easier if I stayed withing one document family. Now I am thinking that some other approach will be easier. Warmest regards, Andrew On Sat, Apr 5, 2014 at 2:09 AM, TomD [via Document Foundation Mail Archive] ml-node+s969070n4104190...@n3.nabble.com wrote: Hi :) Try Gnumeric; http://www.gnumeric.org/download.html It's a dedicated spreadsheet program with a tiny footprint that uses minimal resources, so it's faster, lighter and more robust than Excel or Calc. Many people find Gnumeric to be better than Calc or Excel for serious or hefty spreadsheets and/or for handling many more spreadsheets in a shorter time-frame. It can be installed alongside LibreOffice and/or MS Office. It uses the same format as LibreOffice natively so most spreadsheets can be bounced between the 2 programs quite happily. Part of the advantage of LibreOffice is that it fits well into a wider eco-system and co-operates well with a wider range of programs and suites allowing you to tailor individual machines to specific use-cases and yet still retain the ability to share files between different machines and different people using different OSes and programs. Regards from Tom ) On 5 April 2014 06:43, Andrew Douglas Pitonyak [hidden email]http://user/SendEmail.jtp?type=nodenode=4104190i=0 wrote: On 04/04/2014 07:44 PM, andrewH wrote: I am working with a data set that keeps causing my LibreOffice to freeze. I am pretty sure that this is only because it is big. It is a pipe-separated text from the US Economic Census imported into Calc, about 30 columns and around a million rows. (The actual data set is bigger, but Calc quits at a million-odd. The complete file is about 0.8 gig.) I suspect but can not prove that this is related to file handling somehow, e.g. breaking down during auto-saving. The first time I saved the data as a Calc file it took nearly an hour with the soffice.bin *32 process running at 25 percent of CPU time and using about 825 meg of memory the entire time. (Not sure why this is showing up as a 32-bit version). And when Calc freezes, all the LibreOffice programs freeze. So I can't just switch to another file and noodle away while waiting. It looks to me as though Calc cannot handle more than 1048576 rows of data, Do you have more rows than that? If yes, then I think that you cannot open the file. If you have less than that, and, if you think that you are simply running out of memory if you can figure out how to get the data to me, I can run a test on a 64 bit version (running on Linux). My machine has 32 GB of RAM... -- Andrew Pitonyak My Macro Document: http://www.pitonyak.org/AndrewMacro.odt Info: http://www.pitonyak.org/oo.php -- To unsubscribe e-mail to: [hidden email]http://user/SendEmail.jtp?type=nodenode=4104190i=1 Problems? http://www.libreoffice.org/get-help/mailing-lists/how-to-unsubscribe/ Posting guidelines + more: http://wiki.documentfoundation.org/Netiquette List archive: http://listarchives.libreoffice.org/global/users/ All messages sent to this list will be publicly archived and cannot be deleted -- To unsubscribe e-mail to: [hidden email]http://user/SendEmail.jtp?type=nodenode=4104190i=2 Problems? http://www.libreoffice.org/get-help/mailing-lists/how-to-unsubscribe/ Posting guidelines + more: http://wiki.documentfoundation.org/Netiquette List archive: http://listarchives.libreoffice.org/global/users/ All messages sent to this list will be publicly archived and cannot be deleted -- If you reply to this email, your message will be added to the discussion below: http://nabble.documentfoundation.org/Intentionally-crashing-LibreOffice-when-frozen-LibreOffice-will-not-start-tp4104156p4104190.html To unsubscribe from Intentionally crashing LibreOffice when frozen/LibreOffice will not start., click herehttp://nabble.documentfoundation.org/template/NamlServlet.jtp?macro=unsubscribe_by_codenode=4104156code=YWhvZXJuZXJAcnByb2dyZXNzLm9yZ3w0MTA0MTU2fC0yMDQ3NjI1NDM5 . NAMLhttp
[libreoffice-users] Intentionally crashing LibreOffice when frozen/LibreOffice will not start.
I am working with a data set that keeps causing my LibreOffice to freeze. I am pretty sure that this is only because it is big. It is a pipe-separated text from the US Economic Census imported into Calc, about 30 columns and around a million rows. (The actual data set is bigger, but Calc quits at a million-odd. The complete file is about 0.8 gig.) I suspect but can not prove that this is related to file handling somehow, e.g. breaking down during auto-saving. The first time I saved the data as a Calc file it took nearly an hour with the soffice.bin *32 process running at 25 percent of CPU time and using about 825 meg of memory the entire time. (Not sure why this is showing up as a 32-bit version). And when Calc freezes, all the LibreOffice programs freeze. So I can't just switch to another file and noodle away while waiting. So of course, what I really want is a spreadsheet that reads more rows and doesn't crash. But I assume others are already asking for that, do not know how to add anything helpful to that effort. So instead I am asking a more modest question: What is the best way to crash a frozen LibreOffice if I want to minimize the data and files lost? Note that this is the question I started with. Now I have a new and more urgent question: Why won't LibreOffice start at all? See below. When Calc froze, I closed it in the usual way with the usual upper-right close window button. This results in the message: The file '$(ARG1)' is corrupt and therefore cannot be opened. LibreOffice can try to repair the file. LO can try to repair the file. The corruption could be the result of document manipulation or of structural document damage due to data transmission. We recommend that you do not trust the content of the repaired document. Execution of macros is disabled for this document. Should LO repair the file? I choose Yes Message The file '$(ARG1)' could not be repaired and therefore cannot be opened. The document recovery window said recovery of your documents was finished while the big spreadsheet still said recovery in process and a Writer file said not recovered yet. Three Writer files said recovery successfulBecause no LibreOffice process seemed to be using much CPU time, I assumed that it was again frozen rather than still working (though soffice.bin *32 still had 1.14 gig in memory) and hit the finish button. I got a message, Fatal Error - Bad Allocation and nothing was recovered. I tried opening Writer, and the document recovery box popped up again, but with one fewer Writer files listed. This time it started on the spreadsheet, never finished (though the window again said recovery of your documents was finished) and never got to the remaining files, though I let it run all night. In the morning I hit finis, and the recovery window disappeared with nothing taking its place. I tried to start Writer again. Nothing happened - no error, no window, nothing. I tried opening a Writer document by clicking on it in Windows Explorer. Nothing. I tried opening a (different) Calc file. Windows Task Manager shows five instances of soffice.bin *32 running, five instances of soffice.exe * 32, and four instances of swriter.exe *32 I am running LibreOffice 4.2 on Windows 7 64-bit with SP1, on a Dell i5 machine with 24 gig of RAM. I do not recall there being a separate 64-bit version, but the installer put it in my 64-bit programs directory. Advice and suggestions most welcome. -- View this message in context: http://nabble.documentfoundation.org/Intentionally-crashing-LibreOffice-when-frozen-LibreOffice-will-not-start-tp4104156.html Sent from the Users mailing list archive at Nabble.com. -- To unsubscribe e-mail to: users+unsubscr...@global.libreoffice.org Problems? http://www.libreoffice.org/get-help/mailing-lists/how-to-unsubscribe/ Posting guidelines + more: http://wiki.documentfoundation.org/Netiquette List archive: http://listarchives.libreoffice.org/global/users/ All messages sent to this list will be publicly archived and cannot be deleted
