Cross posted back to the PLUG list - follow up there > On Tue, Oct 11, 2011 at 02:16:43PM -0700, Rich Shepard wrote: > > I've saved copies as LibreOffice .ods files, and deleted the 33 sheets of > > graphs so I have only the data sheet, but each one is about 280 columns by > > 200 rows. What I need to do is extract from the latest version only those > > data not present in the original version, and I'm open to suggestions on how > > to proceed in the least painful way. On Tue, Oct 11, 2011 at 02:39:23PM -0700, Aaron Burt wrote: > For comparing spreadsheets, I use the "Edit->Compare Document..." menu > item. You get an itemized list of changes and a spreadsheet with > differences highlighted in red. I always get confused by the direction of > the comparison, so be aware, test things and keep backups. > > If you expect to receive consistently-formatted XLS spreadsheets weekly > or more often, it might be worth the time to use the xlrd Python module: > http://scienceoss.com/read-excel-files-from-python/ > > Newer versions of LibreOffice can open and read XLSX files with > 65,000 > rows, but it doesn't appear to write them.
In support of Aaron's excellent reply, I want to add that for large volumes of data, spreadsheets are not the best tool. I haven't used xlrd (or the likely Perl equivalent) but I have exported spreadsheets as CVS and processed them with 10 line Perl scripts, even vi sometimes. All spreadsheet programs make really crummy graphs - gnuplot is far superior. All spreadsheet programs have difficulty with recursion and complex formulas. They have a hard time branching, and handling regular expressions for text matching, etc. At some point, you must invest a little time in learning to do "simple" programming. I use Perl mostly because many friends do, but Python is a good option for most. You can ignore 90% of either language, as long as you learn how to read in data, write out data, make simple tables, and do arithmetic. Plagarizing other programs is the best way to learn, which is why you have a disk-full of sources. You can write simple shell scripts to take data from one small program and run it into another. If you are obsessive like me, you write shell scripts that do the entire task, with copious comments explaining to your future self what you think the programs are doing, along with references to web pages, texts, etc. with more explanation. This is not as much to aim for 100% automation, as to aim for 0% stuffing your head with unneeded facts and activities. A side benefit is that if you are stuck in the Nevada desert with a cell phone, and your data and computer and wife are in Troutdale, you can spend a frustrating hour guiding her through preparing that forgotten report for an impatient customer, rather than spending two days going home to do it yourself. It is almost impossible to guide someone else through a GUI program over the phone. Keith -- Keith Lofstrom [email protected] Voice (503)-520-1993 KLIC --- Keith Lofstrom Integrated Circuits --- "Your Ideas in Silicon" Design Contracting in Bipolar and CMOS - Analog, Digital, and Scan ICs _______________________________________________ PLUG mailing list [email protected] http://lists.pdxlinux.org/mailman/listinfo/plug
