Bug#556652: octave can't load files created by libmatio
Hello Bas, Le mardi 17 novembre 2009 à 14:26 +0100, Sylvestre Ledru a écrit : > Le mardi 17 novembre 2009 à 13:48 +0100, Bas Zoetekouw a écrit : > > Package: libmatio0 > > Version: 1.3.3-5 > > Severity: normal > > > > octave can't read compressed files written by libmatio. I'm not sure > > if this is a problem in libmatio or in octave, so feel free to > > reassign as necessary. > It is not an octave bug. I just tried with Scilab and it causes a problem > with Scilab too. Vincent Couvert from Scilab wrote a patch which fixes the issue. It is available here: http://svn.debian.org/viewsvn/debian-science/packages/libmatio/trunk/debian/patches/ (or as attachement). Note that if you generate a foo.mat file from a matio lib without this fix, the file is still buggy and matio won't be able to load it. Vincent is trying to improve the behavior of matio when the file is buggy but the error handling of matio is not perfect. Sylvestre --- libmatio-1.3.3.orig/src/mat5.c 2010-03-01 23:11:21.0 +0100 +++ libmatio-1.3.3/src/mat5.c 2010-03-01 23:15:06.0 +0100 @@ -129,8 +129,15 @@ nBytes += tag_size + sparse->ndata*Mat_SizeOf(matvar->data_type); } else { nBytes += tag_size + nmemb*Mat_SizeOf(matvar->data_type); -if ( matvar->isComplex ) +if (nmemb*Mat_SizeOf(matvar->data_type) % 8) { +nBytes += (8 - (nmemb*Mat_SizeOf(matvar->data_type) % 8)); + } +if ( matvar->isComplex ) { nBytes += tag_size + nmemb*Mat_SizeOf(matvar->data_type); +if (nmemb*Mat_SizeOf(matvar->data_type) % 8) { +nBytes += (8 - (nmemb*Mat_SizeOf(matvar->data_type) % 8)); + } + } } return nBytes; @@ -218,8 +225,15 @@ nBytes += tag_size + sparse->ndata*Mat_SizeOf(matvar->data_type); } else { nBytes += tag_size + nmemb*Mat_SizeOf(matvar->data_type); -if ( matvar->isComplex ) +if ( nmemb*Mat_SizeOf(matvar->data_type) % 8 ) { +nBytes += (8 - (nmemb*Mat_SizeOf(matvar->data_type) % 8)); + } +if ( matvar->isComplex ) { nBytes += tag_size + nmemb*Mat_SizeOf(matvar->data_type); +if ( nmemb*Mat_SizeOf(matvar->data_type) % 8 ) { +nBytes += (8 - (nmemb*Mat_SizeOf(matvar->data_type) % 8)); + } + } } return nBytes; @@ -305,8 +319,15 @@ nBytes += tag_size + sparse->ndata*Mat_SizeOf(matvar->data_type); } else { nBytes += tag_size + nmemb*Mat_SizeOf(matvar->data_type); -if ( matvar->isComplex ) +if ( nmemb*Mat_SizeOf(matvar->data_type) % 8 ) { +nBytes += (8 - (nmemb*Mat_SizeOf(matvar->data_type) % 8)); + } +if ( matvar->isComplex ) { nBytes += tag_size + nmemb*Mat_SizeOf(matvar->data_type); +if ( nmemb*Mat_SizeOf(matvar->data_type) % 8 ) { +nBytes += (8 - (nmemb*Mat_SizeOf(matvar->data_type) % 8)); + } + } } return nBytes;
Bug#556652: octave can't load files created by libmatio
Hello Bas, Le mercredi 02 décembre 2009 à 09:41 +0100, Bas Zoetekouw a écrit : > Hi Sylvestre! > > You wrote: > > > > It is not an octave bug. I just tried with Scilab and it causes a > > > problem with Scilab too. > > > > Ah! Here's a bit more concise test case, which write only 1 integer: > [snip] > > Attached are the output file from this program (foo.mat) and the fiel as > > matlab itself saves it, generated by: > > > load('foo.mat'); > > > save('foo_matlab.mat','test') > > in Matlab 2009b. > > Note that the file are 1 byte different in length. > > I've analysed these files bij hand (well, using a bit of perl), and as > far as I can determine, both of them comply the the file structure as > described in [1]. > > I'll proceed to check why libmathio and octave can't read it. Did you made some progress on this bug ? Thanks, Sylvestre -- To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org
Bug#556652: octave can't load files created by libmatio
Hello Bas, Le mercredi 02 décembre 2009 à 09:41 +0100, Bas Zoetekouw a écrit : > Hi Sylvestre! > > You wrote: > > > > It is not an octave bug. I just tried with Scilab and it causes a > > > problem with Scilab too. > > > > Ah! Here's a bit more concise test case, which write only 1 integer: > [snip] > > Attached are the output file from this program (foo.mat) and the fiel as > > matlab itself saves it, generated by: > > > load('foo.mat'); > > > save('foo_matlab.mat','test') > > in Matlab 2009b. > > Note that the file are 1 byte different in length. > > I've analysed these files bij hand (well, using a bit of perl), and as > far as I can determine, both of them comply the the file structure as > described in [1]. > > I'll proceed to check why libmathio and octave can't read it. Thanks for the feedback. Please let me know how it goes ;) Sylvestre -- To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org
Bug#556652: octave can't load files created by libmatio
Hi Sylvestre! You wrote: > > It is not an octave bug. I just tried with Scilab and it causes a > > problem with Scilab too. > > Ah! Here's a bit more concise test case, which write only 1 integer: [snip] > Attached are the output file from this program (foo.mat) and the fiel as > matlab itself saves it, generated by: > > load('foo.mat'); > > save('foo_matlab.mat','test') > in Matlab 2009b. > Note that the file are 1 byte different in length. I've analysed these files bij hand (well, using a bit of perl), and as far as I can determine, both of them comply the the file structure as described in [1]. I'll proceed to check why libmathio and octave can't read it. Greetings, Bas. [1] http://www.mathworks.com/access/helpdesk/help/pdf_doc/matlab/matfile_format.pdf -- +--+ | Bas Zoetekouw | Sweet day, so cool, so calm, so bright, | || The bridall of the earth and skie: | | b...@zoetekouw.net | The dew shall weep thy fall tonight;| +|For thou must die. | +-+ -- To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org
Bug#556652: octave can't load files created by libmatio
Hi Sylvestre! You wrote: > It is not an octave bug. I just tried with Scilab and it causes a > problem with Scilab too. Ah! Here's a bit more concise test case, which write only 1 integer: #include int main() { mat_t * mat = Mat_Open( "foo.mat", 0 ); if ( mat==NULL ) abort(); int dims[2] = {1,1}; int i = 42; matvar_t * matvar = Mat_VarCreate( "test", MAT_C_INT16, MAT_T_INT16, 2, dims, &i, 0 ); #if 1 /* writes broken file */ Mat_VarWrite( mat, matvar, COMPRESSION_ZLIB ); #else /* writes working file */ Mat_VarWrite( mat, matvar, 0 ); #endif Mat_Close( mat ); return 0; } Attached are the output file from this program (foo.mat) and the fiel as matlab itself saves it, generated by: > load('foo.mat'); > save('foo_matlab.mat','test') in Matlab 2009b. Note that the file are 1 byte different in length. Hope this helps! Kind regards, Bas. foo.mat Description: Binary data foo_matlab.mat Description: Binary data
Bug#556652: octave can't load files created by libmatio
Le mardi 17 novembre 2009 à 13:48 +0100, Bas Zoetekouw a écrit : > Package: libmatio0 > Version: 1.3.3-5 > Severity: normal > > octave can't read compressed files written by libmatio. I'm not sure > if this is a problem in libmatio or in octave, so feel free to > reassign as necessary. It is not an octave bug. I just tried with Scilab and it causes a problem with Scilab too. #0 ReadNextCell (mat=0x8068ef8, matvar=0x806b9d8) at mat5.c:1358 #1 0xb143da96 in Mat_VarReadNextInfo5 (mat=0x8068ef8) at mat5.c:6969 #2 0xb1440330 in Mat_VarReadNextInfo (mat=0xbfffec5c) at mat.c:1703 #3 0xb1440395 in Mat_VarReadNext (mat=0x8068ef8) at mat.c:1813 I am going to dig here. Btw, thanks for the full code to reproduce it. It is very appreciated. Sylvestre -- To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org
Bug#556652: octave can't load files created by libmatio
Package: libmatio0 Version: 1.3.3-5 Severity: normal octave can't read compressed files written by libmatio. I'm not sure if this is a problem in libmatio or in octave, so feel free to reassign as necessary. The following program uses libmatio to write a simple data structure to a compressed mat file: #include #include #include #include #define NRSIGNALS 2 #define NRSAMPLES 100 int write_matfile( const char *fname ) { mat_t *mat = NULL; matvar_t **matchannels = NULL; mat = Mat_Open( fname, 0 ); if ( mat==NULL ) { fprintf( stderr, "Can't open file `%s' for output\n", fname ); return 0; } /* allocate memory for the channels */ matchannels = malloc( NRSIGNALS * sizeof(*matchannels) ); /* create a structure for each channel */ for ( int i = 0; i < NRSIGNALS; i++ ) { int dims[2] = {1,1}; matvar_t ** matdata = malloc( 2 * sizeof(*matdata) ); matdata[0] = Mat_VarCreate( "channel",MAT_C_INT16, MAT_T_INT16, 2, dims, &i, 0 ); matdata[1] = NULL; matchannels[i] = Mat_VarCreate( "", MAT_C_STRUCT, MAT_T_STRUCT, 2, dims, matdata, 0 ); } /* the structures for the channels are combined into a single cell array */ int dims[2] = {1,NRSIGNALS}; matvar_t * matvar_cell = Mat_VarCreate("data",MAT_C_CELL,MAT_T_CELL,2,dims,matchannels,0); /* write the file to disk */ Mat_VarWrite( mat, matvar_cell, COMPRESSION_ZLIB ); //Mat_VarWrite( mat, matvar_cell, 0 ); Mat_Close( mat ); return 1; } int main() { write_matfile( "/tmp/foo.mat" ); return(0); } Observe: [...@basbak2]/tmp> gcc edf2mat.c -std=c99 -lmatio && ./a.out [...@basbak2]/tmp> ls -la foo.mat -rw-r--r-- 1 bas bas 215 nov 17 13:47 foo.mat [...@basbak2]/tmp> file foo.mat foo.mat: Matlab v5 mat-file (little endian) version 0x0100 [...@basbak2]/tmp> octave3.2 -q octave3.2:1> load('foo.mat') error: load: error uncompressing data element The file loads fine in matlab. If the file is written without compression, it also loads fine in octave. -- System Information: Debian Release: 5.0.3 APT prefers stable APT policy: (500, 'stable') Architecture: amd64 (x86_64) Kernel: Linux 2.6.26-2-amd64 (SMP w/1 CPU core) Locale: LANG=en_GB.UTF-8, LC_CTYPE=en_GB.UTF-8 (charmap=UTF-8) Shell: /bin/sh linked to /bin/bash -- To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org