w4nderlust wrote: > Il giorno 06/gen/2011, alle ore 22.27, David Bateman ha scritto: > > >> w4nderlust wrote: >> >>> Hello everyone. >>> >>> I'm still working with sparse matrices in oct-file, but i've got some >>> problems: i'm doing some multithread and as the octave libraries >>> revealed to be not really thread safe (some unexpected crashed ofter >>> repeating executions) i did the dirty work by hand: i copied the 3 >>> vectors of the matrix B (the input matrix) and copied the full matrix >>> A into a vector. >>> >> If you do something like >> >> const SparseMatrix a = args(0).sparse_matrix_value (); >> const *double adata = a.data(); >> const *octave_idx_type acidx = a.cidx(); >> const *octave_idx_type aridx = a.ridx(); >> >> you can clean up a lot of the issues I believe you were having. As the >> rest of you code will then just work on these pointers there is no >> reason that the code shouldn't be thread-safe. However >> > > I'm trying right now, but when i was using the multithread on the nonsparse > version of this project (so using the octve matrix objects) it crashed when > repeating 1000 executions continuously, but when i modifidies to use simble > double * as matrix class replacement it worked fine. That's why i'm doing all > this :) > I think you missed my point. The above gives you double* and octave_idx_type* pointer to the underlying data structures of the Octave sparse matrix that you can manipulate directly. No need to copy from the matrix to a *double pointer.
> I think you are talking about this: > http://www.network-theory.co.uk/docs/octave3/octave_204.html 20.1.1 > I was talking about the oct-file part ( > http://www.gnu.org/software/octave/doc/interpreter/Sparse-Matrices-in-Oct_002dFiles.html > ). The two sections were written together. If you don't know how sparse matrices are used in the octave interpreter its perhaps permature to read the section on using them in oct-files. So section 21 of the manual should be read before the appendix. > I think is lacking in at least 2 points, i write them here not to contrast > your opinion, but simply because maybe someone would agree and improve the > documentation (maybe when i finish this project i could do something for what > i think i understand well > I wrote these sections, and although these were written rapidly, if they aren't read independently, and other basic concepts of oct-files are understood they should be understandable. Yes they might be improved, but I think you need to use sparse matrices a bit more in Octave first as there seems to be some basic points you're still missing. > - the example in 20.1.1 is confusing with the example in > http://www.gnu.org/software/octave/doc/interpreter/Creating-Sparse-Matrices-in-Oct_002dFiles.html#Creating-Sparse-Matrices-in-Oct_002dFiles > (the first one) as the cidx vector is completely different 8and i think it > is wrong here). > What is proposed in 20.1.2 (21.1.2 in the manual of 3.3.x and soon to be 3.4.0) rather than 20.1.1. The data structure cidx, ridx of a sparse matrix are not exposed in the interpreter of Octave and so working with them directly is not an option. What is proposed in section 2.1.2 is to create 3 vectors representing the triplets (row, col, val) that are then internally converted to cidx, ridx, data in an fairly efficient manner. In oct-files you have access to cidx, ridx, data and working directly with them is more efficient and trying to work on a sparse matrix in an oct-file directly with (row, col, val) triplets is going to be extremely slow.. This explains the difference between the two. If you want to do the same thing that the example in 21.1.2 suggests in a oct-file then do something like OCTAVE_LOCAL_BUFFER (double, val, nnz) OCTAVE_LOCAL_BUFFER (octave_idx_type, rows, nnz) OCTAVE_LOCAL_BUFFER (octave_idx_type, cols, nnz) for (octave_idx_type i = 0; i < nnz; i++) { val[i] = ....; rows[i] = ....; cols[i] = ....; } SparseMatix c (rows, cols, val) instead. However working directly with cidx, ridx and data will save you a copying the matrix in memory and will be more efficient. > - the usage page is really poor, the only method shown > sparse_bool_matrix_value () . I have no idea what you mean by "usage page".. I presume you mean section A1.6.3. This section as it says is how to use the sparse matrix type in oct-files themselves and concerns only the interface from the octave_value type and back again so that you can deal with the data. Section A1.6.2 had already discussed how to deal with the data in the sparse matrices themselves.. Section A1.6.3 has the example octave_value_list retval; SparseMatrix sm = args(0).sparse_matrix_value (); SparseComplexMatrix scm = args(1).sparse_complex_matrix_value (); SparseBoolMatrix sbm = args(2).sparse_bool_matrix_value (); ... retval(2) = sbm; retval(1) = scm; retval(0) = sm; and all three types of sparse matrices are treated.. I can accept criticism but at least make it valid. > All he other methods are present in the octave-forge documentation but > there's no description for each one and some parameters are really obscure to > me (for example the boolean value at the end of the constructor with the 3 > vectors) > > I presume by the octave-forge documentation you mean the doxygen pages at http://octave.sourceforge.net/doxygen/html/class_sparse_matrix.html This is created from the octave source code directly by doxygen and has no more information than the source code itself. By boolean value at the end of the constructor I presume you mean the constructor given by http://octave.sourceforge.net/doxygen/html/class_sparse_matrix.html#3d38fab78c54c461739f46ae8e7196ff where the boolean value is called "sum_terms". Nothing stops you (row,col,val) triplets from duplicating (row,col) values. In this case you have two choices. Keep only the second value or sum the values with the same (row,col) values which is the default.. Its kind of implicit in the variable name "sum_terms" I would have thought.. Or you might have figured it out by reading "help sparse" in the section <quote> Note: if multiple values are specified with the same I, J indices, the corresponding values in S will be added. The following are all equivalent: s = sparse (i, j, s, m, n) s = sparse (i, j, s, m, n, "summation") s = sparse (i, j, s, m, n, "sum") Given the option "unique". if more than two values are specified for the same I, J indices, the last specified value will be used. </quote> Sure it might be clearer, but no one has complained about this point yet. The octave code is pretty clear and so you shouldn't hesitate to jump into it directly for more info. > >> I see two solutions. >> >> 1) Precalculate the number of non-zero elements per-column. That is, in >> the code block I sent you for spmaxmin_new.cc calculate the output cidx >> at the same time you calculate nel, rather than in the second loop. If >> you first save the number of non-zero elements to ccidx you can even >> thread this. For example (with code that might be threaded, without the >> threading itself) >> >> c.xcidx (0) = 0; >> for (octave_idx_type i = 0; i < rowsA; i++) >> { >> // From here on this code could be threaded >> octave_idx_type kk = 0 >> >> for(octave_idx_type j = 0; j < colsB; j++) >> { >> if (b.cidx(j) != b.cidx(j+1)) >> kk++; >> else >> { >> for (octave_idx_type k = b.cidx(j); >> k <= (b.cidx(j+1) - 1); k++) >> { >> if (a(i, b.ridx(k)) != 0) >> { >> kk++; >> break; >> } >> } >> } >> } >> c.xcidx(i+1) = kk; >> } >> >> // Now that we've calculated the number of elements per column of >> the output matrix in code >> // that might be threaded, correct the indexing (Can't be threaded) >> for (octave_idx_type i = 0; i < rowsA; i++) >> c.xcidx(i+1) += c.xcidx(i); >> >> octave_idx_type nel = c.xcidx(rowsA); >> >> It might not be profitable to thread the above and it could be >> simplified to >> >> octave_idx_type nel = 0; >> >> c.xcidx(0) = 0; >> for (octave_idx_type i = 0; i < rowsA; i++) >> { >> for(octave_idx_type j = 0; j < colsB; j++) >> { >> if (b.cidx(j) != b.cidx(j+1)) >> nel++; >> else >> { >> for (octave_idx_type k = b.cidx(j); >> k <= (b.cidx(j+1) - 1); k++) >> { >> if (a(i, b.ridx(k)) != 0) >> { >> nel++; >> break; >> } >> } >> } >> } >> c.xcidx(i+1) = nel; >> } >> >> You can then uses c.xcidx in your thread_function to index correctly >> into c.xridx and c.xdata. >> >> >> 2) You could create N different sparse matrices, one per thread, and >> then concatenate them together with the concat method. This will >> probably be slower and more memory hungry than the above so I won't >> develop this idea further. >> >> > > I think the first is the wa i will follow. Mabe i'll try to execute my test > adding directly to c(i,j) for each thread (this makes the code really cleaner > for the threads) and not working directly on the c index vectors. If this > still gives me creashes on multiple executions, i'll try to wark the way you > described on point 1. > I reaIly wouldn't suggest working directly with c(i,j) as writing to c(i,j) will essentially move the data in the matrix to to make way for the new data. If you want to work with c(i,j) then the loops should be in this order for (octave_idx_type i = 0; i < m; i++) for (octave_idx_type j = 0; j < n; j++) c(i,j) = ....; and not the reverse or any out of order row,col calculation as this will allow the new elements to be written directly to the end of the ridx and data arrays. However, to just simply search the matrix to ensure that such a copy isn't needed and modifying all the elements of cidx implied by a write to c(i,j) is a O(n) operation, whereas working directly with an element using cidx,ridx,data should be O(1). So I know which way I'd prefer to write my code. D. ------------------------------------------------------------------------------ Gaining the trust of online customers is vital for the success of any company that requires sensitive data to be transmitted over the Web. Learn how to best implement a security strategy that keeps consumers' information secure and instills the confidence they need to proceed with transactions. http://p.sf.net/sfu/oracle-sfdevnl _______________________________________________ Octave-dev mailing list Octave-dev@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/octave-dev
