Re: [julia-users] Re: Immutable type modification of SparseMatrixCSC
Le dimanche 06 novembre 2016 à 15:31 -0800, Kristoffer Carlsson a écrit : > > > > For reference, here's the long discussion that happened before making > > that change: > > https://github.com/JuliaLang/julia/pull/16371 > > > > Indeed I think Tony was right that this has undesirable consequences in > > terms of usability. Not being able to use the same in-place API for > > dense and sparse matrices is a no-go IMHO. > > > > Please elaborate. You cannot resize dense matrices in-place either. > > Just create a new matrix from the fields of the old one if you want > to change the size. How does this limit usability? Right, I was being sloppy. That remark applies to sparse vectors, though. Regards > > > > Regards > > > > > > Le samedi 05 novembre 2016 à 20:11 -0700, vav...@uwaterloo.ca a > > écrit : > > > Just to clarify, you don't actually need to allocate the memory for > > > two large sparse matrices. You can change colptr, rowval, and nzval > > > in place (as you have already done) and then create a new sparse > > > matrix using these modified arrays. In this case, the arrays won't > > > be copied over; instead the new sparse matrix will get a pointer to > > > the already-allocated arrays. This technique could lead to some > > > subtle program bugs (because if two sparse matrices share an nzval > > > array, then any change to one affects the other), but you can guard > > > against this by making the old matrix's name inaccessible, for > > > instance, by giving the new matrix the same name as the old one. > > > > > > -- Steve Vavasis > > > > > > > > > > I see. returning a new, smaller instance of the matrix was my > > > > temporary solution. Bummer that it's necessary... Is it really so > > > > strange to want to change the size of a > > > > > > > > Another alternative which occurs to me is to make some child class > > > > of SparseCSCMatrix with two more arrays, each with one element, > > > > which track the sizes? Since I am making the matrix smaller, not > > > > larger, I can't imagine that this would be an issue. However, if I > > > > need to allocate two large sparse matrices just to remove a row/col > > > > pair, then I can suddenly use half the memory I could before... > > > > > > > > > > > > > When a type is immutable, that means you can't modify the data in > > > > > the type. If a field of a type is mutable, a reference to it is > > > > > stored in the type, and you cannot modify that reference. In the > > > > > case of SparseMatrixCSC, that means you can't make A.colptr point > > > > > to a different array, for example. You are still free to change > > > > > the values in A.colptr, because that does not require modifying > > > > > the reference. If the field is immutable, then the value itself > > > > > is stored in the type, not a reference to the value, and you > > > > > cannot modify the value. That's why you get the error in the > > > > > example. > > > > > > > > > > The best solution I can think of for your rmCol example is to > > > > > create a new SparseMatrixCSC that uses the same arrays as the old > > > > > one (to avoid increasing memory usage) and return it. Replacing > > > > > the last line of rmCol with: > > > > > > > > > > SparseMatrixCSC(m, n-1, A.colptr, A.rowval, A.nzval) > > > > > > > > > > should do the trick. The only sticky point is that the > > > > > original A now contains invalid data, so you'll have to make sure > > > > > only the the matrix returned from rmCol is used in the future, > > > > > not the original A. > > > > > > > > > > Jared Crean > > > > > > > > > > > > > > > > The SparseMatrixCSC type is immutable, which I understand to > > > > > > mean that I must respect the types of the attributes of > > > > > > SparseMatrixCSC types, as well as not changing the attributes > > > > > > themselves. > > > > > > > > > > > > The following gives a loadError (type is immutable). I am > > > > > > confused that I can modify the colptr and rowval attributes of > > > > > > A, but not the A.n attribute. Why is this? I am trying to > > > > > > change the size of a sparse matrix after removing the data from > > > > > > a row and column. > > > > > > > > > > > > I have checked that both A.n and A.n - 1 are type Int. > > > > > > > > > > > > function rmCol(A::SparseMatrixCSC, rmCol::Int) > > > > > > colRmRange = A.colptr[rmCol]:(A.colptr[rmCol+1]-1) > > > > > > filter!(e -> e != rmCol, A.colptr) > > > > > > > > > > > > deleteat!(A.rowval, rowval[colRmRange]) > > > > > > deleteat!(A.nzval, colRmRange) > > > > > > A.n -= 1 # inclusion of this line throws error > > > > > > > > > > > > end > > > > > > > > > > > > > > >
Re: [julia-users] Re: Immutable type modification of SparseMatrixCSC
On Sunday, November 6, 2016 at 2:25:27 AM UTC-8, Milan Bouchet-Valat wrote: > > For reference, here's the long discussion that happened before making > that change: > https://github.com/JuliaLang/julia/pull/16371 > > Indeed I think Tony was right that this has undesirable consequences in > terms of usability. Not being able to use the same in-place API for > dense and sparse matrices is a no-go IMHO. > > Please elaborate. You cannot resize dense matrices in-place either. Just create a new matrix from the fields of the old one if you want to change the size. How does this limit usability? > > Regards > > Le samedi 05 novembre 2016 à 20:11 -0700, vav...@uwaterloo.ca > a > écrit : > > Just to clarify, you don't actually need to allocate the memory for > > two large sparse matrices. You can change colptr, rowval, and nzval > > in place (as you have already done) and then create a new sparse > > matrix using these modified arrays. In this case, the arrays won't > > be copied over; instead the new sparse matrix will get a pointer to > > the already-allocated arrays. This technique could lead to some > > subtle program bugs (because if two sparse matrices share an nzval > > array, then any change to one affects the other), but you can guard > > against this by making the old matrix's name inaccessible, for > > instance, by giving the new matrix the same name as the old one. > > > > -- Steve Vavasis > > > > > > > I see. returning a new, smaller instance of the matrix was my > > > temporary solution. Bummer that it's necessary... Is it really so > > > strange to want to change the size of a > > > > > > Another alternative which occurs to me is to make some child class > > > of SparseCSCMatrix with two more arrays, each with one element, > > > which track the sizes? Since I am making the matrix smaller, not > > > larger, I can't imagine that this would be an issue. However, if I > > > need to allocate two large sparse matrices just to remove a row/col > > > pair, then I can suddenly use half the memory I could before... > > > > > > > > > > When a type is immutable, that means you can't modify the data in > > > > the type. If a field of a type is mutable, a reference to it is > > > > stored in the type, and you cannot modify that reference. In the > > > > case of SparseMatrixCSC, that means you can't make A.colptr point > > > > to a different array, for example. You are still free to change > > > > the values in A.colptr, because that does not require modifying > > > > the reference. If the field is immutable, then the value itself > > > > is stored in the type, not a reference to the value, and you > > > > cannot modify the value. That's why you get the error in the > > > > example. > > > > > > > > The best solution I can think of for your rmCol example is to > > > > create a new SparseMatrixCSC that uses the same arrays as the old > > > > one (to avoid increasing memory usage) and return it. Replacing > > > > the last line of rmCol with: > > > > > > > > SparseMatrixCSC(m, n-1, A.colptr, A.rowval, A.nzval) > > > > > > > > should do the trick. The only sticky point is that the > > > > original A now contains invalid data, so you'll have to make sure > > > > only the the matrix returned from rmCol is used in the future, > > > > not the original A. > > > > > > > > Jared Crean > > > > > > > > > > > > > The SparseMatrixCSC type is immutable, which I understand to > > > > > mean that I must respect the types of the attributes of > > > > > SparseMatrixCSC types, as well as not changing the attributes > > > > > themselves. > > > > > > > > > > The following gives a loadError (type is immutable). I am > > > > > confused that I can modify the colptr and rowval attributes of > > > > > A, but not the A.n attribute. Why is this? I am trying to > > > > > change the size of a sparse matrix after removing the data from > > > > > a row and column. > > > > > > > > > > I have checked that both A.n and A.n - 1 are type Int. > > > > > > > > > > function rmCol(A::SparseMatrixCSC, rmCol::Int) > > > > > colRmRange = A.colptr[rmCol]:(A.colptr[rmCol+1]-1) > > > > > filter!(e -> e != rmCol, A.colptr) > > > > > > > > > > deleteat!(A.rowval, rowval[colRmRange]) > > > > > deleteat!(A.nzval, colRmRange) > > > > > A.n -= 1 # inclusion of this line throws error > > > > > > > > > > end > > > > > > > > > > > > >
Re: [julia-users] Re: Immutable type modification of SparseMatrixCSC
For reference, here's the long discussion that happened before making that change: https://github.com/JuliaLang/julia/pull/16371 Indeed I think Tony was right that this has undesirable consequences in terms of usability. Not being able to use the same in-place API for dense and sparse matrices is a no-go IMHO. Regards Le samedi 05 novembre 2016 à 20:11 -0700, vava...@uwaterloo.ca a écrit : > Just to clarify, you don't actually need to allocate the memory for > two large sparse matrices. You can change colptr, rowval, and nzval > in place (as you have already done) and then create a new sparse > matrix using these modified arrays. In this case, the arrays won't > be copied over; instead the new sparse matrix will get a pointer to > the already-allocated arrays. This technique could lead to some > subtle program bugs (because if two sparse matrices share an nzval > array, then any change to one affects the other), but you can guard > against this by making the old matrix's name inaccessible, for > instance, by giving the new matrix the same name as the old one. > > -- Steve Vavasis > > > > I see. returning a new, smaller instance of the matrix was my > > temporary solution. Bummer that it's necessary... Is it really so > > strange to want to change the size of a > > > > Another alternative which occurs to me is to make some child class > > of SparseCSCMatrix with two more arrays, each with one element, > > which track the sizes? Since I am making the matrix smaller, not > > larger, I can't imagine that this would be an issue. However, if I > > need to allocate two large sparse matrices just to remove a row/col > > pair, then I can suddenly use half the memory I could before... > > > > > > > When a type is immutable, that means you can't modify the data in > > > the type. If a field of a type is mutable, a reference to it is > > > stored in the type, and you cannot modify that reference. In the > > > case of SparseMatrixCSC, that means you can't make A.colptr point > > > to a different array, for example. You are still free to change > > > the values in A.colptr, because that does not require modifying > > > the reference. If the field is immutable, then the value itself > > > is stored in the type, not a reference to the value, and you > > > cannot modify the value. That's why you get the error in the > > > example. > > > > > > The best solution I can think of for your rmCol example is to > > > create a new SparseMatrixCSC that uses the same arrays as the old > > > one (to avoid increasing memory usage) and return it. Replacing > > > the last line of rmCol with: > > > > > > SparseMatrixCSC(m, n-1, A.colptr, A.rowval, A.nzval) > > > > > > should do the trick. The only sticky point is that the > > > original A now contains invalid data, so you'll have to make sure > > > only the the matrix returned from rmCol is used in the future, > > > not the original A. > > > > > > Jared Crean > > > > > > > > > > The SparseMatrixCSC type is immutable, which I understand to > > > > mean that I must respect the types of the attributes of > > > > SparseMatrixCSC types, as well as not changing the attributes > > > > themselves. > > > > > > > > The following gives a loadError (type is immutable). I am > > > > confused that I can modify the colptr and rowval attributes of > > > > A, but not the A.n attribute. Why is this? I am trying to > > > > change the size of a sparse matrix after removing the data from > > > > a row and column. > > > > > > > > I have checked that both A.n and A.n - 1 are type Int. > > > > > > > > function rmCol(A::SparseMatrixCSC, rmCol::Int) > > > > colRmRange = A.colptr[rmCol]:(A.colptr[rmCol+1]-1) > > > > filter!(e -> e != rmCol, A.colptr) > > > > > > > > deleteat!(A.rowval, rowval[colRmRange]) > > > > deleteat!(A.nzval, colRmRange) > > > > A.n -= 1 # inclusion of this line throws error > > > > > > > > end > > > > > > > > >
[julia-users] Re: Immutable type modification of SparseMatrixCSC
Just to clarify, you don't actually need to allocate the memory for two large sparse matrices. You can change colptr, rowval, and nzval in place (as you have already done) and then create a new sparse matrix using these modified arrays. In this case, the arrays won't be copied over; instead the new sparse matrix will get a pointer to the already-allocated arrays. This technique could lead to some subtle program bugs (because if two sparse matrices share an nzval array, then any change to one affects the other), but you can guard against this by making the old matrix's name inaccessible, for instance, by giving the new matrix the same name as the old one. -- Steve Vavasis On Saturday, November 5, 2016 at 8:32:58 PM UTC-4, Corbin Foucart wrote: > > I see. returning a new, smaller instance of the matrix was my temporary > solution. Bummer that it's necessary... Is it really so strange to want to > change the size of a > > Another alternative which occurs to me is to make some child class of > SparseCSCMatrix with two more arrays, each with one element, which track > the sizes? Since I am making the matrix smaller, not larger, I can't > imagine that this would be an issue. However, if I need to allocate two > large sparse matrices just to remove a row/col pair, then I can suddenly > use half the memory I could before... > > > On Saturday, November 5, 2016 at 7:42:55 PM UTC-4, Jared Crean wrote: >> >> When a type is immutable, that means you can't modify the data in the >> type. If a field of a type is mutable, a reference to it is stored in the >> type, and you cannot modify that reference. In the case of >> SparseMatrixCSC, that means you can't make A.colptr point to a different >> array, for example. You are still free to change the values in A.colptr, >> because that does not require modifying the reference. If the field is >> immutable, then the value itself is stored in the type, not a reference to >> the value, and you cannot modify the value. That's why you get the error >> in the example. >> >> The best solution I can think of for your rmCol example is to create a >> new SparseMatrixCSC that uses the same arrays as the old one (to avoid >> increasing memory usage) and return it. Replacing the last line of rmCol >> with: >> >> SparseMatrixCSC(m, n-1, A.colptr, A.rowval, A.nzval) >> >> should do the trick. The only sticky point is that the original A now >> contains invalid data, so you'll have to make sure only the the matrix >> returned from rmCol is used in the future, not the original A. >> >> Jared Crean >> >> >> On Saturday, November 5, 2016 at 5:07:45 PM UTC-4, Corbin Foucart wrote: >>> >>> The SparseMatrixCSC type is immutable, which I understand to mean that I >>> must respect the types of the attributes of SparseMatrixCSC types, as well >>> as not changing the attributes themselves. >>> >>> The following gives a loadError (type is immutable). I am confused that >>> I can modify the colptr and rowval attributes of A, but not the A.n >>> attribute. Why is this? I am trying to change the size of a sparse matrix >>> after removing the data from a row and column. >>> >>> I have checked that both A.n and A.n - 1 are type Int. >>> >>> function rmCol(A::SparseMatrixCSC, rmCol::Int) >>> colRmRange = A.colptr[rmCol]:(A.colptr[rmCol+1]-1) >>> filter!(e -> e != rmCol, A.colptr) >>> >>> deleteat!(A.rowval, rowval[colRmRange]) >>> deleteat!(A.nzval, colRmRange) >>> A.n -= 1 # inclusion of this line throws error >>> >>> end >>> >>
[julia-users] Re: Immutable type modification of SparseMatrixCSC
I see. returning a new, smaller instance of the matrix was my temporary solution. Bummer that it's necessary... Is it really so strange to want to change the size of a Another alternative which occurs to me is to make some child class of SparseCSCMatrix with two more arrays, each with one element, which track the sizes? Since I am making the matrix smaller, not larger, I can't imagine that this would be an issue. However, if I need to allocate two large sparse matrices just to remove a row/col pair, then I can suddenly use half the memory I could before... On Saturday, November 5, 2016 at 7:42:55 PM UTC-4, Jared Crean wrote: > > When a type is immutable, that means you can't modify the data in the > type. If a field of a type is mutable, a reference to it is stored in the > type, and you cannot modify that reference. In the case of > SparseMatrixCSC, that means you can't make A.colptr point to a different > array, for example. You are still free to change the values in A.colptr, > because that does not require modifying the reference. If the field is > immutable, then the value itself is stored in the type, not a reference to > the value, and you cannot modify the value. That's why you get the error > in the example. > > The best solution I can think of for your rmCol example is to create a new > SparseMatrixCSC that uses the same arrays as the old one (to avoid > increasing memory usage) and return it. Replacing the last line of rmCol > with: > > SparseMatrixCSC(m, n-1, A.colptr, A.rowval, A.nzval) > > should do the trick. The only sticky point is that the original A now > contains invalid data, so you'll have to make sure only the the matrix > returned from rmCol is used in the future, not the original A. > > Jared Crean > > > On Saturday, November 5, 2016 at 5:07:45 PM UTC-4, Corbin Foucart wrote: >> >> The SparseMatrixCSC type is immutable, which I understand to mean that I >> must respect the types of the attributes of SparseMatrixCSC types, as well >> as not changing the attributes themselves. >> >> The following gives a loadError (type is immutable). I am confused that >> I can modify the colptr and rowval attributes of A, but not the A.n >> attribute. Why is this? I am trying to change the size of a sparse matrix >> after removing the data from a row and column. >> >> I have checked that both A.n and A.n - 1 are type Int. >> >> function rmCol(A::SparseMatrixCSC, rmCol::Int) >> colRmRange = A.colptr[rmCol]:(A.colptr[rmCol+1]-1) >> filter!(e -> e != rmCol, A.colptr) >> >> deleteat!(A.rowval, rowval[colRmRange]) >> deleteat!(A.nzval, colRmRange) >> A.n -= 1 # inclusion of this line throws error >> >> end >> >
[julia-users] Re: Immutable type modification of SparseMatrixCSC
When a type is immutable, that means you can't modify the data in the type. If a field of a type is mutable, a reference to it is stored in the type, and you cannot modify that reference. In the case of SparseMatrixCSC, that means you can't make A.colptr point to a different array, for example. You are still free to change the values in A.colptr, because that does not require modifying the reference. If the field is immutable, then the value itself is stored in the type, not a reference to the value, and you cannot modify the value. That's why you get the error in the example. The best solution I can think of for your rmCol example is to create a new SparseMatrixCSC that uses the same arrays as the old one (to avoid increasing memory usage) and return it. Replacing the last line of rmCol with: SparseMatrixCSC(m, n-1, A.colptr, A.rowval, A.nzval) should do the trick. The only sticky point is that the original A now contains invalid data, so you'll have to make sure only the the matrix returned from rmCol is used in the future, not the original A. Jared Crean On Saturday, November 5, 2016 at 5:07:45 PM UTC-4, Corbin Foucart wrote: > > The SparseMatrixCSC type is immutable, which I understand to mean that I > must respect the types of the attributes of SparseMatrixCSC types, as well > as not changing the attributes themselves. > > The following gives a loadError (type is immutable). I am confused that I > can modify the colptr and rowval attributes of A, but not the A.n > attribute. Why is this? I am trying to change the size of a sparse matrix > after removing the data from a row and column. > > I have checked that both A.n and A.n - 1 are type Int. > > function rmCol(A::SparseMatrixCSC, rmCol::Int) > colRmRange = A.colptr[rmCol]:(A.colptr[rmCol+1]-1) > filter!(e -> e != rmCol, A.colptr) > > deleteat!(A.rowval, rowval[colRmRange]) > deleteat!(A.nzval, colRmRange) > A.n -= 1 # inclusion of this line throws error > > end >
