Re: [eigen] Using Eigen decompositions with ceres autodiff Jet data type.

2020-06-29 Thread Rasmus Munk Larsen
hanged my algorithm >>>> slightly and now only need .pow(), but this does not work with Jets either. >>>> I think the problem with .pow() looks easier to fix? >>>> >>>> >>>> >>>> Thanks, >>>> >>>> Toby >>&

Re: [eigen] Using Eigen decompositions with ceres autodiff Jet data type.

2020-06-29 Thread Oleg Shirokobrod
o, I have changed my algorithm >>> slightly and now only need .pow(), but this does not work with Jets either. >>> I think the problem with .pow() looks easier to fix? >>> >>> >>> >>> Thanks, >>> >>> Toby >>> >>> >&g

Re: [eigen] Using Eigen decompositions with ceres autodiff Jet data type.

2020-06-28 Thread Sameer Agarwal
Oleg, would you mind posting the ceres issues either on the ceres mailinglist or on https://github.com/ceres-solver/ceres-solver/issues. Sameer On Mon, Jun 1, 2020 at 8:44 AM Oleg Shirokobrod wrote: > Hi Sameer, > > Thank you for quick reply and advices. > 1. I implemented using both

Re: [eigen] Using Eigen decompositions with ceres autodiff Jet data type.

2020-06-28 Thread Sameer Agarwal
>> *Reply to: *"eigen@lists.tuxfamily.org" >> *Date: *Thursday, 4 June 2020 at 18:46 >> *To: *eigen , Sameer Agarwal < >> sameeragar...@google.com> >> *Subject: *Re: [eigen] Using Eigen decompositions with ceres autodiff >> Jet data typ

Re: [eigen] Using Eigen decompositions with ceres autodiff Jet data type.

2020-06-18 Thread Sameer Agarwal
18:46 > *To: *eigen , Sameer Agarwal < > sameeragar...@google.com> > *Subject: *Re: [eigen] Using Eigen decompositions with ceres autodiff Jet > data type. > > > > Hi Tobias, > > Please do. Sameer, since this is Ceres solver related, could I ask you to > help

Re: [eigen] Using Eigen decompositions with ceres autodiff Jet data type.

2020-06-06 Thread Wood, Tobias
: Thursday, 4 June 2020 at 18:46 To: eigen , Sameer Agarwal Subject: Re: [eigen] Using Eigen decompositions with ceres autodiff Jet data type. Hi Tobias, Please do. Sameer, since this is Ceres solver related, could I ask you to help out with this issue. Rasmus On Thu, Jun 4, 2020 at 3:06 AM Wo

Re: [eigen] Using Eigen decompositions with ceres autodiff Jet data type.

2020-06-04 Thread Oleg Shirokobrod
Rasmus, I expected that ceres-solver traits do the job #if EIGEN_VERSION_AT_LEAST(3, 3, 0) // Specifying the return type of binary operations between Jets and scalar types // allows you to perform matrix/array operations with Eigen matrices and arrays // such as addition, subtraction,

Re: [eigen] Using Eigen decompositions with ceres autodiff Jet data type.

2020-06-04 Thread Rasmus Munk Larsen
Oleg, Basically my response is that A.jacobiSVD().solve( b.cast()) is what you should be doing - we made a conscious decision to disallow code to rely on implicit casting. Just speculation: Perhaps add your own specialization of BinOp(T, double) that does the right thing and is faster? On Wed,

Re: [eigen] Using Eigen decompositions with ceres autodiff Jet data type.

2020-06-04 Thread Rasmus Munk Larsen
; Thanks, > > Toby > > > > *From: *Oleg Shirokobrod > *Reply to: *"eigen@lists.tuxfamily.org" > *Date: *Thursday, 4 June 2020 at 06:36 > *To: *"eigen@lists.tuxfamily.org" > *Subject: *Re: [eigen] Using Eigen decompositions with ceres autodiff Jet > d

Re: [eigen] Using Eigen decompositions with ceres autodiff Jet data type.

2020-06-04 Thread Wood, Tobias
. Should I create an issue on the Eigen gitlab? Thanks, Toby From: Oleg Shirokobrod Reply to: "eigen@lists.tuxfamily.org" Date: Thursday, 4 June 2020 at 06:36 To: "eigen@lists.tuxfamily.org" Subject: Re: [eigen] Using Eigen decompositions with ceres autodiff Jet data type. 1.

Re: [eigen] Using Eigen decompositions with ceres autodiff Jet data type.

2020-06-03 Thread Oleg Shirokobrod
1. I would like to have autodiff ability, so I cannot use double for both A and b. If I cast b: A.jacobiSVD().solve( b.cast()) everything works fine, but BinOp(T,T) is more expensive than BinOp(T, double). I would like to keep b as a vector of doubles. 2. T=Jet is ceres solver autodiff

Re: [eigen] Using Eigen decompositions with ceres autodiff Jet data type.

2020-06-03 Thread Rasmus Munk Larsen
Try to compile your code in debug mode with the type assertions on. On Wed, Jun 3, 2020 at 1:14 PM Rasmus Munk Larsen wrote: > Are you saying that you compute the decomposition in one type and solve > with a RHS of a different type? Why do you say that VS^-1U^T * b should be > Matrix? That

Re: [eigen] Using Eigen decompositions with ceres autodiff Jet data type.

2020-06-03 Thread Rasmus Munk Larsen
Are you saying that you compute the decomposition in one type and solve with a RHS of a different type? Why do you say that VS^-1U^T * b should be Matrix? That makes an assumption about type coercion rules. In fact, you cannot generally mix types in Eigen expressions without explicit casting, and

Re: [eigen] Using Eigen decompositions with ceres autodiff Jet data type.

2020-06-03 Thread Oleg Shirokobrod
Rasmuss, I do not quite understand this issue. Decomposition solve should propagate scalar type of a matrix but not scalar type of its argument. Example: template Matrix A; VectorXd b; A.jcobiSVD().solve(b) should be of type Matrix but it is not. Type of result is Matrix. If we make SVD

Re: [eigen] Using Eigen decompositions with ceres autodiff Jet data type.

2020-06-03 Thread Rasmus Munk Larsen
OK, I opened https://gitlab.com/libeigen/eigen/-/issues/1909 for this. On Tue, Jun 2, 2020 at 11:06 PM Oleg Shirokobrod wrote: > Yes. At the time of computing only 1d observation (VectorXd) is known. > > On Tue, Jun 2, 2020 at 9:42 PM Rasmus Munk Larsen > wrote: > >> OK, so b is declared as

Re: [eigen] Using Eigen decompositions with ceres autodiff Jet data type.

2020-06-03 Thread Oleg Shirokobrod
Yes. At the time of computing only 1d observation (VectorXd) is known. On Tue, Jun 2, 2020 at 9:42 PM Rasmus Munk Larsen wrote: > OK, so b is declared as VectorXf or some other type with > ColsAtCompileTime=1? > > On Tue, Jun 2, 2020 at 11:27 AM Oleg Shirokobrod < > oleg.shirokob...@gmail.com>

Re: [eigen] Using Eigen decompositions with ceres autodiff Jet data type.

2020-06-02 Thread Rasmus Munk Larsen
OK, so b is declared as VectorXf or some other type with ColsAtCompileTime=1? On Tue, Jun 2, 2020 at 11:27 AM Oleg Shirokobrod wrote: > > Yes, b is measured spectrum that is 1d array. I have to get x for 1d array > at a time. I fit sum of peak models to 1d rhs. 1d array of peak model > values

Re: [eigen] Using Eigen decompositions with ceres autodiff Jet data type.

2020-06-02 Thread Oleg Shirokobrod
Yes, b is measured spectrum that is 1d array. I have to get x for 1d array at a time. I fit sum of peak models to 1d rhs. 1d array of peak model values is one column of matrix A. On Tue 2. Jun 2020 at 20.06, Rasmus Munk Larsen wrote: > Why do you say that? You could be solving for multiple

Re: [eigen] Using Eigen decompositions with ceres autodiff Jet data type.

2020-06-02 Thread Rasmus Munk Larsen
Why do you say that? You could be solving for multiple right-hand sides. Is b know to have 1 column at compile time? On Tue, Jun 2, 2020 at 1:31 AM Oleg Shirokobrod wrote: > Hi Rasmus, > > I have just tested COD decomposition in Eigen library. It arises the same > problem. This is defect of

Re: [eigen] Using Eigen decompositions with ceres autodiff Jet data type.

2020-06-02 Thread Oleg Shirokobrod
Hi Rasmus, I have just tested COD decomposition in Eigen library. It arises the same problem. This is defect of Eigen decomposition module type reduction of result of solve method. If template Matrix A; and ArraXd b;, than x = A.solve(b) should be of type Matrix. I like the idea to use COD

Re: [eigen] Using Eigen decompositions with ceres autodiff Jet data type.

2020-06-02 Thread Oleg Shirokobrod
Rasmus, I wiil have a look at COD. Brad, I did not try CppAD.I am working in given framework: ceres nonlinear least squares solver + ceres autodiff + Eigen decomposition modules SVD or QR. The problem is not just on autodiff side. The problem is that Eigen decomposition modul does not work

Re: [eigen] Using Eigen decompositions with ceres autodiff Jet data type.

2020-06-01 Thread Rasmus Munk Larsen
On Mon, Jun 1, 2020 at 10:33 AM Patrik Huber wrote: > Hi Rasmus, > > This is slightly off-topic to this thread here, but it would be great if > you added your COD to the list/table of decompositions in Eigen: > https://eigen.tuxfamily.org/dox/group__TopicLinearAlgebraDecompositions.html > >

Re: [eigen] Using Eigen decompositions with ceres autodiff Jet data type.

2020-06-01 Thread David Tellenbach
Hi Patrik, Thanks for mentioning. This is a known issue (see e.g.  https://gitlab.com/libeigen/eigen/-/issues/1889) Cheers, David On 1. Jun 2020, 19:33 +0200, Patrik Huber , wrote: > Hi Rasmus, > > This is slightly off-topic to this thread here, but it would be great if you > added your COD to

Re: [eigen] Using Eigen decompositions with ceres autodiff Jet data type.

2020-06-01 Thread Patrik Huber
Hi Rasmus, This is slightly off-topic to this thread here, but it would be great if you added your COD to the list/table of decompositions in Eigen: https://eigen.tuxfamily.org/dox/group__TopicLinearAlgebraDecompositions.html First, it would make it easier for people to find, and second, it

Re: [eigen] Using Eigen decompositions with ceres autodiff Jet data type.

2020-06-01 Thread Rasmus Munk Larsen
The matrix_solve_ls kernel in TensorFlow show a few different variants of how to solve under- and over-determined least-squares problems using Either LLT or COD: https://github.com/tensorflow/tensorflow/blob/master/tensorflow/core/kernels/matrix_solve_ls_op_impl.h#L76 On Mon, Jun 1, 2020 at 9:58

Re: [eigen] Using Eigen decompositions with ceres autodiff Jet data type.

2020-06-01 Thread Rasmus Munk Larsen
Hi Oleg and Sameer, A faster option than SVD, but more robust than QR (since it also handles the under-determined case) is the complete orthogonal decomposition that I implemented in Eigen a few years ago. https://eigen.tuxfamily.org/dox/classEigen_1_1CompleteOrthogonalDecomposition.html (Looks

Re: [eigen] Using Eigen decompositions with ceres autodiff Jet data type.

2020-06-01 Thread Oleg Shirokobrod
Hi Sameer, Thank you for quick reply and advices. 1. I implemented using both decompositions SVD and QR. QR decomposition has the same problem as SVD for Jet but explicit expression for solution is much more complex. 2. I implemented not only CostFunctor with operator()() for autodiff but also

Re: [eigen] Using Eigen decompositions with ceres autodiff Jet data type.

2020-06-01 Thread Sameer Agarwal
Oleg, Two ideas: 1. You may have an easier time using QR factorization instead of SVD to solve your least squares problem. 2. But you can do better, instead of trying to solve linear least squares problem involving a matrix of Jets, you are better off, solving the linear least squares problem on

[eigen] Using Eigen decompositions with ceres autodiff Jet data type.

2020-06-01 Thread Oleg Shirokobrod
Hi list, I am using Eigen 3.3.7 release with ceres solver 1.14.0 with autodiff Jet data type and I have some problems. I need to solve linear least square subproblem within variable projection algorithm, namely I need to solve LLS equation A(p)*x = b Where matrix A(p) depends on nonlinear