[Gretl-devel] Re: qrdecomp with column pivoting?

2022-06-07 Thread Allin Cottrell
On Tue, 7 Jun 2022, Sven Schreiber wrote: Am 06.06.2022 um 18:55 schrieb Cottrell, Allin: Meanwhile (in git), you can now call QR with pivoting, the trigger being a non-null third (matrix-pointer) argument. For m < n (more rows) this seems to crash gretl. (I don't need it, but I kind of

[Gretl-devel] Re: qrdecomp with column pivoting?

2022-06-07 Thread Sven Schreiber
Am 06.06.2022 um 18:55 schrieb Cottrell, Allin: Meanwhile (in git), you can now call QR with pivoting, the trigger being a non-null third (matrix-pointer) argument. For m < n (more rows) this seems to crash gretl. (I don't need it, but I kind of thought it might cause trouble.) cheers sven

[Gretl-devel] Re: qrdecomp with column pivoting?

2022-06-07 Thread Sven Schreiber
Am 06.06.2022 um 23:56 schrieb Cottrell, Allin: On Mon, Jun 6, 2022 at 5:24 PM Sven Schreiber wrote: OTOH, I ran some very crude and simple speed comparisons between qrdecomp with and without pivoting (on Linux, current git), and depending on the input the advantage could go either way it

[Gretl-devel] Re: qrdecomp with column pivoting?

2022-06-06 Thread Cottrell, Allin
On Mon, Jun 6, 2022 at 5:24 PM Sven Schreiber wrote: > > OTOH, I ran some very crude and simple speed comparisons between > qrdecomp with and without pivoting (on Linux, current git), and > depending on the input the advantage could go either way it seemed, > which I found a little surprising.

[Gretl-devel] Re: qrdecomp with column pivoting?

2022-06-06 Thread Sven Schreiber
On 06.06.22 22:33, Cottrell, Allin wrote: Here's what I found: gretl's (SVD) rank() agrees with the numerical rank values shown in the database. So do the results of counting abs(R[i,i]) values from QR, provided the minimal value is in the range 1.0e-10 to 1.0e-8. Plain QR and QR with column

[Gretl-devel] Re: qrdecomp with column pivoting?

2022-06-06 Thread Cottrell, Allin
On Mon, Jun 6, 2022 at 12:13 PM Sven Schreiber wrote: > > What might be useful is to check out the database of singular matrices > at San Jose State U. (It's astounding what things exist in science!) Most of the matrices in that database are square. From the point of view of libgretl I'm

[Gretl-devel] Re: qrdecomp with column pivoting?

2022-06-06 Thread Cottrell, Allin
On Mon, Jun 6, 2022 at 12:13 PM Sven Schreiber wrote: > > Am 06.06.2022 um 17:29 schrieb Cottrell, Allin: > > > Is there any general characterization of what it takes for plain QR > > not to reveal rank? > > > > It would appear that the presence of all-zero rows and/or columns is > > necessary

[Gretl-devel] Re: qrdecomp with column pivoting?

2022-06-06 Thread Sven Schreiber
Am 06.06.2022 um 17:29 schrieb Cottrell, Allin: Is there any general characterization of what it takes for plain QR not to reveal rank? It would appear that the presence of all-zero rows and/or columns is necessary but not sufficient. With m > n you can construct mostly-zero matrices where

[Gretl-devel] Re: qrdecomp with column pivoting?

2022-06-06 Thread Cottrell, Allin
On Mon, Jun 6, 2022 at 6:47 AM Sven Schreiber wrote: > > Am 06.06.2022 um 02:46 schrieb Cottrell, Allin: > > > > I tend to think these are different error conditions: (a) the input > > contains exactly collinear data, and (b) one or more of the input > > series/columns are all zero. > > But that

[Gretl-devel] Re: qrdecomp with column pivoting?

2022-06-06 Thread Sven Schreiber
Am 06.06.2022 um 02:46 schrieb Cottrell, Allin: On Sun, Jun 5, 2022 at 10:21 AM Riccardo (Jack) Lucchetti wrote: On Sun, 5 Jun 2022, Riccardo (Jack) Lucchetti wrote: Very nice counterxample, Sven, kudos for that. Let me think about it. Full disclosure: I didn't invent it myself, but found

[Gretl-devel] Re: qrdecomp with column pivoting?

2022-06-05 Thread Cottrell, Allin
On Sun, Jun 5, 2022 at 8:46 PM Cottrell, Allin wrote: > > I suppose dropcoll() could add a first check for any list element > being all-zero. Though in most cases this would be a waste of time. But also in most cases it won't cost much (since we should reject the all-zeros condition quite

[Gretl-devel] Re: qrdecomp with column pivoting?

2022-06-05 Thread Cottrell, Allin
On Sun, Jun 5, 2022 at 10:21 AM Riccardo (Jack) Lucchetti wrote: > > On Sun, 5 Jun 2022, Riccardo (Jack) Lucchetti wrote: > > > Very nice counterxample, Sven, kudos for that. Let me think about it. > > In fact this bug also affects the dropcoll() built-in function: > > > nulldata 4 > series z =

[Gretl-devel] Re: qrdecomp with column pivoting?

2022-06-05 Thread Riccardo (Jack) Lucchetti
On Sun, 5 Jun 2022, Riccardo (Jack) Lucchetti wrote: Very nice counterxample, Sven, kudos for that. Let me think about it. In fact this bug also affects the dropcoll() built-in function: nulldata 4 series z = 0 series x = 1 | zeros($nobs - 1,1) list X = z x D = dropcoll(X) list print D So

[Gretl-devel] Re: qrdecomp with column pivoting?

2022-06-05 Thread Riccardo (Jack) Lucchetti
On Sat, 4 Jun 2022, Sven Schreiber wrote: Am 04.06.2022 um 15:51 schrieb Sven Schreiber: Am 04.06.2022 um 13:11 schrieb Riccardo (Jack) Lucchetti: On Fri, 3 Jun 2022, Cottrell, Allin wrote: columns the execution time is not that different. Note: we currently assess rank using regular QR,

[Gretl-devel] Re: qrdecomp with column pivoting?

2022-06-04 Thread Sven Schreiber
Am 04.06.2022 um 15:51 schrieb Sven Schreiber: Am 04.06.2022 um 13:11 schrieb Riccardo (Jack) Lucchetti: On Fri, 3 Jun 2022, Cottrell, Allin wrote: columns the execution time is not that different. Note: we currently assess rank using regular QR, by counting the R elements greater than some

[Gretl-devel] Re: qrdecomp with column pivoting?

2022-06-04 Thread Sven Schreiber
Am 04.06.2022 um 13:11 schrieb Riccardo (Jack) Lucchetti: On Fri, 3 Jun 2022, Cottrell, Allin wrote: Maybe, though I gather that for a matrix with a lot more rows than columns the execution time is not that different. Note: we currently assess rank using regular QR, by counting the R

[Gretl-devel] Re: qrdecomp with column pivoting?

2022-06-04 Thread Allin Cottrell
On Sat, 4 Jun 2022, Sven Schreiber wrote: Am 03.06.2022 um 22:46 schrieb Cottrell, Allin: Note: we currently assess rank using regular QR, by counting the R elements greater than some specified "tiny" value. Aha? The doc for "rank" says: "numerically computed via the singular value

[Gretl-devel] Re: qrdecomp with column pivoting?

2022-06-04 Thread Riccardo (Jack) Lucchetti
On Fri, 3 Jun 2022, Cottrell, Allin wrote: On Fri, Jun 3, 2022 at 11:05 AM Sven Schreiber wrote: I'm wondering whether it would be useful (or more precisely, whether the cost-benefit calculation would be net positive...) to generalize the qrdecomp() function to allow column pivoting. Here

[Gretl-devel] Re: qrdecomp with column pivoting?

2022-06-04 Thread Sven Schreiber
Am 03.06.2022 um 22:46 schrieb Cottrell, Allin: On Fri, Jun 3, 2022 at 11:05 AM Sven Schreiber wrote: 2) A QR decomposition with pivoting would provide a rank-revealing operation. The natural workaround and alternative way to do this is SVD. It is my understanding that SVD would be

[Gretl-devel] Re: qrdecomp with column pivoting?

2022-06-03 Thread Cottrell, Allin
On Fri, Jun 3, 2022 at 11:05 AM Sven Schreiber wrote: > > I'm wondering whether it would be useful (or more precisely, whether the > cost-benefit calculation would be net positive...) to generalize the > qrdecomp() function to allow column pivoting. Here are a couple of > thoughts and remarks on