Re: [R] RSQLite slowness

2021-10-07 Thread Rasmus Liland
Dear Martin Morgan, Thanks for all those links! Yes, my question can be characterized like that I think, traditional way writing a temporary table into the database and left JOINing the others vs. parameterized query. A relevant example would be to first create the database from the

Re: [R] RSQLite slowness

2021-10-07 Thread Rasmus Liland
On 2021-10-06 12:11 -0700, Jeff Newmiller wrote: > FWIW all SQL implementations work > better with indexes An index seems to be a good way to improve sql performance, I'll look into it. Best, Rasmus signature.asc Description: PGP signature __

Re: [R] RSQLite slowness

2021-10-07 Thread Rasmus Liland
Dear Ivan, Thanks for that explaination! I think it explains the slowness clearly. It is possible to use carray is in Rust [1] so it might be available in R in the future(?) I'll look into rusqlite some time at least. sqlite is supposed to be one of the fastest sql implementations.

Re: [R] RSQLite slowness

2021-10-07 Thread Ivan Krylov
On Wed, 6 Oct 2021 16:23:15 + Rasmus Liland wrote: >"SELECT * FROM gene2refseq > LEFT JOIN gene_info ON > gene_info.GeneID = gene2refseq.GeneID > WHERE gene2refseq.`RNA_nucleotide_accession.version` > LIKE ?" <...> > x1 <-

Re: [R] RSQLite slowness

2021-10-06 Thread Jeff Newmiller
Since the sqlite package is contributed, it is NOT related to "core R", and is in fact technically off-topic on this list. FWIW all SQL implementations work better with indexes, but AFAIK the R data frame support does nothing with indexes. This may be related to your question, or not. I am not

Re: [R] RSQLite slowness

2021-10-06 Thread Martin Morgan
https://support.bioconductor.org and the community slack (sign up at https://bioc-community.herokuapp.com/ ) as well as the general site https://bioconductor.org . Actually your question sounds like a SQLite question � JOIN a table, versus parameterized query. One could perhaps construct the

Re: [R] RSQLite slowness

2021-10-06 Thread Rasmus Liland
Thank you Bert, I set up a new thread on BioStars [1]. So far, I'm a bit unfamilliar with Bioconductor (but will hopefully attend a course about it in November, which I'm kinda hyped about), other than installing and updating R packages using BiocManager Did you think of something

Re: [R] RSQLite slowness

2021-10-06 Thread Bert Gunter
Generally, such gene-related questions are better asked on Bioconductor than here. They also might know of more efficient, purpose built tools for your efforts there. No guarantees, of course, and you might get a helpful response here. But if not ... Bert Gunter "The trouble with having an open

[R] RSQLite slowness

2021-10-06 Thread Rasmus Liland
Dear r-help readers, why is it so much slower to query an sqlite database using RSQlite «from the outside» using param like statement <- "SELECT * FROM gene2refseq LEFT JOIN gene_info ON gene_info.GeneID = gene2refseq.GeneID WHERE