Re: [Lazarus] data matrix with thousands of columns

2013-04-04 Thread Andrea Mauri
Il 03/04/2013 01:00, ListMember ha scritto: On 2013-03-26 16:38, Andrea Mauri wrote: I am looking for the best option in order to store big datasets with thousands of columns. The dataset can contains from tens to hundred thousands lines and thousand of columns (some columns are string some

Re: [Lazarus] data matrix with thousands of columns

2013-04-02 Thread ListMember
On 2013-03-26 16:38, Andrea Mauri wrote: I am looking for the best option in order to store big datasets with thousands of columns. The dataset can contains from tens to hundred thousands lines and thousand of columns (some columns are string some numbers). Have you looked at '/Synopse Big

Re: [Lazarus] data matrix with thousands of columns

2013-03-29 Thread Marc Santhoff
Am Dienstag, den 26.03.2013, 15:44 +0100 schrieb Andrea Mauri: one more thing. my data is more similar to a huge spreadsheet than a relational DB, anyway I am looking for the best option already available in lazarus/fpc to store and query the dataset Il 26/03/2013 15:38, Andrea Mauri ha

Re: [Lazarus] data matrix with thousands of columns

2013-03-29 Thread Alexander Klenin
On Wed, Mar 27, 2013 at 7:18 PM, Andrea Mauri andrea.mauri...@gmail.com wrote: Il 27/03/2013 16:11, Max Vlasov ha scritto: CREATE TABLE [Data] ( ... , [Row] INTEGER, [Column] INTEGER, [Value] TEXT ... ) As I answered to Leonardo I tried something like this but I have to insert millions

Re: [Lazarus] data matrix with thousands of columns

2013-03-29 Thread Mark Morgan Lloyd
Alexander Klenin wrote: On Wed, Mar 27, 2013 at 7:18 PM, Andrea Mauri andrea.mauri...@gmail.com wrote: Il 27/03/2013 16:11, Max Vlasov ha scritto: CREATE TABLE [Data] ( ... , [Row] INTEGER, [Column] INTEGER, [Value] TEXT ... ) As I answered to Leonardo I tried something like this but I have

Re: [Lazarus] data matrix with thousands of columns

2013-03-28 Thread Michael Schnell
On 03/27/2013 10:24 AM, Andrea Mauri wrote: My app is actually compiled both on 32 and 64 bit. As this is a really typical application that requests for 64 Bit I would drop the 32 Bit route here. -Michael -- ___ Lazarus mailing list

Re: [Lazarus] data matrix with thousands of columns

2013-03-28 Thread Michael Schnell
On 03/27/2013 12:48 PM, Mark Morgan Lloyd wrote: Even if half the address room (i.e the upper Bit) is stolen by whatever organization. 63 Bits is by far enough for any thinkable purpose. -Michael -- ___ Lazarus mailing list

Re: [Lazarus] data matrix with thousands of columns

2013-03-28 Thread Mark Morgan Lloyd
Michael Schnell wrote: On 03/27/2013 12:48 PM, Mark Morgan Lloyd wrote: Even if half the address room (i.e the upper Bit) is stolen by whatever organization. 63 Bits is by far enough for any thinkable purpose. And then somebody steals another bit, and somebody else a couple more... and

Re: [Lazarus] data matrix with thousands of columns

2013-03-27 Thread Michael Schnell
On 03/26/2013 03:44 PM, Andrea Mauri wrote: one more thing. my data is more similar to a huge spreadsheet than a relational DB store: Do you mean for working with it in realtime or for keeping it for the next time the program is started ? When doing a with a 64 bit program it might be

Re: [Lazarus] data matrix with thousands of columns

2013-03-27 Thread Andrea Mauri
Il 27/03/2013 09:27, Michael Schnell ha scritto: On 03/26/2013 03:44 PM, Andrea Mauri wrote: one more thing. my data is more similar to a huge spreadsheet than a relational DB store: Do you mean for working with it in realtime or for keeping it for the next time the program is started ? Ok

Re: [Lazarus] data matrix with thousands of columns

2013-03-27 Thread Mark Morgan Lloyd
Andrea Mauri wrote: Il 27/03/2013 09:27, Michael Schnell ha scritto: On 03/26/2013 03:44 PM, Andrea Mauri wrote: one more thing. my data is more similar to a huge spreadsheet than a relational DB store: Do you mean for working with it in realtime or for keeping it for the next time the

Re: [Lazarus] data matrix with thousands of columns

2013-03-27 Thread Michael Schnell
On 03/27/2013 10:24 AM, Andrea Mauri wrote: but I am inhabited to not put all the data in memory, maybe I need to change my mind. Is it an option to not store on files but to take everything in memory and let app and os to perform the dirty work? That is the main reason why 64 Bit has been

Re: [Lazarus] data matrix with thousands of columns

2013-03-27 Thread Mark Morgan Lloyd
Michael Schnell wrote: On 03/27/2013 10:24 AM, Andrea Mauri wrote: but I am inhabited to not put all the data in memory, maybe I need to change my mind. Is it an option to not store on files but to take everything in memory and let app and os to perform the dirty work? That is the main

Re: [Lazarus] data matrix with thousands of columns

2013-03-27 Thread Reinier Olislagers
On 27-3-2013 10:49, Mark Morgan Lloyd wrote: Andrea Mauri wrote: Il 27/03/2013 09:27, Michael Schnell ha scritto: On 03/26/2013 03:44 PM, Andrea Mauri wrote: Ok I will explain better. I have a GUI app. The user loads samples (the rows), my app performs calculations on samples and for

Re: [Lazarus] data matrix with thousands of columns

2013-03-27 Thread Max Vlasov
On Wed, Mar 27, 2013 at 1:24 PM, Andrea Mauri andrea.mauri...@gmail.comwrote: After calculations the app/user should be able to search for one/more samples (or for one/more columns) getting all/some values for the sample/column. Briefly I need to be able to rapidly get some values from this

[Lazarus] data matrix with thousands of columns

2013-03-26 Thread Andrea Mauri
Dear all, I am looking for the best option in order to store big datasets with thousands of columns. The dataset can contains from tens to hundred thousands lines and thousand of columns (some columns are string some numbers). Which is the best option to store and retrieve information from a

Re: [Lazarus] data matrix with thousands of columns

2013-03-26 Thread Andrea Mauri
one more thing. my data is more similar to a huge spreadsheet than a relational DB, anyway I am looking for the best option already available in lazarus/fpc to store and query the dataset Il 26/03/2013 15:38, Andrea Mauri ha scritto: Dear all, I am looking for the best option in order to

Re: [Lazarus] data matrix with thousands of columns

2013-03-26 Thread Leonardo M . Ramé
On 2013-03-26 15:38:57 +0100, Andrea Mauri wrote: Dear all, I am looking for the best option in order to store big datasets with thousands of columns. The dataset can contains from tens to hundred thousands lines and thousand of columns (some columns are string some numbers). Which is the

Re: [Lazarus] data matrix with thousands of columns

2013-03-26 Thread Andrea Mauri
If you think you'll be adding columns regularly, one solution that comes to my mind is this: 1) Create a data_columns table: create table data_columns( idcolumn integer, column_name varchar(20), primary key(idcolumn)); 2) Create the data_rows table: create table data_rows(

Re: [Lazarus] data matrix with thousands of columns

2013-03-26 Thread Leonardo M . Ramé
On 2013-03-26 21:22:26 +0100, Andrea Mauri wrote: If you think you'll be adding columns regularly, one solution that comes to my mind is this: 1) Create a data_columns table: create table data_columns( idcolumn integer, column_name varchar(20), primary key(idcolumn)); 2)