Arrays of data are used are used in almost every modeling application; a
much smaller subset of those applications use complex numbers.
As Peter Baumann has argued:
"Science and engineering data often come as sensor, image, simulation, and
statistics data digitized/sampled from some natural phenomenon or generated
from some simulation. These naturally resemble raster data ("arrays" in
programming) of some particular dimensionality. In the Earth Sciences, for
example, we find 1-D sensor time series, 2-D satellite images, 3-D x/y/t
image time series and x/y/z geophysical cubes, and 4-D x/y/z/t atmosphere
and ocean data. Array data application domains span Earth, Space, Life, and
Social sciences, plus many more. Array Databases are being standardized by
ISO (nickname: "Science SQL")."
https://rd-alliance.org/groups/big-data-analytics-ig/wiki/big-data-use-case-massive-multi-dimensional-arrays.html
For example, in the electric utility industry, looks at "load curves" at
the amount of power used in each hour of the year (24*7*52 = 8,760).
http://www.eia.gov/todayinenergy/detail.cfm?id=4190
The CSV file of the sample load shape can be downloaded here (there are
zeros for the last 2 months):
http://www.eia.gov/todayinenergy/chartdata/AnnualLoadCurve.csv
With "smart metering" utilities have curves like this for every single
customer (except that smart meters measure every 15 minutes (4*8,760)
instead of once per hour). That is a lot different than having a billing
system record for each customer with a name, address and two or three items
per month.
COMPLEX NUMBERS
Calculations involving sine waves such as AC circuits, radio engineering or
seismology typically use complex numbers:
http://en.wikipedia.org/wiki/Mathematics_of_radio_engineering
Since, the advent of SPICE at Berkeley in the 1970s
http://www.ecircuitcenter.com/AboutSPICE.htm
there are a lot of electric circuit simulation models:
http://www.semiwiki.com/forum/showwiki.php?title=Semi+Wiki:EDA+Open+Source+Tools+Wiki
These are all potential users of SQLite3 for an application file format.
So, the application of arrays and complex numbers go far beyond the one
question posed to this list with arrays of integers and reals being far
more common than arrays of complex numbers.
Complex numbers are included as types in C, C#, FORTRAN and Python and are
being added to server databases (Oracle and Microsoft SQL Server) as user
defined types, so programmers and server database administrators may
develop an expectation that complex types or user defined types will be
supported SQLite3.
But, even so, I would expect that users of complex numbers would be an
extremely small subset of data base users (less than 5%? or even less than
1%?).
Jim Callahan
Orlando, FL
On Mon, Apr 27, 2015 at 12:48 PM, Drago, William @ CSG - NARDA-MITEQ <
William.Drago at l-3com.com> wrote:
> All,
>
> I've been enjoying this discussion and have learned a thing or two from
> all the suggestions.
>
> My particular problem is indeed solved. Adding a simple blob to my
> original table is the best solution in this case. If you think about the
> trace data as simply a screenshot of the analyzer display then I think it
> is easier to understand why I don't want to bother with more complex
> solutions. Even though they may be technically more correct, they are
> overkill for what I'm trying to accomplish. I'm just saving a screenshot,
> minus the graticule and all the other decorations on the display. There's
> no need to make it any more complicated than that. I don't even have to
> save the X-axis elements as those are derived from other parameters that
> are already known. Someone asked if the trace data was time stamped. It is
> not. Date and time of the test are stored along with all the other relevant
> information.
>
> If I ever have to deal with complex numbers as my core data I have some
> pretty good ideas on how to handle them now. BTW, the languages I use most,
> VEE & C#, both support complex data types. Given the amount of use database
> programs see in the scientific world I'm surprised there is no native
> complex data type (I can appreciate the difficulty in implementing such a
> thing). Array support for each type would be nice too. We have this in VEE
> and it is so easy to use. But for now every programmer working in science
> or engineering has to re-invent the wheel.
>
> For the curious here are the analyzers I'm working with:
>
> http://www.keysight.com/en/pc-1000001745%3Aepsg%3Apgr/pna-network-analyzers-300-khz-to-11-thz?cc=US&lc=eng
>
> --
> Bill Drago
> Senior Engineer
> L3 Narda-MITEQ
> 435 Moreland Road
> Hauppauge, NY 11788
> 631-272-5947 / William.Drago at L-3COM.com
>
>
>
> > -----Original Message-----
> > From: sqlite-users-bounces at mailinglists.sqlite.org [mailto:sqlite-
> > users-bounces at mailinglists.sqlite.org] On Behalf Of Jim Callahan
> > Sent: Sunday, April 26, 2015 9:56 PM
> > To: General Discussion of SQLite Database
> > Subject: [sqlite] SQLite and Scientific Computing: Arrays and Complex
> > Numbers
> >
> > The original thread asking about an array of complex numbers has been
> > marked as "solved." The requester has decided to serialize the complex
> > numbers and store them in a blob. Earlier, Keith had suggested storing
> > complex numbers as a pair of real numbers and a separate box table. I
> > extended Keith's suggestion with two or three tables, elements, arrays
> > and optionally coordinates.
> >
> > There is some literature on storing arrays in SQL databases. In
> > addition complex numbers seem to be the orphan stepchild of programming
> > languages (let alone databases). Although FORTRAN IV had complex
> > numbers they were not added to the C standard until C99.
> >
> > Language / Standard / Library
> > ----------------------------------------
> > C / C99/ complex.h
> > http://en.wikibooks.org/wiki/C_Programming/C_Reference/complex.h
> > C# / 4.0 / System.Numerics.Complex
> > https://msdn.microsoft.com/en-
> > us/library/system.numerics.complex(v=vs.110).aspx
> >
> > Java /?/ Apache Commons
> > Python/2.6.5/ cmath
> > https://docs.python.org/2/library/cmath.html
> >
> > http://en.wikipedia.org/wiki/Complex_data_type
> >
> > So, I suppose if a company wanted to sponsor it, complex numbers could
> > be supported through an addin library similar to FTS3 and FTS4 for full
> > text searches.
> > http://sqlite.org/fts3.html
> >
> > Here for example, is a discussion on IBM DeveloperWorks concerning the
> > Informix database.
> > https://www.ibm.com/developerworks/community/blogs/gbowerman/entry/sql_
> > and_the_complex_plane?lang=en
> >
> > Some databases have Abstract Defined Type (Oracle) or User Defined
> > Types (Microsoft SQL Server) that could be used for complex numbers.
> >
> > Ironically, the scientific data format NetCDF did not have provision
> > for complex numbers (it was designed for weather data).
> > https://www.unidata.ucar.edu/mailing_lists/archives/netcdfgroup/2011/ms
> > g00027.html
> >
> > There are some discusssions of scientific versions of SQL (such as
> > SciQL):
> >
> > "A Query Language for Multidimensional Arrays:
> > Design, Implementation, and Optimization Techniques"
> > http://homepages.inf.ed.ac.uk/libkin/papers/sigmod96a.pdf
> >
> > "Requirements for Science Data Bases and SciDB"
> > http://www-db.cs.wisc.edu/cidr/cidr2009/Paper_26.pdf
> >
> > "SciQL, A Query Language for Science Applications"
> > http://homepages.cwi.nl/~zhang/papers/arraydb11.pdf
> >
> > Jim
> > _______________________________________________
> > sqlite-users mailing list
> > sqlite-users at mailinglists.sqlite.org
> > http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users
> CONFIDENTIALITY, EXPORT CONTROL AND DISCLAIMER NOTE:This e-mail and any
> attachments are solely for the use of the addressee and may contain
> information that is privileged or confidential. Any disclosure, use or
> distribution of the information contained herein is prohibited. In the
> event this e-mail contains technical data within the definition of the
> International Traffic in Arms Regulations or Export Administration
> Regulations, it is subject to the export control laws of the
> U.S.Government. The recipient should check this e-mail and any attachments
> for the presence of viruses as L-3 does not accept any liability associated
> with the transmission of this e-mail. If you have received this
> communication in error, please notify the sender by reply e-mail and
> immediately delete this message and any attachments.
> _______________________________________________
> sqlite-users mailing list
> sqlite-users at mailinglists.sqlite.org
> http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users
>
