Oops, left out two links, the Rasdaman 10.0 quote is from the Rasdaman roadmap. http://rasdaman.org/roadmap
and the quote about the ISO designation is from the Array SQL Rasdaman page: http://rasdaman.com/ArraySQL/ But, the conclusion remains the same: So, Multi-dimensional arrays may be coming to SQL via the ISO Standard 9075 and to SQLite3 via Rasdaman, but only if, Rasdaman decides to use SQLite3 as their default backend in their 10.0.0 release. Jim Callahan Orlando, FL On Mon, Apr 27, 2015 at 12:33 PM, Jim Callahan < jim.callahan.orlando at gmail.com> wrote: > Two more thoughts (and I have to get back to work!): > 1. Rasdaman > 2. ISO SQL/MDA (multi-dimensional arrays) > > In June 2014, British tech website "The Register" reported, "the ISO SQL > working group agreed to start work on SQL/MDA (multi-dimensional array) > specs. ...A separate effort, called Rasdaman <http://rasdaman.org/> (a > scalable multi-dimensional array analytics server) has been working for > some time to apply an SQL-like query language to array databases. > Rasdaman's backer, Peter Baumann of Jacobs University Bremen in Germany, > put forward the proposal now adopted by the ISO." > > http://www.theregister.co.uk/2014/06/26/sql_to_worlddog_we_doing_big_data_too/ > > Apparently, Rasdaman runs with PostgreSQL, but is considering having > SQLite as a default: > > "Suggested *big* changes for 10.0 > Improve UX by removing big external dependencies (by default) > > - rasserver: use *SQLite*/filesystem backend by default instead of > PostgreSQL > - petascope: use embedded ASQLDB instead of PostgreSQL > - petascope: local deployment with an embedded jetty in $RMANHOME/bin > for example, rather than in an external Tomcat. This would be managed by > similarly start/stop_petascope.sh for example > ? " > > Rasdaman lists the official ISO designation: > "ISO Array-SQL: > ... > It's ISO: In June 2014, ISO/IEC JTC1 SC32 WG3 has decided to establish ISO > 9075 Part 15: SQL/MDA as ISO's latest SQL extension, specifically for Big > Science Data." > > The only semi-official (non-Rasdaman) ISO document I could find (with one > or two Google searches) was this presentation: > > "WG3 is incorporating two interesting new capabilities into > the next version of 9075: > ? Support for JSON ... [detail omitted] > ? Support for Multi Dimensional Arrays ... [detail omitted]" > http://jtc1sc32.org/doc/N2501-2550/32N2528-WG3-Tutorial-Opening-Plenary.pdf > > So, Multi-dimensional arrays may be coming to SQL via the ISO Standard > 9075 and to SQLite3 via Rasdaman, but only if, Rasdaman decides to use > SQLite3 as their default backend in their 10.0.0 release. > > Jim Callahan > Orlando, FL > > > > On Mon, Apr 27, 2015 at 11:23 AM, Jim Callahan < > jim.callahan.orlando at gmail.com> wrote: > >> Two more thoughts (and you'll stay with SQLite): >> 1. SciQL -> MonetDB -> sockets -> embedding R in MonetDB (ala PostgresQL) >> http://en.wikipedia.org/wiki/MonetDB >> >> "there was one issue that we could not solve: Processing data from the >> database with R required transferring the relevant data over the socket >> first. This is fine and "fast enough" for up to ? say ? several thousand >> rows, but not for much more. We have had a lot of demand for transferring >> larger amounts of data from users. Hence, we chose to go in a different >> direction. >> Starting with the Oct2014 release, MonetDB will ship with a feature we >> call *R-Integration*. R has some support for running embedded >> <http://developer.r-project.org/embedded.html> in another application, a >> fact we leverage heavily. What it does is make R scripts a first class >> citizen of the SQL world in MonetDB." >> https://www.monetdb.org/content/embedded-r-monetdb >> >> So, instead of having an embedded database (like SQLite3) embedded in a >> language, (like Python or R), they windup having the database swallow the >> language!!! (embeding the language in the database -- instead of the other >> way arround). >> >> 2. ROLAP -> Mondrian -> TomCat >> As opposed the the scientific community, the business community thinks in >> terms of "cubes" rather than arrays which gets one to Online Analytic >> Processing (OLAP) or when implemented over a relational database: ROLAP. Or >> the whole thing gets rebranded by marketing as "Business Intelligence" or >> "Analysis Services" or "Pivot Tables". >> The open source ROLAP tool is "Mondrian". >> >> Mondrian can run over SQLite, but seems to run more often with a Java >> application server such as TomCat or JBoss. >> >> Mondrian Cube with SQLite >> This entry was written by haoccheng, posted on October 7, 2012 >> "Mondrian is an in-memory OLAP cube (cache) built on top of relational >> database. Cube is a multi-dimensional data structure. A cell holds numeric >> measurements of a business; aggregation can be executed efficiently along >> dimensions. MDX is the query language of Mondrian and a query can be broken >> down to lookup at the cube (cache hit) and direct SQL query against the >> underneath relational database. >> I play with the Mondrian and SQLite. >> #1: Create a sqlite database (I borrow the data model from the Mondrian >> tutorial by Slawomir, architect of Pentaho)." >> https://haoccheng.wordpress.com/2012/10/07/mondrian-cube-with-sqlite/ >> >> Creating a basic Mondrian OLAP Cube >> BY SLAWOMIR CHODNICKI, ON JULY 10TH, 2010 >> "If you?d like to follow the examples you will need access to a >> database, a copy of Pentaho Kettle and a Mondrian installation. I will be >> using MySQL as RDBMS and JasperServer 3.7. CE >> <http://jasperforge.org/projects/jasperserver> for the Mondrian >> installation. Other possibilities include Pentaho BI-Server >> <http://sourceforge.net/projects/pentaho/files/Business%20Intelligence%20Server/> >> and >> a bare bones Mondrian/JPivot >> <http://sourceforge.net/projects/mondrian/files/> and PAT >> <http://code.google.com/p/pentahoanalysistool/> installation." >> >> http://type-exit.org/adventures-with-open-source-bi/2010/07/creating-a-basic-mondrian-olap-cube/ >> >> So, taking any of the paths (MonetDB or Mondrian) gets you to complex >> server setups pretty quickly, the one exception is haoccheng makes a brake >> attempt to setup Mondrian over SQLite (but even his attempt requires a >> TomCat Java application server): >> >> "#3: Install the Tomcat; verify the http://localhost:8080 up and running. >> #4: Download Mondrian; unzip mondrian-version.war to tomcat_home/webapps/ >> folder. >> #5: Get the sqlite jdbc driver: sqlite-jdbc-version.jar; deploy to >> tomcat_home/webapps/mondrian/WEB-INF/lib >> #6: Change the connection string in Mondrian web.xml >> (tomcat_home/webapps/mondrian/WEB-INF) to point to the sqlite database." >> >> >> Almost makes one pine for MS Excel PivotTables. >> >> Jim Callahan >> Orlando, FL >> >> On Sun, Apr 26, 2015 at 9:55 PM, Jim Callahan < >> jim.callahan.orlando at gmail.com> wrote: >> >>> The original thread asking about an array of complex numbers has been >>> marked as "solved." The requester has decided to serialize the complex >>> numbers and store them in a blob. Earlier, Keith had suggested storing >>> complex numbers as a pair of real numbers and a separate box table. I >>> extended Keith's suggestion with two or three tables, elements, arrays and >>> optionally coordinates. >>> >>> There is some literature on storing arrays in SQL databases. In addition >>> complex numbers seem to be the orphan stepchild of programming languages >>> (let alone databases). Although FORTRAN IV had complex numbers they were >>> not added to the C standard until C99. >>> >>> Language / Standard / Library >>> ---------------------------------------- >>> C / C99/ complex.h >>> http://en.wikibooks.org/wiki/C_Programming/C_Reference/complex.h >>> C# / 4.0 / System.Numerics.Complex >>> >>> https://msdn.microsoft.com/en-us/library/system.numerics.complex(v=vs.110).aspx >>> >>> Java /?/ Apache Commons >>> Python/2.6.5/ cmath >>> https://docs.python.org/2/library/cmath.html >>> >>> http://en.wikipedia.org/wiki/Complex_data_type >>> >>> So, I suppose if a company wanted to sponsor it, complex numbers could >>> be supported through an addin library similar to FTS3 and FTS4 for full >>> text searches. >>> http://sqlite.org/fts3.html >>> >>> Here for example, is a discussion on IBM DeveloperWorks concerning the >>> Informix database. >>> >>> https://www.ibm.com/developerworks/community/blogs/gbowerman/entry/sql_and_the_complex_plane?lang=en >>> >>> Some databases have Abstract Defined Type (Oracle) or User Defined Types >>> (Microsoft SQL Server) that could be used for complex numbers. >>> >>> Ironically, the scientific data format NetCDF did not have provision for >>> complex numbers (it was designed for weather data). >>> >>> https://www.unidata.ucar.edu/mailing_lists/archives/netcdfgroup/2011/msg00027.html >>> >>> There are some discusssions of scientific versions of SQL (such as >>> SciQL): >>> >>> "A Query Language for Multidimensional Arrays: >>> Design, Implementation, and Optimization Techniques" >>> http://homepages.inf.ed.ac.uk/libkin/papers/sigmod96a.pdf >>> >>> "Requirements for Science Data Bases and SciDB" >>> http://www-db.cs.wisc.edu/cidr/cidr2009/Paper_26.pdf >>> >>> "SciQL, A Query Language for Science Applications" >>> http://homepages.cwi.nl/~zhang/papers/arraydb11.pdf >>> >>> Jim >>> >>> >>> >>> >>> >>> >>> >> >
