MonetDB: iot - Approve output
Changeset: e0598c7af394 for MonetDB URL: http://dev.monetdb.org/hg/MonetDB?cmd=changeset;node=e0598c7af394 Removed Files: sql/backends/monet5/iot/Tests/cleanup.sql sql/backends/monet5/iot/Tests/cleanup.stable.err sql/backends/monet5/iot/Tests/cleanup.stable.out sql/backends/monet5/iot/Tests/iot05.stable.err sql/backends/monet5/iot/Tests/iot05.stable.out Modified Files: sql/backends/monet5/iot/Tests/All sql/backends/monet5/iot/Tests/inputoutput.sql sql/backends/monet5/iot/Tests/inputoutput.stable.out sql/backends/monet5/iot/Tests/iot00.sql sql/backends/monet5/iot/Tests/iot00.stable.out sql/backends/monet5/iot/Tests/iot02.sql sql/backends/monet5/iot/Tests/iot03.sql sql/backends/monet5/iot/Tests/iot10.sql sql/backends/monet5/iot/Tests/iot10.stable.out sql/backends/monet5/iot/Tests/iot12.sql sql/backends/monet5/iot/Tests/receptor00.sql sql/backends/monet5/iot/Tests/receptor00.stable.out sql/backends/monet5/iot/Tests/receptor01.sql Branch: iot Log Message: Approve output diffs (truncated from 485 to 300 lines): diff --git a/sql/backends/monet5/iot/Tests/All b/sql/backends/monet5/iot/Tests/All --- a/sql/backends/monet5/iot/Tests/All +++ b/sql/backends/monet5/iot/Tests/All @@ -1,13 +1,11 @@ iot00 iot02 iot03 -#iot05 iot10 iot12 #iot15 receptor00 receptor01 -cleanup #petrinet00 webtest inputoutput diff --git a/sql/backends/monet5/iot/Tests/cleanup.sql b/sql/backends/monet5/iot/Tests/cleanup.sql deleted file mode 100644 --- a/sql/backends/monet5/iot/Tests/cleanup.sql +++ /dev/null @@ -1,9 +0,0 @@ --- initialize the stream testing environment -set schema iot; - -drop procedure cq00; -drop procedure clk1; -drop procedure clk3; -drop table result; -drop table stmp; -drop table stmp1; diff --git a/sql/backends/monet5/iot/Tests/cleanup.stable.err b/sql/backends/monet5/iot/Tests/cleanup.stable.err deleted file mode 100644 --- a/sql/backends/monet5/iot/Tests/cleanup.stable.err +++ /dev/null @@ -1,34 +0,0 @@ -stderr of test 'cleanup` in directory 'sql/backends/monet5/iot` itself: - - -# 00:19:46 > -# 00:19:46 > "mserver5" "--debug=10" "--set" "gdk_nr_threads=0" "--set" "mapi_open=true" "--set" "mapi_port=38802" "--set" "mapi_usock=/var/tmp/mtest-14198/.s.monetdb.38802" "--set" "monet_prompt=" "--forcemito" "--dbpath=/export/scratch1/mk/iot//Linux/var/MonetDB/mTests_sql_backends_monet5_iot" -# 00:19:46 > - -# builtin opt gdk_dbpath = /export/scratch1/mk/iot//Linux/var/monetdb5/dbfarm/demo -# builtin opt gdk_debug = 0 -# builtin opt gdk_vmtrim = no -# builtin opt monet_prompt = > -# builtin opt monet_daemon = no -# builtin opt mapi_port = 5 -# builtin opt mapi_open = false -# builtin opt mapi_autosense = false -# builtin opt sql_optimizer = default_pipe -# builtin opt sql_debug = 0 -# cmdline opt gdk_nr_threads = 0 -# cmdline opt mapi_open = true -# cmdline opt mapi_port = 38802 -# cmdline opt mapi_usock = /var/tmp/mtest-14198/.s.monetdb.38802 -# cmdline opt monet_prompt = -# cmdline opt gdk_dbpath = /export/scratch1/mk/iot//Linux/var/MonetDB/mTests_sql_backends_monet5_iot -# cmdline opt gdk_debug = 536870922 - -# 00:19:46 > -# 00:19:46 > "mclient" "-lsql" "-ftest" "-Eutf-8" "-i" "-e" "--host=/var/tmp/mtest-14198" "--port=38802" -# 00:19:46 > - - -# 00:19:46 > -# 00:19:46 > "Done." -# 00:19:46 > - diff --git a/sql/backends/monet5/iot/Tests/cleanup.stable.out b/sql/backends/monet5/iot/Tests/cleanup.stable.out deleted file mode 100644 --- a/sql/backends/monet5/iot/Tests/cleanup.stable.out +++ /dev/null @@ -1,40 +0,0 @@ -stdout of test 'cleanup` in directory 'sql/backends/monet5/iot` itself: - - -# 00:19:46 > -# 00:19:46 > "mserver5" "--debug=10" "--set" "gdk_nr_threads=0" "--set" "mapi_open=true" "--set" "mapi_port=38802" "--set" "mapi_usock=/var/tmp/mtest-14198/.s.monetdb.38802" "--set" "monet_prompt=" "--forcemito" "--dbpath=/export/scratch1/mk/iot//Linux/var/MonetDB/mTests_sql_backends_monet5_iot" -# 00:19:46 > - -# MonetDB 5 server v11.24.0 -# This is an unreleased version -# Serving database 'mTests_sql_backends_monet5_iot', using 8 threads -# Compiled for x86_64-unknown-linux-gnu/64bit with 64bit OIDs and 128bit integers dynamically linked -# Found 15.589 GiB available main-memory. -# Copyright (c) 1993-July 2008 CWI. -# Copyright (c) August 2008-2016 MonetDB B.V., all rights reserved -# Visit http://www.monetdb.org/ for further information -# Listening for connection requests on mapi:monetdb://vienna.da.cwi.nl:38802/ -# Listening for UNIX domain connection requests on mapi:monetdb:///var/tmp/mtest-14198/.s.monetdb.38802 -# MonetDB/GIS module loaded -# MonetDB/SQL module loaded -# MonetDB/iot loaded - -Ready. - -# 00:19:46 > -# 00:19:46 > "mclient" "-lsql" "-ftest" "-Eutf-8" "-i" "-e" "--host=/var/tmp/mtest-14198" "--port=38802" -# 00:19:46 > - -#set schema iot; -#drop procedure cq00; -#drop
MonetDB: iot - Add update operation
Changeset: 9ac25e667365 for MonetDB URL: http://dev.monetdb.org/hg/MonetDB?cmd=changeset;node=9ac25e667365 Modified Files: sql/backends/monet5/iot/basket.mal Branch: iot Log Message: Add update operation diffs (14 lines): diff --git a/sql/backends/monet5/iot/basket.mal b/sql/backends/monet5/iot/basket.mal --- a/sql/backends/monet5/iot/basket.mal +++ b/sql/backends/monet5/iot/basket.mal @@ -30,6 +30,10 @@ pattern append(mvc:int, sch:str, tbl:str address BSKTappend comment "Append new tuples to a basket"; +pattern update(mvc:int, sch:str, tbl:str, col:str, rid:bat[:oid],val:bat[:any]):int +address BSKTupdate +comment "Update tuples in a basket"; + pattern clear_table(sname:str,tname:str):lng address mvc_clear_table_wrap; ___ checkin-list mailing list checkin-list@monetdb.org https://www.monetdb.org/mailman/listinfo/checkin-list
MonetDB: iot - Add a private MAL client per continous query
Changeset: ae4e6e2095ec for MonetDB
URL: http://dev.monetdb.org/hg/MonetDB?cmd=changeset;node=ae4e6e2095ec
Modified Files:
sql/backends/monet5/iot/50_iot.sql
sql/backends/monet5/iot/basket.c
sql/backends/monet5/iot/basket.h
sql/backends/monet5/iot/iot.c
sql/backends/monet5/iot/petrinet.c
Branch: iot
Log Message:
Add a private MAL client per continous query
always make the errors table accessiblg
diffs (246 lines):
diff --git a/sql/backends/monet5/iot/50_iot.sql
b/sql/backends/monet5/iot/50_iot.sql
--- a/sql/backends/monet5/iot/50_iot.sql
+++ b/sql/backends/monet5/iot/50_iot.sql
@@ -94,9 +94,9 @@ create function iot.outputs()
returns table( "s" string, "t" string, "sch" string, "qry" string)
external name iot.outputplaces;
--- create function iot.errors()
--- returns table( "schema" string, "table" string, error string)
--- external name iot.errors;
+create function iot.errors()
+returns table( "table" string, error string)
+external name iot.errors;
-- tables for iotwebserver
diff --git a/sql/backends/monet5/iot/basket.c b/sql/backends/monet5/iot/basket.c
--- a/sql/backends/monet5/iot/basket.c
+++ b/sql/backends/monet5/iot/basket.c
@@ -74,8 +74,8 @@ BSKTclean(int idx)
baskets[idx].table_name = NULL;
BBPreclaim(baskets[idx].errors);
+ baskets[idx].errors = NULL;
baskets[idx].winstride = -1;
- baskets[idx].errors = NULL;
baskets[idx].count = 0;
}
for(idx = 1; idx < bsktTop; idx++){
@@ -85,8 +85,8 @@ BSKTclean(int idx)
baskets[idx].table_name = NULL;
BBPreclaim(baskets[idx].errors);
+ baskets[idx].errors = NULL;
baskets[idx].winstride = -1;
- baskets[idx].errors = NULL;
baskets[idx].count = 0;
}
}
@@ -607,15 +607,11 @@ BSKTdump(void *ret)
int bskt;
BUN cnt;
BAT *b;
- mvc *m = NULL;
str msg = MAL_SUCCEED;
mnstr_printf(GDKout, "#baskets table\n");
for (bskt = 1; bskt < bsktLimit; bskt++)
if (baskets[bskt].table_name) {
- msg = getSQLContext(mal_clients, 0, , NULL);
- if ( msg != MAL_SUCCEED)
- break;
cnt = 0;
b = baskets[bskt].bats[0];
if( b)
@@ -684,6 +680,48 @@ BSKTappend(Client cntxt, MalBlkPtr mb, M
}
str
+BSKTupdate(Client cntxt, MalBlkPtr mb, MalStkPtr stk, InstrPtr pci)
+{
+int *res = getArgReference_int(stk, pci, 0);
+str sname = *getArgReference_str(stk, pci, 2);
+str tname = *getArgReference_str(stk, pci, 3);
+str cname = *getArgReference_str(stk, pci, 4);
+bat rows = *getArgReference_bat(stk, pci, 5);
+bat val = *getArgReference_bat(stk, pci, 6);
+BAT *bn=0, *rid=0, *bval = 0;
+ int bskt;
+
+ return 0;
+ (void) cntxt;
+ (void) mb;
+*res = 0;
+
+rid = BATdescriptor(rows);
+ if( rid == NULL)
+throw(SQL, "basket.append", "Cannot access source oid descriptor");
+bval = BATdescriptor(val);
+ if( bval == NULL){
+ BBPunfix(rid->batCacheid);
+throw(SQL, "basket.append", "Cannot access source descriptor");
+ }
+
+ bskt = BSKTlocate(sname,tname);
+ if( bskt == 0)
+ throw(SQL, "basket.append", "Cannot access basket descriptor
%s.%s",sname,tname);
+ bn = BSKTbindColumn(sname,tname,cname);
+
+ if( bn){
+ void_replace_bat(bn, rid, bval, TRUE);
+ BATderiveProps(bn, FALSE);
+ } else throw(SQL, "basket.append", "Cannot access target column
%s.%s.%s",sname,tname,cname);
+
+ baskets[bskt].status = BSKTFILLED;
+ BBPunfix(rid->batCacheid);
+ BBPunfix(bval->batCacheid);
+ return MAL_SUCCEED;
+}
+
+str
BSKTreset(Client cntxt, MalBlkPtr mb, MalStkPtr stk, InstrPtr pci)
{
lng *res = getArgReference_lng(stk, pci, 0);
@@ -894,7 +932,7 @@ BSKTtableerrors(bat *nameId, bat *errorI
}
for (i = 1; i < bsktTop; i++)
- if (BATcount(baskets[i].errors) > 0) {
+ if (baskets[i].errors && BATcount(baskets[i].errors) > 0) {
bi = bat_iterator(baskets[i].errors);
BATloop(baskets[i].errors, p, q)
{
diff --git a/sql/backends/monet5/iot/basket.h b/sql/backends/monet5/iot/basket.h
--- a/sql/backends/monet5/iot/basket.h
+++ b/sql/backends/monet5/iot/basket.h
@@ -89,6 +89,7 @@ iot_export int BSKTlocate(str sch, str t
iot_export int BSKTunlocate(str sch, str tbl);
iot_export int BSKTlocate(str sch, str tbl);
iot_export str BSKTappend(Client cntxt, MalBlkPtr mb, MalStkPtr stk, InstrPtr
pci);
+iot_export str BSKTupdate(Client cntxt, MalBlkPtr mb, MalStkPtr stk, InstrPtr
pci);
iot_export str BSKTimportInternal(Client cntxt, int
MonetDB: iot - Add a defense line.
Changeset: e2e0f3f542a1 for MonetDB
URL: http://dev.monetdb.org/hg/MonetDB?cmd=changeset;node=e2e0f3f542a1
Modified Files:
sql/backends/monet5/sql_scenario.c
Branch: iot
Log Message:
Add a defense line.
diffs (12 lines):
diff --git a/sql/backends/monet5/sql_scenario.c
b/sql/backends/monet5/sql_scenario.c
--- a/sql/backends/monet5/sql_scenario.c
+++ b/sql/backends/monet5/sql_scenario.c
@@ -447,7 +447,7 @@ SQLinitClient(Client c)
if (c->sqlcontext == 0) {
m = mvc_create(c->idx, 0, SQLdebug, c->fdin, c->fdout);
global_variables(m, "monetdb", "sys");
- if (isAdministrator(c) || strcmp(c->scenario, "msql") == 0)
/* console should return everything */
+ if (isAdministrator(c) || c->scenario == 0 ||
strcmp(c->scenario, "msql") == 0) /* console should return everything */
m->reply_size = -1;
be = (void *) backend_create(m, c);
} else {
___
checkin-list mailing list
checkin-list@monetdb.org
https://www.monetdb.org/mailman/listinfo/checkin-list
MonetDB: iot - Additional defense line
Changeset: 47307c2815ab for MonetDB
URL: http://dev.monetdb.org/hg/MonetDB?cmd=changeset;node=47307c2815ab
Modified Files:
monetdb5/mal/mal_interpreter.c
Branch: iot
Log Message:
Additional defense line
diffs (12 lines):
diff --git a/monetdb5/mal/mal_interpreter.c b/monetdb5/mal/mal_interpreter.c
--- a/monetdb5/mal/mal_interpreter.c
+++ b/monetdb5/mal/mal_interpreter.c
@@ -541,7 +541,7 @@ str runMALsequence(Client cntxt, MalBlkP
//Ensure we spread system resources over multiple users as well.
runtimeProfileBegin(cntxt, mb, stk, pci, );
if (runtimeProfile.ticks > lastcheck + CHECKINTERVAL) {
- if (cntxt->fdin && !mnstr_isalive(cntxt->fdin->s)) {
+ if (cntxt->fdin && cntxt->fdin->s &&
!mnstr_isalive(cntxt->fdin->s)) {
cntxt->mode = FINISHCLIENT;
stkpc = stoppc;
ret= createException(MAL, "mal.interpreter",
"prematurely stopped client");
___
checkin-list mailing list
checkin-list@monetdb.org
https://www.monetdb.org/mailman/listinfo/checkin-list
MonetDB: iot - Avoid freeing buffers
Changeset: 6b20fa4371a4 for MonetDB
URL: http://dev.monetdb.org/hg/MonetDB?cmd=changeset;node=6b20fa4371a4
Modified Files:
monetdb5/optimizer/opt_iot.c
monetdb5/optimizer/opt_pipes.c
Branch: iot
Log Message:
Avoid freeing buffers
and unnecessary tumblings
diffs (28 lines):
diff --git a/monetdb5/optimizer/opt_iot.c b/monetdb5/optimizer/opt_iot.c
--- a/monetdb5/optimizer/opt_iot.c
+++ b/monetdb5/optimizer/opt_iot.c
@@ -182,7 +182,7 @@ OPTiotImplementation(Client cntxt, MalBl
if (p->token == ENDsymbol && btop > 0 && noerror==0) {
// empty all baskets used only when we are
optimizing a cq
for(j = 0; j < btop; j++)
- if( input[j] ){
+ if( input[j] && !output[j] ){
r = newStmt(mb, basketRef, tumbleRef);
r = pushArgument(mb,r, lastmvc);
r = pushStr(mb,r, schemas[j]);
diff --git a/monetdb5/optimizer/opt_pipes.c b/monetdb5/optimizer/opt_pipes.c
--- a/monetdb5/optimizer/opt_pipes.c
+++ b/monetdb5/optimizer/opt_pipes.c
@@ -483,9 +483,9 @@ compileOptimizer(Client cntxt, str name)
/* we must clear c.mythread because we're reusing a
Thread
* and must not delete that one */
c.mythread = 0;
- /* destroy bstream using free */
- free(c.fdin->buf);
- free(c.fdin);
+ /* WHY? destroy bstream using free */
+ //free(c.fdin->buf);
+ //free(c.fdin);
/* remove garbage from previous connection */
if (c.nspace) {
freeModule(c.nspace);
___
checkin-list mailing list
checkin-list@monetdb.org
https://www.monetdb.org/mailman/listinfo/checkin-list
MonetDB: Jun2016 - Memory leak.
Changeset: a15f7ffe73a3 for MonetDB URL: http://dev.monetdb.org/hg/MonetDB?cmd=changeset;node=a15f7ffe73a3 Modified Files: monetdb5/mal/mal_instruction.c Branch: Jun2016 Log Message: Memory leak. diffs (21 lines): diff --git a/monetdb5/mal/mal_instruction.c b/monetdb5/mal/mal_instruction.c --- a/monetdb5/mal/mal_instruction.c +++ b/monetdb5/mal/mal_instruction.c @@ -211,6 +211,9 @@ freeMalBlk(MalBlkPtr mb) if (mb->help) GDKfree(mb->help); mb->help = 0; + if (mb->marker) + GDKfree(mb->marker); + mb->marker = 0; mb->inlineProp = 0; mb->unsafeProp = 0; GDKfree(mb); @@ -298,6 +301,7 @@ addtoMalBlkHistory(MalBlkPtr mb, str mar if (cpy == NULL) return; /* ignore history */ cpy->history = NULL; + GDKfree(mb->marker); mb->marker = GDKstrdup(marker); if (mb->history == NULL) mb->history = cpy; ___ checkin-list mailing list checkin-list@monetdb.org https://www.monetdb.org/mailman/listinfo/checkin-list
MonetDB: iot - Merged with backend
Changeset: ac718d7895af for MonetDB URL: http://dev.monetdb.org/hg/MonetDB?cmd=changeset;node=ac718d7895af Added Files: sql/backends/monet5/Tests/pyapi30.sql sql/backends/monet5/Tests/pyapi30.stable.err sql/backends/monet5/Tests/pyapi30.stable.out Removed Files: clients/python2/ChangeLog clients/python2/ChangeLog-Archive clients/python2/ChangeLog.Jun2016 clients/python2/MANIFEST.in clients/python2/Makefile.ag clients/python2/README.rst clients/python2/monetdb/__init__.py clients/python2/monetdb/control.py clients/python2/monetdb/exceptions.py clients/python2/monetdb/mapi.py clients/python2/monetdb/sql/__init__.py clients/python2/monetdb/sql/connections.py clients/python2/monetdb/sql/converters.py clients/python2/monetdb/sql/cursors.py clients/python2/monetdb/sql/monetize.py clients/python2/monetdb/sql/pythonize.py clients/python2/monetdb/sql/types.py clients/python2/setup.py clients/python2/test/capabilities.py clients/python2/test/dbapi20.py clients/python2/test/run.cmd clients/python2/test/run.sh clients/python2/test/runtests.py clients/python2/test/test_control.py clients/python2/test/test_monetize.py clients/python2/test/test_pythonize.py clients/python3/ChangeLog clients/python3/ChangeLog-Archive clients/python3/ChangeLog.Jun2016 clients/python3/MANIFEST.in clients/python3/Makefile.ag clients/python3/README.rst clients/python3/monetdb/__init__.py clients/python3/monetdb/control.py clients/python3/monetdb/exceptions.py clients/python3/monetdb/mapi.py clients/python3/monetdb/sql/__init__.py clients/python3/monetdb/sql/connections.py clients/python3/monetdb/sql/converters.py clients/python3/monetdb/sql/cursors.py clients/python3/monetdb/sql/monetize.py clients/python3/monetdb/sql/pythonize.py clients/python3/monetdb/sql/types.py clients/python3/setup.py clients/python3/test/capabilities.py clients/python3/test/dbapi20.py clients/python3/test/run.cmd clients/python3/test/run.sh clients/python3/test/runtests.py clients/python3/test/test_control.py clients/python3/test/test_monetize.py clients/python3/test/test_pythonize.py monetdb5/tests/BugReports/Tests/no.146.stable.out sql/test/mapi/Tests/python2_test_monetdb_sql.SQL.bat sql/test/mapi/Tests/python2_test_monetdb_sql.SQL.sh sql/test/mapi/Tests/python2_test_monetdb_sql.stable.err sql/test/mapi/Tests/python2_test_monetdb_sql.stable.out sql/test/mapi/Tests/python3_test_monetdb_sql.SQL.bat sql/test/mapi/Tests/python3_test_monetdb_sql.SQL.sh sql/test/mapi/Tests/python3_test_monetdb_sql.stable.err sql/test/mapi/Tests/python3_test_monetdb_sql.stable.out Modified Files: .hgtags MonetDB.spec NT/Makefile NT/installer32/MonetDB-ODBC-Installer.vdproj NT/installer32/MonetDB5-Geom-Module.vdproj NT/installer32/MonetDB5-SQL-Installer.vdproj NT/installer64/MonetDB-ODBC-Installer.vdproj NT/installer64/MonetDB5-Geom-Module.vdproj NT/installer64/MonetDB5-SQL-Installer.vdproj NT/monetdb_config.h.in NT/rules.msc buildtools/autogen/autogen/am.py buildtools/autogen/autogen/msc.py clients/Makefile.ag clients/Tests/SQL-dump.stable.out clients/Tests/SQL-dump.stable.out.int128 clients/examples/python/basics.py clients/examples/python/mclient-python2.py clients/examples/python/mclient-python3.py clients/examples/python/perf.py clients/examples/python/sqlsample.py clients/mapiclient/msqldump.c clients/mapilib/mapi.rc clients/odbc/driver/driver.rc clients/odbc/winsetup/setup.rc common/stream/stream.c configure.ag debian/changelog debian/control debian/fix-deb.sh gdk/libbat.rc geom/lib/libgeom.c geom/monetdb5/geom.c geom/sql/Makefile.ag java/Makefile.ag java/build.properties java/pom.xml java/release.txt java/src/main/java/nl/cwi/monetdb/jdbc/MonetDatabaseMetaData.java java/src/main/java/nl/cwi/monetdb/jdbc/MonetPreparedStatement.java java/src/main/java/nl/cwi/monetdb/jdbc/MonetResultSet.java libversions monetdb5/ChangeLog-Archive monetdb5/extras/pyapi/pyapi.c monetdb5/mal/mal_dataflow.c monetdb5/mal/mal_instruction.c monetdb5/mal/mal_parser.c monetdb5/modules/atoms/url.c monetdb5/modules/mal/groupby.c monetdb5/modules/mal/sysmon.c monetdb5/modules/mal/tablet.c
MonetDB: iot - Added first Front-End test
Changeset: dce31e77 for MonetDB
URL: http://dev.monetdb.org/hg/MonetDB?cmd=changeset;node=dce31e77
Added Files:
clients/iotapi/tests/__init__.py
clients/iotapi/tests/frontendtests.py
clients/iotapi/tests/main.py
Modified Files:
clients/iotapi/requirements.txt
clients/iotclient/src/Streams/jsonschemas.py
clients/iotclient/src/Streams/streams.py
clients/iotclient/tests/main.py
Branch: iot
Log Message:
Added first Front-End test
diffs (truncated from 364 to 300 lines):
diff --git a/clients/iotapi/requirements.txt b/clients/iotapi/requirements.txt
--- a/clients/iotapi/requirements.txt
+++ b/clients/iotapi/requirements.txt
@@ -4,7 +4,9 @@ jsonschema==2.5.1
python-dateutil==2.5.3
python-monetdb==11.24.0
pytz==2016.4
+requests==2.10.0
Sphinx==1.4.4
sphinx-rtd-theme==0.1.9
tzlocal==1.2.2
watchdog==0.8.3
+websocket-client==0.37.0
diff --git a/clients/iotapi/tests/__init__.py b/clients/iotapi/tests/__init__.py
new file mode 100644
diff --git a/clients/iotapi/tests/frontendtests.py
b/clients/iotapi/tests/frontendtests.py
new file mode 100644
--- /dev/null
+++ b/clients/iotapi/tests/frontendtests.py
@@ -0,0 +1,148 @@
+import json
+import os
+import requests
+import websocket
+
+from distutils.dir_util import copy_tree
+from threading import Thread
+from time import sleep
+from unittest import TestCase
+
+__all__ = ['NullablesTest']
+
+WEB_SOCKETS_THREADS_TIMEOUT = 15
+
+
+class BaseFrontEndTest(TestCase):
+
+def __init__(self, **kwargs):
+super(BaseFrontEndTest, self).__init__()
+self._web_server_baskets_location =
os.path.join(kwargs['iot_client_path'], 'baskets')
+self._web_api_baskets_location = os.path.join(kwargs['iot_api_path'],
'baskets')
+self.schema = "tests"
+
+def export_inserts(self, schema, stream, basket):
+input_dir = os.path.join(self._web_server_baskets_location, schema,
stream, basket)
+output_dir = os.path.join(self._web_api_baskets_location, schema,
stream, basket)
+copy_tree(input_dir, output_dir)
+
+
+class TestWebSocket(websocket.WebSocketApp):
+
+def __init__(self, test, url, header=[], on_open=None, on_message=None,
on_error=None, on_close=None, on_ping=None,
+ on_pong=None, on_cont_message=None, keep_running=True,
get_mask_key=None, cookie=None,
+ subprotocols=None, on_data=None):
+super(TestWebSocket, self).__init__(url, header, on_open, on_message,
on_error, on_close, on_ping,
+on_pong, on_cont_message,
keep_running, get_mask_key, cookie,
+subprotocols, on_data)
+self.test = test
+self.ws_state = 1
+
+
+def on_open(ws):
+ws.send(json.dumps({"request": "sub", "schema": ws.test.schema, "stream":
ws.test.stream}))
+
+
+def on_message(ws, message):
+resp = json.loads(message)
+if resp['response'] == 'error':
+ws.test.fail(msg=resp['message'])
+elif ws.ws_state == 1:
+ws.test.assertDictEqual({"response": "subscribed", "schema":
ws.test.schema, "stream": ws.test.stream}, resp)
+ws.ws_state = 2
+elif ws.ws_state == 2:
+ws.test.assertDictEqual({"response": "notification", "schema":
ws.test.schema, "stream": ws.test.stream,
+ "basket": 1, "count": 3}, resp)
+ws.send(json.dumps({"request": "read", "schema": ws.test.schema,
"stream": ws.test.stream,
+"basket": 1, "offset": 0, "limit": 3}))
+ws.ws_state = 3
+elif ws.ws_state == 3:
+ws.test.assertIn('implicit_timestamp', resp['tuples'][0],
msg='Timestamp not in stream')
+del resp['tuples'][0]['implicit_timestamp']
+del resp['tuples'][1]['implicit_timestamp']
+del resp['tuples'][2]['implicit_timestamp']
+res_dic = {'vala': None, "valb": None, "valc": None, "vald": None,
"vale": None, "valf": None, "valg": None,
+ "valh": None, "vali": None, "valj": None, "valk": None,
"vall": None, "valm": None, "valn": None,
+ "valo": None, "valp": None, "valq": None, "valr": None,
"vals": None, "valt": None, "valu": None,
+ "valv": None, "valw": None, "valx": None, "valy": None,
"valz": None}
+tuples_response = {"response": "read", "schema": ws.test.schema,
"stream": ws.test.stream,
+ "count": 3, "tuples": [res_dic, res_dic, res_dic]}
+ws.test.assertDictEqual(tuples_response, resp)
+ws.close()
+elif ws.ws_state == 4:
+ws.test.assertDictEqual({"response": "removed", "schema":
ws.test.schema, "stream": ws.test.stream}, resp)
+ws.close()
+
+
+def on_error(ws, error):
+ws.close()
+ws.test.fail(msg=error)
+
+
+def web_socket(test):
+ws = TestWebSocket(test=test, url="ws://127.0.0.1:8002/",
on_message=on_message, on_open=on_open, on_error=on_error)
+test.ws = ws
+
MonetDB: Jun2016 - Memory leak.
Changeset: d0f4202627b8 for MonetDB
URL: http://dev.monetdb.org/hg/MonetDB?cmd=changeset;node=d0f4202627b8
Modified Files:
monetdb5/mal/mal_interpreter.c
Branch: Jun2016
Log Message:
Memory leak.
diffs (17 lines):
diff --git a/monetdb5/mal/mal_interpreter.c b/monetdb5/mal/mal_interpreter.c
--- a/monetdb5/mal/mal_interpreter.c
+++ b/monetdb5/mal/mal_interpreter.c
@@ -725,11 +725,13 @@ str runMALsequence(Client cntxt, MalBlkP
nstk->up = stk;
if (nstk->calldepth > 256) {
ret= createException(MAL,
"mal.interpreter", MAL_CALLDEPTH_FAIL);
+ GDKfree(nstk);
break;
}
if ((unsigned)nstk->stkdepth >
THREAD_STACK_SIZE / sizeof(mb->var[0]) / 4 && THRhighwater()){
/* we are running low on stack
space */
ret= createException(MAL,
"mal.interpreter", MAL_STACK_FAIL);
+ GDKfree(nstk);
break;
}
___
checkin-list mailing list
checkin-list@monetdb.org
https://www.monetdb.org/mailman/listinfo/checkin-list
MonetDB: Jun2016 - Memory leak.
Changeset: b36537c033c2 for MonetDB
URL: http://dev.monetdb.org/hg/MonetDB?cmd=changeset;node=b36537c033c2
Modified Files:
monetdb5/mal/mal_factory.c
monetdb5/mal/mal_interpreter.c
Branch: Jun2016
Log Message:
Memory leak.
diffs (49 lines):
diff --git a/monetdb5/mal/mal_factory.c b/monetdb5/mal/mal_factory.c
--- a/monetdb5/mal/mal_factory.c
+++ b/monetdb5/mal/mal_factory.c
@@ -382,13 +382,12 @@ void mal_factory_reset(void)
/* MSresetVariables(mb, pl->stk, 0);*/
/* freeStack(pl->stk); there may be a reference?*/
/* we are inside the body of the factory and about to
return */
- pl->factory = 0;
- if (pl->stk)
+ if (pl->stk) {
pl->stk->keepAlive = FALSE;
- if ( pl->stk) {
- //garbageCollector(cntxt, mb, pl->stk,TRUE);
+ garbageCollector(NULL, pl->factory,
pl->stk,TRUE);
GDKfree(pl->stk);
}
+ pl->factory = 0;
pl->stk=0;
pl->pc = 0;
pl->inuse = 0;
diff --git a/monetdb5/mal/mal_interpreter.c b/monetdb5/mal/mal_interpreter.c
--- a/monetdb5/mal/mal_interpreter.c
+++ b/monetdb5/mal/mal_interpreter.c
@@ -1431,8 +1431,10 @@ void garbageCollector(Client cntxt, MalB
ValPtr v;
#ifdef STACKTRACE
- mnstr_printf(cntxt->fdout, "#--->stack before garbage collector\n");
- printStack(cntxt->fdout, mb, stk, 0);
+ if (cntxt) {
+ mnstr_printf(cntxt->fdout, "#--->stack before garbage
collector\n");
+ printStack(cntxt->fdout, mb, stk, 0);
+ }
#endif
for (k = 0; k < mb->vtop; k++) {
if (isVarCleanup(mb, k) && (flag || isTmpVar(mb, k))) {
@@ -1442,8 +1444,10 @@ void garbageCollector(Client cntxt, MalB
}
}
#ifdef STACKTRACE
- mnstr_printf(cntxt->fdout, "#-->stack after garbage collector\n");
- printStack(cntxt->fdout, mb, stk, 0);
+ if (cntxt) {
+ mnstr_printf(cntxt->fdout, "#-->stack after garbage
collector\n");
+ printStack(cntxt->fdout, mb, stk, 0);
+ }
#else
(void)cntxt;
#endif
___
checkin-list mailing list
checkin-list@monetdb.org
https://www.monetdb.org/mailman/listinfo/checkin-list
MonetDB: default - Merge with Jun2016 branch.
Changeset: e7ebd125b103 for MonetDB
URL: http://dev.monetdb.org/hg/MonetDB?cmd=changeset;node=e7ebd125b103
Modified Files:
clients/mapiclient/mclient.c
monetdb5/mal/mal_profiler.c
monetdb5/mal/mal_recycle.c
monetdb5/modules/mal/pcre.c
monetdb5/optimizer/opt_mergetable.c
monetdb5/optimizer/opt_pipes.c
monetdb5/optimizer/optimizer.c
monetdb5/optimizer/optimizer_private.h
sql/server/rel_semantic.c
sql/storage/bat/bat_storage.c
Branch: default
Log Message:
Merge with Jun2016 branch.
diffs (truncated from 623 to 300 lines):
diff --git a/clients/mapiclient/mclient.c b/clients/mapiclient/mclient.c
--- a/clients/mapiclient/mclient.c
+++ b/clients/mapiclient/mclient.c
@@ -3322,6 +3322,7 @@ main(int argc, char **argv)
if (command != NULL) {
#ifdef HAVE_ICONV
iconv_t cd_in;
+ int free_command = 0;
if (encoding != NULL &&
(cd_in = iconv_open("utf-8", encoding)) != (iconv_t) -1) {
@@ -3331,6 +3332,7 @@ main(int argc, char **argv)
int factor = 4;
size_t tolen = factor * fromlen + 1;
char *to = malloc(tolen);
+ free_command = 1;
try_again:
command = to;
@@ -3367,6 +3369,10 @@ main(int argc, char **argv)
timerStart();
c = doRequest(mid, command);
timerEnd();
+#ifdef HAVE_ICONV
+ if (free_command)
+ free(command);
+#endif
}
if (optind < argc) {
diff --git a/monetdb5/mal/mal_profiler.c b/monetdb5/mal/mal_profiler.c
--- a/monetdb5/mal/mal_profiler.c
+++ b/monetdb5/mal/mal_profiler.c
@@ -626,9 +626,11 @@ static BAT *TRACE_id_stmt = 0;
int
TRACEtable(BAT **r)
{
- if (TRACE_init == 0)
+ MT_lock_set(_profileLock);
+ if (TRACE_init == 0) {
+ MT_lock_unset(_profileLock);
return -1; /* not initialized */
- MT_lock_set(_profileLock);
+ }
r[0] = COLcopy(TRACE_id_event, TRACE_id_event->ttype, 0, TRANSIENT);
r[1] = COLcopy(TRACE_id_time, TRACE_id_time->ttype, 0, TRANSIENT);
r[2] = COLcopy(TRACE_id_pc, TRACE_id_pc->ttype, 0, TRANSIENT);
@@ -649,35 +651,39 @@ TRACEtable(BAT **r)
BAT *
getTrace(const char *nme)
{
- if (TRACE_init == 0)
- return NULL;
- if (strcmp(nme, "event") == 0)
- return COLcopy(TRACE_id_event, TRACE_id_event->ttype, 0,
TRANSIENT);
- if (strcmp(nme, "time") == 0)
- return COLcopy(TRACE_id_time, TRACE_id_time->ttype, 0,
TRANSIENT);
- if (strcmp(nme, "pc") == 0)
- return COLcopy(TRACE_id_pc, TRACE_id_pc->ttype, 0, TRANSIENT);
- if (strcmp(nme, "thread") == 0)
- return COLcopy(TRACE_id_thread, TRACE_id_thread->ttype, 0,
TRANSIENT);
- if (strcmp(nme, "usec") == 0)
- return COLcopy(TRACE_id_ticks, TRACE_id_ticks->ttype, 0,
TRANSIENT);
- if (strcmp(nme, "rssMB") == 0)
- return COLcopy(TRACE_id_rssMB, TRACE_id_rssMB->ttype, 0,
TRANSIENT);
- if (strcmp(nme, "tmpspace") == 0)
- return COLcopy(TRACE_id_tmpspace, TRACE_id_tmpspace->ttype, 0,
TRANSIENT);
- if (strcmp(nme, "reads") == 0)
- return COLcopy(TRACE_id_inblock, TRACE_id_inblock->ttype, 0,
TRANSIENT);
- if (strcmp(nme, "writes") == 0)
- return COLcopy(TRACE_id_oublock, TRACE_id_oublock->ttype, 0,
TRANSIENT);
- if (strcmp(nme, "minflt") == 0)
- return COLcopy(TRACE_id_minflt, TRACE_id_minflt->ttype, 0,
TRANSIENT);
- if (strcmp(nme, "majflt") == 0)
- return COLcopy(TRACE_id_majflt, TRACE_id_majflt->ttype, 0,
TRANSIENT);
- if (strcmp(nme, "nvcsw") == 0)
- return COLcopy(TRACE_id_nvcsw, TRACE_id_nvcsw->ttype, 0,
TRANSIENT);
- if (strcmp(nme, "stmt") == 0)
- return COLcopy(TRACE_id_stmt, TRACE_id_stmt->ttype, 0,
TRANSIENT);
- return NULL;
+ BAT *bn = NULL;
+
+ MT_lock_set(_profileLock);
+ if (TRACE_init) {
+ if (strcmp(nme, "event") == 0)
+ bn = COLcopy(TRACE_id_event, TRACE_id_event->ttype, 0,
TRANSIENT);
+ else if (strcmp(nme, "time") == 0)
+ bn = COLcopy(TRACE_id_time, TRACE_id_time->ttype, 0,
TRANSIENT);
+ else if (strcmp(nme, "pc") == 0)
+ bn = COLcopy(TRACE_id_pc, TRACE_id_pc->ttype, 0,
TRANSIENT);
+ else if (strcmp(nme, "thread") == 0)
+ bn = COLcopy(TRACE_id_thread, TRACE_id_thread->ttype,
0, TRANSIENT);
+ else if (strcmp(nme, "usec") == 0)
+ bn = COLcopy(TRACE_id_ticks, TRACE_id_ticks->ttype, 0,
TRANSIENT);
+ else if (strcmp(nme, "rssMB") == 0)
+ bn =
MonetDB: Jun2016 - Memory leaks.
Changeset: ff63f6342e2f for MonetDB
URL: http://dev.monetdb.org/hg/MonetDB?cmd=changeset;node=ff63f6342e2f
Modified Files:
clients/mapiclient/mclient.c
monetdb5/modules/mal/pcre.c
Branch: Jun2016
Log Message:
Memory leaks.
diffs (110 lines):
diff --git a/clients/mapiclient/mclient.c b/clients/mapiclient/mclient.c
--- a/clients/mapiclient/mclient.c
+++ b/clients/mapiclient/mclient.c
@@ -3322,6 +3322,7 @@ main(int argc, char **argv)
if (command != NULL) {
#ifdef HAVE_ICONV
iconv_t cd_in;
+ int free_command = 0;
if (encoding != NULL &&
(cd_in = iconv_open("utf-8", encoding)) != (iconv_t) -1) {
@@ -3331,6 +3332,7 @@ main(int argc, char **argv)
int factor = 4;
size_t tolen = factor * fromlen + 1;
char *to = malloc(tolen);
+ free_command = 1;
try_again:
command = to;
@@ -3367,6 +3369,10 @@ main(int argc, char **argv)
timerStart();
c = doRequest(mid, command);
timerEnd();
+#ifdef HAVE_ICONV
+ if (free_command)
+ free(command);
+#endif
}
if (optind < argc) {
diff --git a/monetdb5/modules/mal/pcre.c b/monetdb5/modules/mal/pcre.c
--- a/monetdb5/modules/mal/pcre.c
+++ b/monetdb5/modules/mal/pcre.c
@@ -318,14 +318,14 @@ pcre_likesubselect(BAT **bnp, BAT *b, BA
pe = pcre_study(re, 0, );
if (error != NULL) {
my_pcre_free(re);
- my_pcre_free(pe);
+ pcre_free_study(pe);
throw(MAL, "pcre.likesubselect",
OPERATION_FAILED ": studying pattern \"%s\"
failed\n", pat);
}
bn = BATnew(TYPE_void, TYPE_oid, s ? BATcount(s) : BATcount(b),
TRANSIENT);
if (bn == NULL) {
my_pcre_free(re);
- my_pcre_free(pe);
+ pcre_free_study(pe);
throw(MAL, "pcre.likesubselect", MAL_MALLOC_FAIL);
}
off = b->hseqbase - BUNfirst(b);
@@ -373,7 +373,7 @@ pcre_likesubselect(BAT **bnp, BAT *b, BA
pcre_exec(re, pe, v, (int) strlen(v), 0, 0,
ovector, 10) >= 0);
}
my_pcre_free(re);
- my_pcre_free(pe);
+ pcre_free_study(pe);
bn->tsorted = 1;
bn->trevsorted = bn->batCount <= 1;
bn->tkey = 1;
@@ -391,7 +391,7 @@ pcre_likesubselect(BAT **bnp, BAT *b, BA
bunins_failed:
BBPreclaim(bn);
my_pcre_free(re);
- my_pcre_free(pe);
+ pcre_free_study(pe);
*bnp = NULL;
throw(MAL, "pcre.likesubselect", OPERATION_FAILED);
}
@@ -567,6 +567,7 @@ pcre_replace(str *res, const char *origi
offset = ovector[1];
}
} while((j > 0) && (offset < len_origin_str) && (ncaptures <
MAX_NR_CAPTURES));
+ pcre_free_study(extra);
if (ncaptures > 0){
tmpres = GDKmalloc(len_origin_str - len_del + (len_replacement
* ncaptures) + 1);
@@ -688,6 +689,7 @@ pcre_replace_bat(BAT **res, BAT *origin_
replaced_str = GDKmalloc(len_origin_str - len_del +
(len_replacement * ncaptures) + 1);
if (!replaced_str) {
my_pcre_free(pcre_code);
+ pcre_free_study(extra);
GDKfree(ovector);
throw(MAL, "pcre_replace_bat", MAL_MALLOC_FAIL);
}
@@ -725,6 +727,7 @@ pcre_replace_bat(BAT **res, BAT *origin_
}
}
+ pcre_free_study(extra);
my_pcre_free(pcre_code);
GDKfree(ovector);
BATseqbase(tmpbat, origin_strs->hseqbase);
@@ -1576,7 +1579,7 @@ pcresubjoin(BAT *r1, BAT *r2, BAT *l, BA
}
if (pcrere) {
my_pcre_free(pcrere);
- my_pcre_free(pcreex);
+ pcre_free_study(pcreex);
pcrere = NULL;
pcreex = NULL;
}
@@ -1618,7 +1621,7 @@ pcresubjoin(BAT *r1, BAT *r2, BAT *l, BA
if (pcrere)
my_pcre_free(pcrere);
if (pcreex)
- my_pcre_free(pcreex);
+ pcre_free_study(pcreex);
assert(msg != MAL_SUCCEED);
return msg;
}
___
checkin-list mailing list
checkin-list@monetdb.org
https://www.monetdb.org/mailman/listinfo/checkin-list
MonetDB: Jun2016 - Don't initialize optimizer pipes concurrently...
Changeset: dc6677e2dbcc for MonetDB
URL: http://dev.monetdb.org/hg/MonetDB?cmd=changeset;node=dc6677e2dbcc
Modified Files:
monetdb5/optimizer/opt_pipes.c
monetdb5/optimizer/optimizer.c
monetdb5/optimizer/optimizer_private.h
Branch: Jun2016
Log Message:
Don't initialize optimizer pipes concurrently: that causes memory leaks.
diffs (140 lines):
diff --git a/monetdb5/optimizer/opt_pipes.c b/monetdb5/optimizer/opt_pipes.c
--- a/monetdb5/optimizer/opt_pipes.c
+++ b/monetdb5/optimizer/opt_pipes.c
@@ -238,6 +238,17 @@ static struct PIPELINES {
* as it is intended for internal use.",
*/
#include "opt_pipes.h"
+#include "optimizer_private.h"
+
+static MT_Lock pipeLock MT_LOCK_INITIALIZER("pipeLock");
+
+void
+optPipeInit(void)
+{
+#ifdef NEED_MT_LOCK_INIT
+ MT_lock_init(, "pipeLock");
+#endif
+}
/* the session_pipe is the one defined by the user */
str
@@ -247,14 +258,19 @@ addPipeDefinition(Client cntxt, str name
str msg;
struct PIPELINES oldpipe;
+ MT_lock_set();
for (i = 0; i < MAXOPTPIPES && pipes[i].name; i++)
if (strcmp(name, pipes[i].name) == 0)
break;
- if (i == MAXOPTPIPES)
+ if (i == MAXOPTPIPES) {
+ MT_lock_unset();
throw(MAL, "optimizer.addPipeDefinition", "Out of slots");
- if (pipes[i].name && pipes[i].builtin)
+ }
+ if (pipes[i].name && pipes[i].builtin) {
+ MT_lock_unset();
throw(MAL, "optimizer.addPipeDefinition", "No overwrite of
built in allowed");
+ }
/* save old value */
oldpipe = pipes[i];
@@ -262,15 +278,18 @@ addPipeDefinition(Client cntxt, str name
pipes[i].def = GDKstrdup(pipe);
pipes[i].status = GDKstrdup("experimental");
pipes[i].mb = NULL;
+ MT_lock_unset();
msg = compileOptimizer(cntxt, name);
if (msg) {
/* failed: restore old value */
+ MT_lock_set();
GDKfree(pipes[i].name);
GDKfree(pipes[i].def);
if (pipes[i].mb)
freeMalBlk(pipes[i].mb);
GDKfree(pipes[i].status);
pipes[i] = oldpipe;
+ MT_lock_unset();
} else {
/* succeeded: destroy old value */
if (oldpipe.name)
@@ -281,11 +300,6 @@ addPipeDefinition(Client cntxt, str name
freeMalBlk(oldpipe.mb);
if (oldpipe.status)
GDKfree(oldpipe.status);
- if (++i < MAXOPTPIPES) {
- pipes[i].name = pipes[i].def = pipes[i].status =
pipes[i].prerequisite = NULL;
- pipes[i].mb = NULL;
- pipes[i].builtin = 0;
- }
}
return msg;
}
@@ -430,6 +444,7 @@ compileOptimizer(Client cntxt, str name)
ClientRec c;
memset((char*), 0, sizeof(c));
+ MT_lock_set();
for (i = 0; i < MAXOPTPIPES && pipes[i].name; i++) {
if (strcmp(pipes[i].name, name) == 0 && pipes[i].mb == 0) {
/* precompile the pipeline as MAL string */
@@ -437,8 +452,10 @@ compileOptimizer(Client cntxt, str name)
c.nspace = newModule(NULL, putName("user"));
c.father = cntxt; /* to avoid conflicts on GDKin
*/
c.fdout = cntxt->fdout;
- if (setScenario(, "mal"))
+ if (setScenario(, "mal")) {
+ MT_lock_unset();
throw(MAL, "optimizer.addOptimizerPipe",
"failed to set scenario");
+ }
(void) MCinitClientThread();
for (j = 0; j < MAXOPTPIPES && pipes[j].def; j++) {
if (pipes[j].mb == NULL) {
@@ -466,12 +483,13 @@ compileOptimizer(Client cntxt, str name)
c.nspace = 0;
}
MCcloseClient();
- if (msg != MAL_SUCCEED ||
- (msg = validateOptimizerPipes()) != MAL_SUCCEED)
- return msg;
+ if (msg != MAL_SUCCEED ||
+ (msg = validateOptimizerPipes()) != MAL_SUCCEED)
+ break;
}
}
- return MAL_SUCCEED;
+ MT_lock_unset();
+ return msg;
}
str
@@ -486,7 +504,7 @@ addOptimizerPipe(Client cntxt, MalBlkPtr
break;
if (i == MAXOPTPIPES)
- throw(MAL, "optimizer.addOptimizerPipe", "Out of slots");;
+ throw(MAL, "optimizer.addOptimizerPipe", "Out of slots");
if (pipes[i].mb == NULL)
msg = compileOptimizer(cntxt, name);
diff --git a/monetdb5/optimizer/optimizer.c
MonetDB: Jun2016 - Merge with Jul2015 branch.
Changeset: b50521adb424 for MonetDB URL: http://dev.monetdb.org/hg/MonetDB?cmd=changeset;node=b50521adb424 Modified Files: sql/storage/bat/bat_storage.c Branch: Jun2016 Log Message: Merge with Jul2015 branch. ___ checkin-list mailing list checkin-list@monetdb.org https://www.monetdb.org/mailman/listinfo/checkin-list
MonetDB: Jul2015 - Use correct type.
Changeset: 298262f55163 for MonetDB URL: http://dev.monetdb.org/hg/MonetDB?cmd=changeset;node=298262f55163 Modified Files: sql/storage/bat/bat_storage.c Branch: Jul2015 Log Message: Use correct type. (grafted from 42207239a7ed6a086f802a7b375abeb2e49de4d5) diffs (12 lines): diff --git a/sql/storage/bat/bat_storage.c b/sql/storage/bat/bat_storage.c --- a/sql/storage/bat/bat_storage.c +++ b/sql/storage/bat/bat_storage.c @@ -2087,7 +2087,7 @@ tr_update_dbat_swap(sql_dbat *tdb, sql_d * Swap both tdb->dbid/fdb->dbid */ int ok = LOG_OK; BAT *db = NULL; - size_t cnt; + BUN cnt; if (!fdb) return ok; ___ checkin-list mailing list checkin-list@monetdb.org https://www.monetdb.org/mailman/listinfo/checkin-list
MonetDB: iot - Merge with default
Changeset: b921c9a1bb4a for MonetDB
URL: http://dev.monetdb.org/hg/MonetDB?cmd=changeset;node=b921c9a1bb4a
Added Files:
sql/backends/monet5/Tests/pyapi30.sql
sql/backends/monet5/Tests/pyapi30.stable.err
sql/backends/monet5/Tests/pyapi30.stable.out
Modified Files:
monetdb5/extras/pyapi/pyapi.c
Branch: iot
Log Message:
Merge with default
diffs (197 lines):
diff --git a/monetdb5/extras/pyapi/pyapi.c b/monetdb5/extras/pyapi/pyapi.c
--- a/monetdb5/extras/pyapi/pyapi.c
+++ b/monetdb5/extras/pyapi/pyapi.c
@@ -150,8 +150,10 @@ static bool enable_zerocopy_output = tru
bat->T->nil = 1;
\
}
\
}
\
+bat->T->nonil = 1 - bat->T->nil;
\
+} else {
\
+bat->T->nil = 0; bat->T->nonil = 0;
\
}
\
-bat->T->nonil = 1 - bat->T->nil;
\
/*When we create a BAT a small part of memory is allocated, free it*/
\
GDKfree(bat->T->heap.base);
\
bat->T->heap.base = [(index_offset * ret->count) *
ret->memory_size]; \
@@ -194,8 +196,10 @@ static bool enable_zerocopy_output = tru
bat->T->nil = 1;
\
}
\
}
\
+bat->T->nonil = 1 - bat->T->nil;
\
+} else {
\
+bat->T->nil = 0; bat->T->nonil = 0;
\
}
\
-bat->T->nonil = 1 - bat->T->nil;
\
/*When we create a BAT a small part of memory is allocated, free it*/
\
GDKfree(bat->T->heap.base);
\
bat->T->heap.base = [(index_offset * ret->count) *
ret->memory_size]; \
@@ -372,7 +376,8 @@ Array of type %s no copying will be need
goto wrapup;
\
}
\
bat->T->nonil = 1 - bat->T->nil;
\
-BATsetcount(bat, (BUN) ret->count);
\
+if (!mask) { bat->T->nil = 0; bat->T->nonil = 0; }
\
+BATsetcount(bat, (BUN) ret->count);
\
BATsettrivprop(bat);
\
}
\
}
diff --git a/sql/backends/monet5/Tests/pyapi30.sql
b/sql/backends/monet5/Tests/pyapi30.sql
new file mode 100644
--- /dev/null
+++ b/sql/backends/monet5/Tests/pyapi30.sql
@@ -0,0 +1,16 @@
+
+# test returning of numpy.nan values
+
MonetDB: iot - Reduce dependency on SQL context
Changeset: dabecc9c4bbc for MonetDB
URL: http://dev.monetdb.org/hg/MonetDB?cmd=changeset;node=dabecc9c4bbc
Modified Files:
sql/backends/monet5/iot/basket.c
sql/backends/monet5/iot/iot.c
Branch: iot
Log Message:
Reduce dependency on SQL context
All BAT's belonging to a stream basket are
'stolen' for direct manipulation by the
iot actions.
diffs (truncated from 476 to 300 lines):
diff --git a/sql/backends/monet5/iot/basket.c b/sql/backends/monet5/iot/basket.c
--- a/sql/backends/monet5/iot/basket.c
+++ b/sql/backends/monet5/iot/basket.c
@@ -108,10 +108,10 @@ BSKTlocate(str sch, str tbl)
// Instantiate a basket description for a particular table
static str
-BSKTnewbasket(sql_schema *s, sql_table *t)
+BSKTnewbasket(mvc *m, sql_schema *s, sql_table *t)
{
- int i, idx;
- int colcnt=0;
+ int i, idx, colcnt=0;
+ BAT *b;
node *o;
// Don't introduce the same basket twice
@@ -128,7 +128,9 @@ BSKTnewbasket(sql_schema *s, sql_table *
baskets[idx].count = 0;
baskets[idx].winstride = -1; /* all tuples are removed */
- for (o = t->columns.set->h; o; o = o->next){
+
+ // Check the column types first
+ for (o = t->columns.set->h; o && colcnt next){
sql_column *col = o->data;
int tpe = col->type.type->localtype;
@@ -138,24 +140,29 @@ BSKTnewbasket(sql_schema *s, sql_table *
}
colcnt++;
}
- // collect the column names
- baskets[idx].cols = (str*) GDKzalloc(sizeof(str) * (colcnt+1));
- for (i=0, o = t->columns.set->h; o; o = o->next){
+ if( colcnt == MAXCOLS){
+ BSKTclean(idx);
+ throw(MAL,"baskets.register","too many columns");
+ }
+
+ // collect the column names and the storage
+ i=0;
+ for ( i=0, o = t->columns.set->h; i next){
sql_column *col = o->data;
- baskets[idx].cols[i++]= col->base.name;
+ b = store_funcs.bind_col(m->session->tr,col,RD_INS);
+ assert(b);
+ BBPfix(b->batCacheid);
+ baskets[idx].bats[i]= b;
+ baskets[idx].cols[i++]= GDKstrdup(col->base.name);
}
- //
+ // Create the error table
baskets[idx].errors = BATnew(TYPE_void, TYPE_str, BATTINY, TRANSIENT);
- if (baskets[idx].table_name == NULL ||
- baskets[idx].errors == NULL) {
+ if (baskets[idx].table_name == NULL || baskets[idx].errors == NULL) {
BSKTclean(idx);
MT_lock_unset();
throw(MAL,"baskets.register",MAL_MALLOC_FAIL);
}
-
- baskets[idx].schema = s;
- baskets[idx].table = t;
MT_lock_unset();
return MAL_SUCCEED;
}
@@ -186,7 +193,7 @@ BSKTregisterInternal(Client cntxt, MalBl
if (t == NULL)
throw(SQL, "iot.register", "Table missing '%s'", tbl);
- msg= BSKTnewbasket(s, t);
+ msg= BSKTnewbasket(m, s, t);
return msg;
}
@@ -256,16 +263,15 @@ BSKTwindow(Client cntxt, MalBlkPtr mb, M
}
static BAT *
-BSKTbindColumn(Client cntxt, str sch, str tbl, str col)
+BSKTbindColumn(str sch, str tbl, str col)
{
- int bskt;
- int i;
+ int idx =0,i;
- if( (idx = BSKTlocate(sch,tbl) < 0))
+ if( (idx = BSKTlocate(sch,tbl)) < 0)
return NULL;
for( i=0; i < MAXCOLS && baskets[idx].cols[i]; i++)
- if( strcmp(basket[idx].cols[i], col)== 0)
+ if( strcmp(baskets[idx].cols[i], col)== 0)
break;
if( i < MAXCOLS)
return baskets[idx].bats[i];
@@ -279,9 +285,12 @@ BSKTbind(Client cntxt, MalBlkPtr mb, Mal
str sch = *getArgReference_str(stk,pci,2);
str tbl = *getArgReference_str(stk,pci,3);
str col = *getArgReference_str(stk,pci,4);
- BAT *bn, *b = BSKTbindColumn(cntxt, sch,tbl,col);
+ BAT *bn, *b = BSKTbindColumn(sch,tbl,col);
int bskt;
+ (void) cntxt;
+ (void) mb;
+
*ret = 0;
if( b){
bskt = BSKTlocate(sch,tbl);
@@ -291,17 +300,15 @@ BSKTbind(Client cntxt, MalBlkPtr mb, Mal
if( bn){
VIEWbounds(b,bn, 0,
baskets[bskt].winsize);
BBPkeepref(*ret = bn->batCacheid);
- BBPunfix(b->batCacheid);
- } else {
- BBPunfix(b->batCacheid);
+ } else
throw(SQL,"iot.bind","Can not create
view %s.%s.%s["BUNFMT"]",sch,tbl,col,baskets[bskt].winsize );
- }
- } else
- BBPkeepref(*ret = b->batCacheid);
+ } else{
+ BBPkeepref( *ret
MonetDB: iot - Merge with default
Changeset: b2982a1acaee for MonetDB URL: http://dev.monetdb.org/hg/MonetDB?cmd=changeset;node=b2982a1acaee Removed Files: clients/python2/ChangeLog clients/python2/ChangeLog-Archive clients/python2/ChangeLog.Jun2016 clients/python2/MANIFEST.in clients/python2/Makefile.ag clients/python2/README.rst clients/python2/monetdb/__init__.py clients/python2/monetdb/control.py clients/python2/monetdb/exceptions.py clients/python2/monetdb/mapi.py clients/python2/monetdb/sql/__init__.py clients/python2/monetdb/sql/connections.py clients/python2/monetdb/sql/converters.py clients/python2/monetdb/sql/cursors.py clients/python2/monetdb/sql/monetize.py clients/python2/monetdb/sql/pythonize.py clients/python2/monetdb/sql/types.py clients/python2/setup.py clients/python2/test/capabilities.py clients/python2/test/dbapi20.py clients/python2/test/run.cmd clients/python2/test/run.sh clients/python2/test/runtests.py clients/python2/test/test_control.py clients/python2/test/test_monetize.py clients/python2/test/test_pythonize.py clients/python3/ChangeLog clients/python3/ChangeLog-Archive clients/python3/ChangeLog.Jun2016 clients/python3/MANIFEST.in clients/python3/Makefile.ag clients/python3/README.rst clients/python3/monetdb/__init__.py clients/python3/monetdb/control.py clients/python3/monetdb/exceptions.py clients/python3/monetdb/mapi.py clients/python3/monetdb/sql/__init__.py clients/python3/monetdb/sql/connections.py clients/python3/monetdb/sql/converters.py clients/python3/monetdb/sql/cursors.py clients/python3/monetdb/sql/monetize.py clients/python3/monetdb/sql/pythonize.py clients/python3/monetdb/sql/types.py clients/python3/setup.py clients/python3/test/capabilities.py clients/python3/test/dbapi20.py clients/python3/test/run.cmd clients/python3/test/run.sh clients/python3/test/runtests.py clients/python3/test/test_control.py clients/python3/test/test_monetize.py clients/python3/test/test_pythonize.py monetdb5/tests/BugReports/Tests/no.146.stable.out sql/test/mapi/Tests/python2_test_monetdb_sql.SQL.bat sql/test/mapi/Tests/python2_test_monetdb_sql.SQL.sh sql/test/mapi/Tests/python2_test_monetdb_sql.stable.err sql/test/mapi/Tests/python2_test_monetdb_sql.stable.out sql/test/mapi/Tests/python3_test_monetdb_sql.SQL.bat sql/test/mapi/Tests/python3_test_monetdb_sql.SQL.sh sql/test/mapi/Tests/python3_test_monetdb_sql.stable.err sql/test/mapi/Tests/python3_test_monetdb_sql.stable.out Modified Files: .hgtags MonetDB.spec NT/Makefile NT/installer32/MonetDB-ODBC-Installer.vdproj NT/installer32/MonetDB5-Geom-Module.vdproj NT/installer32/MonetDB5-SQL-Installer.vdproj NT/installer64/MonetDB-ODBC-Installer.vdproj NT/installer64/MonetDB5-Geom-Module.vdproj NT/installer64/MonetDB5-SQL-Installer.vdproj NT/monetdb_config.h.in NT/rules.msc buildtools/autogen/autogen/am.py buildtools/autogen/autogen/msc.py clients/Makefile.ag clients/Tests/SQL-dump.stable.out clients/Tests/SQL-dump.stable.out.int128 clients/examples/python/basics.py clients/examples/python/mclient-python2.py clients/examples/python/mclient-python3.py clients/examples/python/perf.py clients/examples/python/sqlsample.py clients/mapiclient/msqldump.c clients/mapilib/mapi.rc clients/odbc/driver/driver.rc clients/odbc/winsetup/setup.rc common/stream/stream.c configure.ag debian/changelog debian/control debian/fix-deb.sh gdk/libbat.rc geom/lib/libgeom.c geom/monetdb5/geom.c geom/sql/Makefile.ag java/Makefile.ag java/build.properties java/pom.xml java/release.txt java/src/main/java/nl/cwi/monetdb/jdbc/MonetDatabaseMetaData.java java/src/main/java/nl/cwi/monetdb/jdbc/MonetPreparedStatement.java java/src/main/java/nl/cwi/monetdb/jdbc/MonetResultSet.java libversions monetdb5/ChangeLog-Archive monetdb5/mal/mal_dataflow.c monetdb5/mal/mal_instruction.c monetdb5/mal/mal_parser.c monetdb5/modules/atoms/url.c monetdb5/modules/mal/groupby.c monetdb5/modules/mal/sysmon.c monetdb5/modules/mal/tablet.c monetdb5/optimizer/opt_generator.c monetdb5/optimizer/opt_matpack.c monetdb5/tools/libmonetdb5.rc sql/backends/monet5/UDF/README sql/backends/monet5/iot/Makefile.am
MonetDB: Jun2016 - Use correct type.
Changeset: 42207239a7ed for MonetDB URL: http://dev.monetdb.org/hg/MonetDB?cmd=changeset;node=42207239a7ed Modified Files: sql/storage/bat/bat_storage.c Branch: Jun2016 Log Message: Use correct type. diffs (12 lines): diff --git a/sql/storage/bat/bat_storage.c b/sql/storage/bat/bat_storage.c --- a/sql/storage/bat/bat_storage.c +++ b/sql/storage/bat/bat_storage.c @@ -2091,7 +2091,7 @@ tr_update_dbat_swap(sql_dbat *tdb, sql_d * Swap both tdb->dbid/fdb->dbid */ int ok = LOG_OK; BAT *db = NULL; - size_t cnt; + BUN cnt; if (!fdb) return ok; ___ checkin-list mailing list checkin-list@monetdb.org https://www.monetdb.org/mailman/listinfo/checkin-list
