Re: [Monetdb-developers] [Monetdb-pf-checkins] pathfinder/modules/pftijah pftijah.mx, , 1.256, 1.257
Jan, out of curiosity: - on nwhich architectures and/or in which cases does semijoin not propagate the 'sorted' properties? - shouldn't this checkin have gone as bug fix to the release branch such that is would be included in the next release? Thanks. Stefan On Fri, Feb 19, 2010 at 08:22:08AM +, Jan Flokstra wrote: > Update of /cvsroot/monetdb/pathfinder/modules/pftijah > In directory sfp-cvsdas-1.v30.ch3.sourceforge.com:/tmp/cvs-serv19238 > > Modified Files: > pftijah.mx > Log Message: > - temporary patch to fix a case where on architectures a semijoin does not > propagate the 'sorted' properties. > > > > Index: pftijah.mx > === > RCS file: /cvsroot/monetdb/pathfinder/modules/pftijah/pftijah.mx,v > retrieving revision 1.256 > retrieving revision 1.257 > diff -u -d -r1.256 -r1.257 > --- pftijah.mx12 Feb 2010 10:25:43 - 1.256 > +++ pftijah.mx19 Feb 2010 08:22:06 - 1.257 > @@ -4337,6 +4337,7 @@ > # get collection count of term > var t_pre := _getTermPositions($h); > e_pre := e_p...@5(_gettermd...@1_@2(e_pre, e_size, t_pre)); > + e_pre := e_pre.chk_order(); # jf order patch > } > > var res := e_pre; > @@ -4536,6 +4537,7 @@ > # get collection count of term > var t_pre := _getConceptPositions($h); > e_pre := e_p...@5(_gettermd...@1_@2(e_pre, e_size, t_pre)); > + e_pre := e_pre.chk_order(); # jf order patch > } > > var res := e_pre; > > > -- > Download Intel® Parallel Studio Eval > Try the new software tools for yourself. Speed compiling, find bugs > proactively, and fine-tune applications for parallel performance. > See why Intel Parallel Studio got high marks during beta. > http://p.sf.net/sfu/intel-sw-dev > ___ > Monetdb-pf-checkins mailing list > monetdb-pf-check...@lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/monetdb-pf-checkins > -- | Dr. Stefan Manegold | mailto:stefan.maneg...@cwi.nl | | CWI, P.O.Box 94079 | http://www.cwi.nl/~manegold/ | | 1090 GB Amsterdam | Tel.: +31 (20) 592-4212 | | The Netherlands | Fax : +31 (20) 592-4199 | -- Download Intel® Parallel Studio Eval Try the new software tools for yourself. Speed compiling, find bugs proactively, and fine-tune applications for parallel performance. See why Intel Parallel Studio got high marks during beta. http://p.sf.net/sfu/intel-sw-dev ___ Monetdb-developers mailing list Monetdb-developers@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/monetdb-developers
Re: [Monetdb-developers] [Monetdb-pf-checkins] pathfinder/runtimeshredder.mx, Feb2010, 1.162, 1.162.2.1
I don't care how it is solved, but if the DTD can't be loaded, dtd is
NULL, and you cannot dereference NULL (in handle_externalSubset), so
there have to be checks to make sure that dtd isn't NULL before it is
dereferenced. This was the easy way, but perhaps there are other ways.
On 2010-02-19 08:46, Jan Flokstra wrote:
> On Thursday 18 February 2010 21:00, Jan Rittinger wrote:
>> Hi Sjoerd,
>>
>> as far as I can remember we only use the DTD to resolve entities and id
>> references. So if no DTD is available an error is triggered, but the
>> document is still imported.
>>
>> Just my two cents...
>>
>
> Hi Sjoerd and Jan,
>
> This is correct. But when the shredder was built I also had a couple of
> examples with DTD's which could not be found but where the document loaded
> correct so we decided not to bailout and only print an error. Maybe this
> solution is better,
>
> Regards, Jan.
>
>> Jan
>>
>> On Feb 18, 2010, at 10:50, Sjoerd Mullender wrote:
>>> Update of /cvsroot/monetdb/pathfinder/runtime
>>> In directory sfp-cvsdas-1.v30.ch3.sourceforge.com:/tmp/cvs-serv30011
>>>
>>> Modified Files:
>>> Tag: Feb2010
>>> shredder.mx
>>> Log Message:
>>> If there is no DTD, I guess there is not much point continuing.
>>> Found by Coverity.
>>>
>>>
>>> Index: shredder.mx
>>> ===
>>> RCS file: /cvsroot/monetdb/pathfinder/runtime/shredder.mx,v
>>> retrieving revision 1.162
>>> retrieving revision 1.162.2.1
>>> diff -u -d -r1.162 -r1.162.2.1
>>> --- shredder.mx 7 Jan 2010 15:24:29 - 1.162
>>> +++ shredder.mx 18 Feb 2010 09:50:12 - 1.162.2.1
>>> @@ -1362,8 +1362,10 @@
>>> ctx->myDoc = xmlNewDoc(ctx->version);
>>> }
>>>
>>> -if (!dtd)
>>> +if (!dtd) {
>>> GDKerror("shred_external_subset: problem with reading
>>> external DTD (\"%s\")\n", SystemID); +BAILOUT(shredCtx);
>>> +}
>>>
>>> if (ctx->myDoc->extSubset) {
>>> stream_printf(GDKout, "!WARNING: double external subset
>>> (\"%s\") may cause problems.\n", SystemID);
>>>
>>>
>>> -
>>> - Download Intel® Parallel Studio Eval
>>> Try the new software tools for yourself. Speed compiling, find bugs
>>> proactively, and fine-tune applications for parallel performance.
>>> See why Intel Parallel Studio got high marks during beta.
>>> http://p.sf.net/sfu/intel-sw-dev
>>> ___
>>> Monetdb-pf-checkins mailing list
>>> monetdb-pf-check...@lists.sourceforge.net
>>> https://lists.sourceforge.net/lists/listinfo/monetdb-pf-checkins
>
> --
> Download Intel® Parallel Studio Eval
> Try the new software tools for yourself. Speed compiling, find bugs
> proactively, and fine-tune applications for parallel performance.
> See why Intel Parallel Studio got high marks during beta.
> http://p.sf.net/sfu/intel-sw-dev
> ___
> Monetdb-developers mailing list
> Monetdb-developers@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/monetdb-developers
--
Sjoerd Mullender
signature.asc
Description: OpenPGP digital signature
--
Download Intel® Parallel Studio Eval
Try the new software tools for yourself. Speed compiling, find bugs
proactively, and fine-tune applications for parallel performance.
See why Intel Parallel Studio got high marks during beta.
http://p.sf.net/sfu/intel-sw-dev___
Monetdb-developers mailing list
Monetdb-developers@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/monetdb-developers
Re: [Monetdb-developers] [Monetdb-checkins] MonetDB/src/gdk gdk_posix.mx, Feb2010, 1.176.2.24, 1.176.2.25 gdk_storage.mx, Feb2010, 1.149.2.36, 1.149.2.37
On Fri, Feb 19, 2010 at 03:40:10PM +, Peter Boncz wrote:
> Update of /cvsroot/monetdb/MonetDB/src/gdk
> In directory sfp-cvsdas-1.v30.ch3.sourceforge.com:/tmp/cvs-serv10221
>
> Modified Files:
> Tag: Feb2010
> gdk_posix.mx gdk_storage.mx
> Log Message:
> - fix newly introduced bug (BUF_SEQUENTIAL passed to posix_madvise iso
> MMAP_SEQUENTIAL)
> most of this was done by Stefan, but the intent of the code was when all
> users
> are gone, to give uniform (sequential) advise to the whole heap
> - make sure the length of advise is page aligned (not strictly needed for
> Linux, but maybe of other Un*ces)
> - in case of shared vheaps (as done by leftfetchjoin into a string bat),
> assume that
> access to the string heap will be random.
shouldn't
len & ~MT_pagesize()
then rather be
len & ~(MT_pagesize()-1)
(assuming pages size is usually a power of 2)
?
Stefan
>
>
> Index: gdk_storage.mx
> ===
> RCS file: /cvsroot/monetdb/MonetDB/src/gdk/gdk_storage.mx,v
> retrieving revision 1.149.2.36
> retrieving revision 1.149.2.37
> diff -u -d -r1.149.2.36 -r1.149.2.37
> --- gdk_storage.mx19 Feb 2010 13:13:38 - 1.149.2.36
> +++ gdk_storage.mx19 Feb 2010 15:40:07 - 1.149.2.37
> @@ -707,17 +707,19 @@
> * Peter Feb2010: I tried to do prefetches further apart, to trigger
> multiple readahead
> *units in parallel, but it does improve performance visibly
> */
> -static size_t access_heap(str id, str hp, Heap *h, char* base, size_t sz,
> int touch, int preload, int advise) {
> +static size_t access_heap(str id, str hp, Heap *h, char* base, size_t sz,
> int touch, int preload, int adv) {
> size_t v0 = 0, v1 = 0, v2 = 0, v3 = 0, v4 = 0, v5 =0, v6 = 0, v7 = 0,
> page = MT_pagesize();
> + str advise =
> (adv==MMAP_WILLNEED)?"WILLNEED":(adv==MMAP_SEQUENTIAL)?"SEQUENTIAL":(adv==MMAP_RANDOM)?"RANDOM":(adv==MMAP_NORMAL)?"NORMAL":NULL;
> int t = GDKms();
> + assert(advise);
> if (h->storage != STORE_MEM && h->size > MT_MMAP_TILE) {
> - MT_mmap_inform(h->base, h->size, preload, advise, 0);
> + MT_mmap_inform(h->base, h->size, preload, adv, 0);
> if (preload > 0) {
> - void* alignedbase = (void*) (((size_t) base) &
> ~(page-1));
> - size_t alignedsz = (sz + (page-1)) & ~(page-1);
> - int ret = posix_madvise(alignedbase, sz, advise);
> - if (ret) THRprintf(GDKerr, "#MT_mmap_inform:
> posix_madvise(file=%s, base="PTRFMT", len="SZFMT"MB, advice=%d) = %d\n",
> - h->filename, PTRFMTCAST alignedbase,
> alignedsz >> 20, advise, errno);
> + size_t alignskip = (page - (((size_t) base) &
> (page-1))) & (page-1);
> + size_t alignedsz = (size_t) (((sz <
> alignskip)?0:((size_t) (sz-alignskip))) & ~(page-1));
> + int ret = posix_madvise(base + alignskip, alignedsz,
> adv);
> + if (ret) THRprintf(GDKerr, "#MT_mmap_inform:
> posix_madvise(file=%s, base="PTRFMT", len="SZFMT"MB, advice=%s) = %d\n",
> + h->filename, PTRFMTCAST (base +
> alignskip), alignedsz >> 20, advise, errno);
> }
> }
> if (touch && preload > 0) {
> @@ -731,8 +733,7 @@
> }
> for (hi += 7*page; lo <= hi; lo +=page) v0 += *lo;
> }
> - IODEBUG THRprintf(GDKout,"#BATpreload(%s->%s,preload=%d,sz=%dMB,%s) =
> %dms \n", id, hp, preload, (int) (sz>>20),
> -
> (advise==MMAP_WILLNEED)?"WILLNEED":(advise==MMAP_SEQUENTIAL)?"SEQUENTIAL":"UNKNOWN",
> GDKms()-t);
> + IODEBUG THRprintf(GDKout,"#BATpreload(%s->%s,preload=%d,sz=%dMB,%s) =
> %dms \n", id, hp, preload, (int) (sz>>20), advise, GDKms()-t);
> return v0+v1+v2+v3+v4+v5+v6+v7;
> }
>
> @@ -743,7 +744,6 @@
> str id = BATgetId(b);
> BATiter bi = bat_iterator(b);
>
> -
> assert(advise==MMAP_NORMAL||advise==MMAP_RANDOM||advise==MMAP_SEQUENTIAL||advise==MMAP_WILLNEED||advise==MMAP_DONTNEED);
> if (BATcount(b) == 0) return 0;
>
> /* HASH indices (inherent random access). handle first as they *will*
> be access randomly (one can always hope for locality on the other heaps) */
> @@ -760,30 +760,46 @@
> gdk_unset_lock(GDKhashLock(ABS(b->batCacheid) & BBP_BATMASK),
> "BATaccess");
> }
>
> - /* we only touch stuff that is going to be read randomly (WILLNEED).
> Note varheaps are sequential wrt to the references, or small */
> - if ( what&USE_HEAD) {
> + /* vheaps next, as shared vheaps are not seq-correlated needing
> WILLNEED (use prefetch budget for this first) */
> + if ( what&USE_HEAD ) {
> + if (b->H->vheap && b->H->vheap->base) {
> + char *lo = BUNhead(bi, BUNfirst(b)), *hi = BUNhead(bi,
[Monetdb-developers] pf:new-doc
Hello MonetDB developers,
In my quest to develop an object oriented interface to monetdb/xquery
on ruby using datamapper, I've stumbled upon the following problem. It
is at this moment impossible to add a document to monetdb that does
not have a source outside monetdb. The only function to add documents
to monetdb seems to be pf:add-doc, which takes an url parameter with a
location of an xml document it should import. To fully abstract the
database from the ruby application, it would be nice if the developers
could add a new (empty) document to the database.
Would it be a nice idea to have the following function:
pf:new-doc("document_name.xml"[,
"collection_name.xml"[,"content"]])
It would make users of my api a little happier, and perhaps
MonetDB/XQuery as a whole, a bit more normal ;)
Kind regards,
Tinco Andringa
--
Download Intel® Parallel Studio Eval
Try the new software tools for yourself. Speed compiling, find bugs
proactively, and fine-tune applications for parallel performance.
See why Intel Parallel Studio got high marks during beta.
http://p.sf.net/sfu/intel-sw-dev
___
Monetdb-developers mailing list
Monetdb-developers@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/monetdb-developers
