Hi all, As a follow-up to Djoerd's response and my questions about releases, let me say a little bit more about certain activities and sketch an analogy which we might mimick.
We are trying to get many people enthusiastic about XML databases and XML-IR, as offered by MonetDB/XQuery in particular. To achieve some visibility for our work, I have for example started several activities in building production applications on top of MonetDB/XQuery and which use PF/Tijah. For example, I promised a demonstration in january to university-wide decision makers of the UT of a publication management application which uses XML-exports from our E-Prints system including the full-text of the papers. I hope it will be accepted as a production application used by all researchers of the UT next to E-Prints, which will over time demonstrate more advanced features and take over more and more functionality of E-Prints. As Djoerd pointed out, porting PF/Tijah to the algebra-backend in the proper way, is more a matter of a year than of a few months. However, it would be certain death to such production applications if the support for developing them would stop (since algebra-only means without PF/Tijah for a long time). Furthermore, many students following our courses and doing our projects are very interested in XML databases including XML-IR and probabilistic XML. A significant portion would be rather discouraged if they would not be able to grab a stable release to start working and get some bugs fixed. Microsoft once had the problem that they had ideas for a fundamentally different operating system kernel (NT). It was infeasible to make it fully backward-compatible with Windows 95 (on top of MSDOS), but it could also not abandon a large part of their userbase if their applications would not work anymore. Therefore, they had two release streams: one NT-based and one MSDOS-based. Both knew their releases until in the end. When most of the applications were ported or at least working on the new technology, they stopped releasing in the MSDOS-based stream. The algebra-backend is our `new technology', so can't we do something similar? Have an algebra-based release stream of MonetDB/XQuery (starting with 0.22) capable of running a significant portion of applications, and an mps-based release stream needed to serve the PF/Tijah- and ProbXML-applications. It would mean an extra package under "Download" on SF. Of course the algebra-based releases will have priority, but it would make it possible to have a new release for for example PF/Tijah when it is needed. Of course we will put in the required effort, but cannot do this alone. What do you think of this? Maurice. Djoerd Hiemstra wrote: > Dear all, > > Very good: no more milprinting! In absence of Jan, please let me add the > plans for PF/Tijah. To make things slightly more complicated: The PF/Tijah > team would like to use the algebra release for porting our proprietary > search extensions to XQuery Full-text, see: > http://www.w3.org/TR/xquery-full-text/ Some initial design decisions that > we like to follow have been made by Stefan Klinger at the University of > Konstanz. The consequences of such a port would be: > > 1. XQuery syntax will be extended for text search queries (now only via > user-defined functions) > 2. All algebra operations must be score aware (i.e., if a score column > exists, scores have to be propagated and aggregated) > > We are aware that this affects everyone, although, only the extended > queries will produce a score column, and algebra operations remain exactly > the same if score columns are not there. The time goal of this release is > XMAS 2008. In the mean time, we might not port the current PF/Tijah > extensions to the algebra. > > Best, Djoerd. > > On Tue, 16 Oct 2007, Martin Kersten wrote: > > >> LS, >> >> Good to see this discussion. Although the topic is primarily >> MonetDB/XQuery it is good >> to keep an eye on the global picture. Releases include more stuff and >> requires a lot of >> time to prepare. (the release itself easily consumes several months of >> full time work, >> many delivered by free (weekend) time!!!!) >> Good planning and long-term internal use are a prerequisite. >> Something we still have to learn. >> Once the release cycle starts, the train can not stop and full power >> is/should >> be in the hands of a single person. The release manager currently is Sjoerd, >> one of the project admins! >> >> As soon as a RC is available, those with a non-standard platform >> (windows, sun, >> mac, vista) should immediately double check to assure it basically works. >> This holds for all addressed and developers. >> >> In this respect, the time frame mentioned below is overly optimistic. >> Anyone close to >> the core group should work on the HEAD to help uncover problems from day 1. >> Code changes should *not* be collected by individual programmers until >> release time . >> This is a major source of frustration, because you simply may not expect >> your team members to spent day and night to fight the 'last' bugs. Let >> alone to >> quick-test your latest feature or performance improvement. >> PLEASE, check-in daily your changes and make sure you have at least ran >> mtest on your platform. And most of all, know that your changes are >> scrutinized for potential quality problems by others. It is the best way >> to avoid many bugs. >> >> Furthermore, changes in the interfaces visible to users and the effect >> on their databases >> should be kept to a minimum. Backward compatibility is a must as long as >> it is manageable. >> Beware that a change in storage structure may help a few extreme users, >> but may affects >> hundreds or thousands in their daily use. E.g. one of our users would >> have to unload/reload >> a 0.5Tb business critical application. >> >> MonetDB/Query and MonetDB/SQL are intertwined. This creates responsibility >> both ways. >> >> Peter Boncz wrote: >> >>> Stefan suggested rightly to move this discussion to the mailing list. >>> I also changed the language into english for that purpose. >>> >>> Maurice' original questions were: >>> >>> (a) whether the new release would be algebra-only (short answer: hopefully). >>> >> In my opinion this can only be answered if the algebra version, in its >> full glory (without mps) >> is used for several months internally, e.g. using students for stress >> testing. >> >>> (b) whether our talk of a new release also refers to submodules, such as >>> probxml and pftijah (short answer: yes) >>> >>> >> Submodules basically follow their own release cycle. Regardless the >> source, e.g. datacells >> geom, .... Keeping in line is the responsibility of their owners, and >> the core developers >> should be aware that their changes may break the work of others. Or at >> least drain a >> possibly substantial amount of time. >> >>> (c) whether all issues on the wiki, see: >>> http://www.pathfinder-xquery.org/wiki/index.php/Jupiter:_The_Algebra_Release >>> needed for this action, have already been resolved (short answer: no) >>> >>> >> The pathfinder wiki is a good example of team documentation. >> >> I am looking forward to the upcoming release. The next one might be this >> year, >> but not necessarily includes everything on the drawing board. >> >> regards, Martin >> >>> On Tue, Oct 16, 2007 at 05:23:43PM +0200, Peter Boncz wrote: >>> >>> >>>> Hi Maurice, et al, >>>> >>>> The release of (hopefully) this week will still be mps based. I also >>>> anticipate at least one bug-fix release (ie 0.20.2), or even more. >>>> However, >>>> the next major release (0.22) should be algebra based, in my opinion. I am >>>> certainly reluctant to keep investing any more effort in mps. I would aim >>>> for an mps-free release, yes. >>>> >>>> The time-goal of that release is "XMAS 2007", as far as we are concerned. >>>> For the work of built-in functions I have good hope that with the help of >>>> Lefteris we can come a long way. >>>> >>>> I hope and think updates can also be ported by joerd in that time-frame. As >>>> for pftijah, if Jan Flokstra has time, it should be possible to port that >>>> as >>>> well in 2-3 months. >>>> >>>> There are some hairy issues, one of them being free-form recursion (needed >>>> by you), another one being precompiled modules (needed by XRPC). The >>>> resolution of these needs assistance from Munich, and maybe related. It >>>> needs >>>> a method to compile certain functions into scripted procedures (here MIL) >>>> and calling them. One algorithm is to do this for all functions in modules, >>>> as well as for all recursive functions. The issue to resolve is the >>>> representation of the parameters. >>>> >>>> Depending on how much time Jan Rittinger has available for this, we will >>>> have to find creative solutions for this. Indeed, a backup solution would >>>> be >>>> to keep handling certain queries for the time being with mps, but that >>>> surely is ugly. >>>> >>>> Peter >>>> >>>> -----Original Message----- >>>> From: Keulen, M. van (Maurice) [mailto:[EMAIL PROTECTED] >>>> Sent: dinsdag 16 oktober 2007 16:38 >>>> To: Peter Boncz [CWI]; Stefan Manegold [CWI] >>>> Cc: Djoerd Hiemstra; Jan Flokstra; Henning Rode; Abdel Kader, R. (Riham) >>>> Subject: Nieuwe releases en mps-backend >>>> >>>> >>>> Hoi, >>>> >>>> Sorry dat ik niet bij de Skype-sessie van vandaag kon zijn, maar Riham >>>> vertelde me dat het onderwerp van nieuwe releases en de mps-backend ter >>>> sprake is geweest. Het riep wat specifieke vragen bij me op die ze niet >>>> allemaal kon beantwoorden: >>>> >>>> a.. Is het de bedoeling dat een nieuwe release de mps-backend helemaal >>>> niet meer bevat of alleen maar de algebra-backend als default heeft? >>>> b.. Als jullie praten over "nieuwe release", bedoel je dan alleen een >>>> package van alle modules van MonetDB/XQuery? Maw zegt het niets over >>>> eventuele `subreleases' van individuele modules? >>>> >>>> c.. Op de wiki hebben we een pagina met open issues voor de >>>> algebra-release. Wat is de status van die open issues? Ik zie namelijk op >>>> de >>>> pagina nergens dat bepaalde issues al opgelost/geïmplementeerd zijn. >>>> Ik Cc deze e-mail even aan de PF/Tijah-mensen, want hoewel er levendige >>>> discussies zijn hoe PF/Tijah naar de algebra-backend to porten, ik heb niet >>>> het idee dat er oplossingen zijn die op korte termijn realiseerbaar zijn. >>>> Evenzo het recursie-vraagstuk. Het lijkt me bijvoorbeeld voor de >>>> zichtbaarheid naar buiten toe onwenselijk als deze features ineens zouden >>>> wegvallen voor non-insiders. Of andere onwenselijke consequenties van het >>>> niet meer (sub)releasen van modules die gebaseerd zijn op de mps-backend. >>>> >>>> Groetjes, >>>> Maurice. >>>> >>>> -- >>>> ---------------------------------------------------------------------- >>>> Dr.Ir. M. van Keulen - Assistant Professor, Data Management Technology >>>> Univ. of Twente, Dept of EEMCS, POBox 217, 7500 AE Enschede, Netherlands >>>> Email: [EMAIL PROTECTED], Phone: +31 534893688, Fax: +31 534892927 >>>> Room: ZI 3039, WWW: http://www.cs.utwente.nl/~keulen >>>> >>>> >>> >>> >> ------------------------------------------------------------------------- >> This SF.net email is sponsored by: Splunk Inc. >> Still grepping through log files to find problems? Stop. >> Now Search log events and configuration files using AJAX and a browser. >> Download your FREE copy of Splunk now >> http://get.splunk.com/ >> _______________________________________________ >> Monetdb-developers mailing list >> [email protected] >> https://lists.sourceforge.net/lists/listinfo/monetdb-developers >> >> ------------------------------------------------------------------------ >> >> ------------------------------------------------------------------------- >> This SF.net email is sponsored by: Splunk Inc. >> Still grepping through log files to find problems? Stop. >> Now Search log events and configuration files using AJAX and a browser. >> Download your FREE copy of Splunk now >> http://get.splunk.com/ >> ------------------------------------------------------------------------ >> >> _______________________________________________ >> Monetdb-developers mailing list >> [email protected] >> https://lists.sourceforge.net/lists/listinfo/monetdb-developers >> -- ---------------------------------------------------------------------- Dr.Ir. M. van Keulen - Assistant Professor, Data Management Technology Univ. of Twente, Dept of EEMCS, POBox 217, 7500 AE Enschede, Netherlands Email: [EMAIL PROTECTED], Phone: +31 534893688, Fax: +31 534892927 Room: ZI 3039, WWW: http://www.cs.utwente.nl/~keulen ------------------------------------------------------------------------- This SF.net email is sponsored by: Splunk Inc. Still grepping through log files to find problems? Stop. Now Search log events and configuration files using AJAX and a browser. Download your FREE copy of Splunk now >> http://get.splunk.com/ _______________________________________________ Monetdb-developers mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/monetdb-developers
