Hi Stefan, Valid point yourself :-)
I wholeheartedly support a swift switch to the algebra-backend. What I'm basically looking for is a convenient way to keep on having others developing applications on top of MXQ/PFTijah whereby prohibitive bugs can still be solved for these people. More thoughts: I presume the mps-code will remain in the CVS tree for as long as it takes, so bugs could be resolved, but others can only benefit if they would use a compilation of the CVS tree. Is it perhaps possible to easily create installation packages for the HEAD-version without actually releasing these packages as a formal release? We could 'release' these packages by putting them in the download-section of the PF/Tijah website. This would not make any claims concerning testing and stability, since it would be only a packaged new version of PF/Tijah with no formal status as release, but still allow new versions of PF/Tijah from being packaged and made available. The downside of such an approach is that from the perspective of people in the outside world looking at MonetDB/XQuery as an open source product, MonetDb/XQuery will loose some of its distinguishing features (fulltext querying capability) for a long time. Is that what you want? Any thoughts? Maurice. Stefan Manegold wrote: > Hi Maurice, > > valid point. > > Unfortunately, we (at least here at CWI) do not have the manpower an/or > budget that Microsoft has (recall, our "clients" do not pay for our > "product"). > > Hence, if a dual-path strategy is desired and/or required, please also > consider who should be doing all the administration, maintenance, bug > fixing, testing, releasing, etc... > > Stefan > > On Wed, Oct 17, 2007 at 10:15:17AM +0200, Keulen, M. van (Maurice) wrote: > >> Hi all, >> >> As a follow-up to Djoerd's response and my questions about releases, let >> me say a little bit more about certain activities and sketch an analogy >> which we might mimick. >> >> We are trying to get many people enthusiastic about XML databases and >> XML-IR, as offered by MonetDB/XQuery in particular. To achieve some >> visibility for our work, I have for example started several activities >> in building production applications on top of MonetDB/XQuery and which >> use PF/Tijah. For example, I promised a demonstration in january to >> university-wide decision makers of the UT of a publication management >> application which uses XML-exports from our E-Prints system including >> the full-text of the papers. I hope it will be accepted as a production >> application used by all researchers of the UT next to E-Prints, which >> will over time demonstrate more advanced features and take over more and >> more functionality of E-Prints. As Djoerd pointed out, porting PF/Tijah >> to the algebra-backend in the proper way, is more a matter of a year >> than of a few months. However, it would be certain death to such >> production applications if the support for developing them would stop >> (since algebra-only means without PF/Tijah for a long time). >> Furthermore, many students following our courses and doing our projects >> are very interested in XML databases including XML-IR and probabilistic >> XML. A significant portion would be rather discouraged if they would not >> be able to grab a stable release to start working and get some bugs fixed. >> >> Microsoft once had the problem that they had ideas for a fundamentally >> different operating system kernel (NT). It was infeasible to make it >> fully backward-compatible with Windows 95 (on top of MSDOS), but it >> could also not abandon a large part of their userbase if their >> applications would not work anymore. Therefore, they had two release >> streams: one NT-based and one MSDOS-based. Both knew their releases >> until in the end. When most of the applications were ported or at least >> working on the new technology, they stopped releasing in the MSDOS-based >> stream. >> >> The algebra-backend is our `new technology', so can't we do something >> similar? Have an algebra-based release stream of MonetDB/XQuery >> (starting with 0.22) capable of running a significant portion of >> applications, and an mps-based release stream needed to serve the >> PF/Tijah- and ProbXML-applications. It would mean an extra package under >> "Download" on SF. Of course the algebra-based releases will have >> priority, but it would make it possible to have a new release for for >> example PF/Tijah when it is needed. Of course we will put in the >> required effort, but cannot do this alone. >> >> What do you think of this? >> >> Maurice. >> >> Djoerd Hiemstra wrote: >> >>> Dear all, >>> >>> Very good: no more milprinting! In absence of Jan, please let me add the >>> plans for PF/Tijah. To make things slightly more complicated: The PF/Tijah >>> team would like to use the algebra release for porting our proprietary >>> search extensions to XQuery Full-text, see: >>> http://www.w3.org/TR/xquery-full-text/ Some initial design decisions that >>> we like to follow have been made by Stefan Klinger at the University of >>> Konstanz. The consequences of such a port would be: >>> >>> 1. XQuery syntax will be extended for text search queries (now only via >>> user-defined functions) >>> 2. All algebra operations must be score aware (i.e., if a score column >>> exists, scores have to be propagated and aggregated) >>> >>> We are aware that this affects everyone, although, only the extended >>> queries will produce a score column, and algebra operations remain exactly >>> the same if score columns are not there. The time goal of this release is >>> XMAS 2008. In the mean time, we might not port the current PF/Tijah >>> extensions to the algebra. >>> >>> Best, Djoerd. >>> >>> On Tue, 16 Oct 2007, Martin Kersten wrote: >>> >>> >>> >>>> LS, >>>> >>>> Good to see this discussion. Although the topic is primarily >>>> MonetDB/XQuery it is good >>>> to keep an eye on the global picture. Releases include more stuff and >>>> requires a lot of >>>> time to prepare. (the release itself easily consumes several months of >>>> full time work, >>>> many delivered by free (weekend) time!!!!) >>>> Good planning and long-term internal use are a prerequisite. >>>> Something we still have to learn. >>>> Once the release cycle starts, the train can not stop and full power >>>> is/should >>>> be in the hands of a single person. The release manager currently is >>>> Sjoerd, >>>> one of the project admins! >>>> >>>> As soon as a RC is available, those with a non-standard platform >>>> (windows, sun, >>>> mac, vista) should immediately double check to assure it basically works. >>>> This holds for all addressed and developers. >>>> >>>> In this respect, the time frame mentioned below is overly optimistic. >>>> Anyone close to >>>> the core group should work on the HEAD to help uncover problems from day 1. >>>> Code changes should *not* be collected by individual programmers until >>>> release time . >>>> This is a major source of frustration, because you simply may not expect >>>> your team members to spent day and night to fight the 'last' bugs. Let >>>> alone to >>>> quick-test your latest feature or performance improvement. >>>> PLEASE, check-in daily your changes and make sure you have at least ran >>>> mtest on your platform. And most of all, know that your changes are >>>> scrutinized for potential quality problems by others. It is the best way >>>> to avoid many bugs. >>>> >>>> Furthermore, changes in the interfaces visible to users and the effect >>>> on their databases >>>> should be kept to a minimum. Backward compatibility is a must as long as >>>> it is manageable. >>>> Beware that a change in storage structure may help a few extreme users, >>>> but may affects >>>> hundreds or thousands in their daily use. E.g. one of our users would >>>> have to unload/reload >>>> a 0.5Tb business critical application. >>>> >>>> MonetDB/Query and MonetDB/SQL are intertwined. This creates responsibility >>>> both ways. >>>> >>>> Peter Boncz wrote: >>>> >>>> >>>>> Stefan suggested rightly to move this discussion to the mailing list. >>>>> I also changed the language into english for that purpose. >>>>> >>>>> Maurice' original questions were: >>>>> >>>>> (a) whether the new release would be algebra-only (short answer: >>>>> hopefully). >>>>> >>>>> >>>> In my opinion this can only be answered if the algebra version, in its >>>> full glory (without mps) >>>> is used for several months internally, e.g. using students for stress >>>> testing. >>>> >>>> >>>>> (b) whether our talk of a new release also refers to submodules, such as >>>>> probxml and pftijah (short answer: yes) >>>>> >>>>> >>>>> >>>> Submodules basically follow their own release cycle. Regardless the >>>> source, e.g. datacells >>>> geom, .... Keeping in line is the responsibility of their owners, and >>>> the core developers >>>> should be aware that their changes may break the work of others. Or at >>>> least drain a >>>> possibly substantial amount of time. >>>> >>>> >>>>> (c) whether all issues on the wiki, see: >>>>> http://www.pathfinder-xquery.org/wiki/index.php/Jupiter:_The_Algebra_Release >>>>> needed for this action, have already been resolved (short answer: no) >>>>> >>>>> >>>>> >>>> The pathfinder wiki is a good example of team documentation. >>>> >>>> I am looking forward to the upcoming release. The next one might be this >>>> year, >>>> but not necessarily includes everything on the drawing board. >>>> >>>> regards, Martin >>>> >>>> >>>>> On Tue, Oct 16, 2007 at 05:23:43PM +0200, Peter Boncz wrote: >>>>> >>>>> >>>>> >>>>>> Hi Maurice, et al, >>>>>> >>>>>> The release of (hopefully) this week will still be mps based. I also >>>>>> anticipate at least one bug-fix release (ie 0.20.2), or even more. >>>>>> However, >>>>>> the next major release (0.22) should be algebra based, in my opinion. I >>>>>> am >>>>>> certainly reluctant to keep investing any more effort in mps. I would aim >>>>>> for an mps-free release, yes. >>>>>> >>>>>> The time-goal of that release is "XMAS 2007", as far as we are concerned. >>>>>> For the work of built-in functions I have good hope that with the help of >>>>>> Lefteris we can come a long way. >>>>>> >>>>>> I hope and think updates can also be ported by joerd in that time-frame. >>>>>> As >>>>>> for pftijah, if Jan Flokstra has time, it should be possible to port >>>>>> that as >>>>>> well in 2-3 months. >>>>>> >>>>>> There are some hairy issues, one of them being free-form recursion >>>>>> (needed >>>>>> by you), another one being precompiled modules (needed by XRPC). The >>>>>> resolution of these needs assistance from Munich, and maybe related. It >>>>>> needs >>>>>> a method to compile certain functions into scripted procedures (here MIL) >>>>>> and calling them. One algorithm is to do this for all functions in >>>>>> modules, >>>>>> as well as for all recursive functions. The issue to resolve is the >>>>>> representation of the parameters. >>>>>> >>>>>> Depending on how much time Jan Rittinger has available for this, we will >>>>>> have to find creative solutions for this. Indeed, a backup solution >>>>>> would be >>>>>> to keep handling certain queries for the time being with mps, but that >>>>>> surely is ugly. >>>>>> >>>>>> Peter >>>>>> >>>>>> -----Original Message----- >>>>>> From: Keulen, M. van (Maurice) [mailto:[EMAIL PROTECTED] >>>>>> Sent: dinsdag 16 oktober 2007 16:38 >>>>>> To: Peter Boncz [CWI]; Stefan Manegold [CWI] >>>>>> Cc: Djoerd Hiemstra; Jan Flokstra; Henning Rode; Abdel Kader, R. >>>>>> (Riham) >>>>>> Subject: Nieuwe releases en mps-backend >>>>>> >>>>>> >>>>>> Hoi, >>>>>> >>>>>> Sorry dat ik niet bij de Skype-sessie van vandaag kon zijn, maar Riham >>>>>> vertelde me dat het onderwerp van nieuwe releases en de mps-backend ter >>>>>> sprake is geweest. Het riep wat specifieke vragen bij me op die ze niet >>>>>> allemaal kon beantwoorden: >>>>>> >>>>>> a.. Is het de bedoeling dat een nieuwe release de mps-backend >>>>>> helemaal >>>>>> niet meer bevat of alleen maar de algebra-backend als default heeft? >>>>>> b.. Als jullie praten over "nieuwe release", bedoel je dan alleen een >>>>>> package van alle modules van MonetDB/XQuery? Maw zegt het niets over >>>>>> eventuele `subreleases' van individuele modules? >>>>>> >>>>>> c.. Op de wiki hebben we een pagina met open issues voor de >>>>>> algebra-release. Wat is de status van die open issues? Ik zie namelijk >>>>>> op de >>>>>> pagina nergens dat bepaalde issues al opgelost/geïmplementeerd zijn. >>>>>> Ik Cc deze e-mail even aan de PF/Tijah-mensen, want hoewel er >>>>>> levendige >>>>>> discussies zijn hoe PF/Tijah naar de algebra-backend to porten, ik heb >>>>>> niet >>>>>> het idee dat er oplossingen zijn die op korte termijn realiseerbaar zijn. >>>>>> Evenzo het recursie-vraagstuk. Het lijkt me bijvoorbeeld voor de >>>>>> zichtbaarheid naar buiten toe onwenselijk als deze features ineens zouden >>>>>> wegvallen voor non-insiders. Of andere onwenselijke consequenties van het >>>>>> niet meer (sub)releasen van modules die gebaseerd zijn op de mps-backend. >>>>>> >>>>>> Groetjes, >>>>>> Maurice. >>>>>> >>>>>> -- >>>>>> ---------------------------------------------------------------------- >>>>>> Dr.Ir. M. van Keulen - Assistant Professor, Data Management Technology >>>>>> Univ. of Twente, Dept of EEMCS, POBox 217, 7500 AE Enschede, Netherlands >>>>>> Email: [EMAIL PROTECTED], Phone: +31 534893688, Fax: +31 534892927 >>>>>> Room: ZI 3039, WWW: http://www.cs.utwente.nl/~keulen >>>>>> >>>>>> >>>>>> >>>>> >>>>> >>>>> >>>> ------------------------------------------------------------------------- >>>> This SF.net email is sponsored by: Splunk Inc. >>>> Still grepping through log files to find problems? Stop. >>>> Now Search log events and configuration files using AJAX and a browser. >>>> Download your FREE copy of Splunk now >> http://get.splunk.com/ >>>> _______________________________________________ >>>> Monetdb-developers mailing list >>>> [email protected] >>>> https://lists.sourceforge.net/lists/listinfo/monetdb-developers >>>> >>>> ------------------------------------------------------------------------ >>>> >>>> ------------------------------------------------------------------------- >>>> This SF.net email is sponsored by: Splunk Inc. >>>> Still grepping through log files to find problems? Stop. >>>> Now Search log events and configuration files using AJAX and a browser. >>>> Download your FREE copy of Splunk now >> http://get.splunk.com/ >>>> ------------------------------------------------------------------------ >>>> >>>> _______________________________________________ >>>> Monetdb-developers mailing list >>>> [email protected] >>>> https://lists.sourceforge.net/lists/listinfo/monetdb-developers >>>> >>>> >> -- >> ---------------------------------------------------------------------- >> Dr.Ir. M. van Keulen - Assistant Professor, Data Management Technology >> Univ. of Twente, Dept of EEMCS, POBox 217, 7500 AE Enschede, Netherlands >> Email: [EMAIL PROTECTED], Phone: +31 534893688, Fax: +31 534892927 >> Room: ZI 3039, WWW: http://www.cs.utwente.nl/~keulen >> >> >> ------------------------------------------------------------------------- >> This SF.net email is sponsored by: Splunk Inc. >> Still grepping through log files to find problems? Stop. >> Now Search log events and configuration files using AJAX and a browser. >> Download your FREE copy of Splunk now >> http://get.splunk.com/ >> _______________________________________________ >> Monetdb-developers mailing list >> [email protected] >> https://lists.sourceforge.net/lists/listinfo/monetdb-developers >> > > -- ---------------------------------------------------------------------- Dr.Ir. M. van Keulen - Assistant Professor, Data Management Technology Univ. of Twente, Dept of EEMCS, POBox 217, 7500 AE Enschede, Netherlands Email: [EMAIL PROTECTED], Phone: +31 534893688, Fax: +31 534892927 Room: ZI 3039, WWW: http://www.cs.utwente.nl/~keulen ------------------------------------------------------------------------- This SF.net email is sponsored by: Splunk Inc. Still grepping through log files to find problems? Stop. Now Search log events and configuration files using AJAX and a browser. Download your FREE copy of Splunk now >> http://get.splunk.com/ _______________________________________________ Monetdb-developers mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/monetdb-developers
