Re: [Sedna-discussion] DB API performance issue - fn:doc-available
P.S. This is the corresponding commit: http://sedna.org/gitweb/sedna.git/commit/a7306ad318593d7ff308628f8cfb542ddfa 672b2 As it's dated by today, I expect it'll be available in the night builds since tomorrow. @Ivan, please correct me if I'm wrong. Best regards, Ivan Lagunov From: Ivan Lagunov [mailto:lagi...@gmail.com] Sent: Friday, April 12, 2013 5:30 PM To: 'Charles Foster' Cc: 'sedna-discussion'; 'Ivan Shcheklein' Subject: XML:DB API performance issue - fn:doc-available Hi Charles, We're having serious performance issues, probably reached the critical number of documents in the database. According to Ivan Shcheklein, it is likely to be caused by using doc(documents) function that actually locks all collections in the database, thus, drastically decreasing performance. Your implementation of XML:DB API works great for small datasets but according to event.log it uses doc(documents) quite a lot. So it may indeed result in bad performance I'm facing. As an improvement, there have been implemented a function fn:doc-available that checks if a document is available in the given collection. It locks only one collection. This is an example of usage: doc-available(collection, document) Could you please update the Sedna XML:DB API implementation using this function? I understand it's recommended to migrate to XQJ implementation but it's really hard to rewrite all the code we have. Nevertheless, I've planned to investigate required efforts for this migration but we need some solution on short notice. Thanks a lot! Best regards, Ivan Lagunov -- Precog is a next-generation analytics platform capable of advanced analytics on semi-structured data. The platform includes APIs for building apps and a phenomenal toolset for data science. Developers can use our toolset for easy data analysis visualization. Get a free account! http://www2.precog.com/precogplatform/slashdotnewsletter___ Sedna-discussion mailing list Sedna-discussion@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/sedna-discussion
Re: [Sedna-discussion] DB API performance issue - fn:doc-available
Charles, Development build will be available tomorrow: http://modis.ispras.ru/FTPContent/sedna/development/ . Version will be = 3.5.602. Or you can get latest sources: git clone git://modis.ispras.ru/sedna . Thank you for your prompt response! As far as I remember, there are other users waiting for this improvement. Ivan Shcheklein, Sedna Team Ivan Shcheklein, Sedna Team On Fri, Apr 12, 2013 at 7:36 PM, Charles Foster char...@cfoster.net wrote: Hi Guys, Yes, I'll sort this out later and issue a new release. Just please tell me which specific version of Sedna this new functionality came in please? Kind Regards, Charles On 12 Apr 2013, at 16:33, Ivan Lagunov lagi...@gmail.com wrote: P.S. This is the corresponding commit: http://sedna.org/gitweb/sedna.git/commit/a7306ad318593d7ff308628f8cfb542ddfa672b2 As it’s dated by today, I expect it’ll be available in the night builds since tomorrow. ** ** @Ivan, please correct me if I’m wrong. ** ** Best regards, Ivan Lagunov *From:* Ivan Lagunov [mailto:lagi...@gmail.com] *Sent:* Friday, April 12, 2013 5:30 PM *To:* 'Charles Foster' *Cc:* 'sedna-discussion'; 'Ivan Shcheklein' *Subject:* XML:DB API performance issue - fn:doc-available ** ** Hi Charles, ** ** We’re having serious performance issues, probably reached the critical number of documents in the database. According to Ivan Shcheklein, it is likely to be caused by using doc(“documents”) function that actually locks all collections in the database, thus, drastically decreasing performance. Your implementation of XML:DB API works great for small datasets but according to event.log it uses doc(“documents”) quite a lot. So it may indeed result in bad performance I’m facing. As an improvement, there have been implemented a function fn:doc-available that checks if a document is available in the given collection. It locks only one collection. This is an example of usage: ** ** doc-available(“collection”, “document”) ** ** Could you please update the Sedna XML:DB API implementation using this function? I understand it’s recommended to migrate to XQJ implementation but it’s really hard to rewrite all the code we have. Nevertheless, I’ve planned to investigate required efforts for this migration but we need some solution on short notice. Thanks a lot! ** ** Best regards, Ivan Lagunov -- Precog is a next-generation analytics platform capable of advanced analytics on semi-structured data. The platform includes APIs for building apps and a phenomenal toolset for data science. Developers can use our toolset for easy data analysis visualization. Get a free account! http://www2.precog.com/precogplatform/slashdotnewsletter___ Sedna-discussion mailing list Sedna-discussion@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/sedna-discussion
Re: [Sedna-discussion] DB API performance issue - fn:doc-available
Hi Guys, Yes, I'll sort this out later and issue a new release. Just please tell me which specific version of Sedna this new functionality came in please? Kind Regards, Charles On 12 Apr 2013, at 16:33, Ivan Lagunov lagi...@gmail.com wrote: P.S. This is the corresponding commit: http://sedna.org/gitweb/sedna.git/commit/a7306ad318593d7ff308628f8cfb542ddfa672b2 As it’s dated by today, I expect it’ll be available in the night builds since tomorrow. @Ivan, please correct me if I’m wrong. Best regards, Ivan Lagunov From: Ivan Lagunov [mailto:lagi...@gmail.com] Sent: Friday, April 12, 2013 5:30 PM To: 'Charles Foster' Cc: 'sedna-discussion'; 'Ivan Shcheklein' Subject: XML:DB API performance issue - fn:doc-available Hi Charles, We’re having serious performance issues, probably reached the critical number of documents in the database. According to Ivan Shcheklein, it is likely to be caused by using doc(“documents”) function that actually locks all collections in the database, thus, drastically decreasing performance. Your implementation of XML:DB API works great for small datasets but according to event.log it uses doc(“documents”) quite a lot. So it may indeed result in bad performance I’m facing. As an improvement, there have been implemented a function fn:doc-available that checks if a document is available in the given collection. It locks only one collection. This is an example of usage: doc-available(“collection”, “document”) Could you please update the Sedna XML:DB API implementation using this function? I understand it’s recommended to migrate to XQJ implementation but it’s really hard to rewrite all the code we have. Nevertheless, I’ve planned to investigate required efforts for this migration but we need some solution on short notice. Thanks a lot! Best regards, Ivan Lagunov -- Precog is a next-generation analytics platform capable of advanced analytics on semi-structured data. The platform includes APIs for building apps and a phenomenal toolset for data science. Developers can use our toolset for easy data analysis visualization. Get a free account! http://www2.precog.com/precogplatform/slashdotnewsletter___ Sedna-discussion mailing list Sedna-discussion@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/sedna-discussion
