[Zorba-coders] [Merge] lp:~zorba-coders/zorba/bug-921458 into lp:zorba
The proposal to merge lp:~zorba-coders/zorba/bug-921458 into lp:zorba has been updated. Status: Approved => Merged For more details, see: https://code.launchpad.net/~zorba-coders/zorba/bug-921458/+merge/102058 -- https://code.launchpad.net/~zorba-coders/zorba/bug-921458/+merge/102058 Your team Zorba Coders is subscribed to branch lp:zorba. -- Mailing list: https://launchpad.net/~zorba-coders Post to : zorba-coders@lists.launchpad.net Unsubscribe : https://launchpad.net/~zorba-coders More help : https://help.launchpad.net/ListHelp
[Zorba-coders] [Merge] lp:~zorba-coders/zorba/bug-921458 into lp:zorba
Validation queue job bug-921458-2012-04-16T14-47-08.502Z is finished. The final status was: All tests succeeded! -- https://code.launchpad.net/~zorba-coders/zorba/bug-921458/+merge/102058 Your team Zorba Coders is subscribed to branch lp:zorba. -- Mailing list: https://launchpad.net/~zorba-coders Post to : zorba-coders@lists.launchpad.net Unsubscribe : https://launchpad.net/~zorba-coders More help : https://help.launchpad.net/ListHelp
[Zorba-coders] [Merge] lp:~zorba-coders/zorba/bug-921458 into lp:zorba
Validation queue starting for merge proposal. Log at: http://zorbatest.lambda.nu:8080/remotequeue/bug-921458-2012-04-16T14-47-08.502Z/log.html -- https://code.launchpad.net/~zorba-coders/zorba/bug-921458/+merge/102058 Your team Zorba Coders is subscribed to branch lp:zorba. -- Mailing list: https://launchpad.net/~zorba-coders Post to : zorba-coders@lists.launchpad.net Unsubscribe : https://launchpad.net/~zorba-coders More help : https://help.launchpad.net/ListHelp
[Zorba-coders] [Merge] lp:~zorba-coders/zorba/bug-921458 into lp:zorba
The proposal to merge lp:~zorba-coders/zorba/bug-921458 into lp:zorba has been updated. Status: Needs review => Approved For more details, see: https://code.launchpad.net/~zorba-coders/zorba/bug-921458/+merge/102058 -- https://code.launchpad.net/~zorba-coders/zorba/bug-921458/+merge/102058 Your team Zorba Coders is subscribed to branch lp:zorba. -- Mailing list: https://launchpad.net/~zorba-coders Post to : zorba-coders@lists.launchpad.net Unsubscribe : https://launchpad.net/~zorba-coders More help : https://help.launchpad.net/ListHelp
Re: [Zorba-coders] [Merge] lp:~zorba-coders/zorba/bug-921458 into lp:zorba
Review: Approve -- https://code.launchpad.net/~zorba-coders/zorba/bug-921458/+merge/102058 Your team Zorba Coders is subscribed to branch lp:zorba. -- Mailing list: https://launchpad.net/~zorba-coders Post to : zorba-coders@lists.launchpad.net Unsubscribe : https://launchpad.net/~zorba-coders More help : https://help.launchpad.net/ListHelp
Re: [Zorba-coders] [Merge] lp:~zorba-coders/zorba/bug-921458 into lp:zorba
Review: Approve -- https://code.launchpad.net/~zorba-coders/zorba/bug-921458/+merge/102058 Your team Zorba Coders is subscribed to branch lp:zorba. -- Mailing list: https://launchpad.net/~zorba-coders Post to : zorba-coders@lists.launchpad.net Unsubscribe : https://launchpad.net/~zorba-coders More help : https://help.launchpad.net/ListHelp
[Zorba-coders] [Merge] lp:~zorba-coders/zorba/bug-921458 into lp:zorba
The proposal to merge lp:~zorba-coders/zorba/bug-921458 into lp:zorba has been updated. Status: Approved => Needs review For more details, see: https://code.launchpad.net/~zorba-coders/zorba/bug-921458/+merge/102058 -- https://code.launchpad.net/~zorba-coders/zorba/bug-921458/+merge/102058 Your team Zorba Coders is subscribed to branch lp:zorba. -- Mailing list: https://launchpad.net/~zorba-coders Post to : zorba-coders@lists.launchpad.net Unsubscribe : https://launchpad.net/~zorba-coders More help : https://help.launchpad.net/ListHelp
Re: [Zorba-coders] [Merge] lp:~zorba-coders/zorba/bug-921458 into lp:zorba
There are additional revisions which have not been approved in review. Please seek review and approval of these new revisions. -- https://code.launchpad.net/~zorba-coders/zorba/bug-921458/+merge/102058 Your team Zorba Coders is subscribed to branch lp:zorba. -- Mailing list: https://launchpad.net/~zorba-coders Post to : zorba-coders@lists.launchpad.net Unsubscribe : https://launchpad.net/~zorba-coders More help : https://help.launchpad.net/ListHelp
[Zorba-coders] [Merge] lp:~zorba-coders/zorba/bug-921458 into lp:zorba
The proposal to merge lp:~zorba-coders/zorba/bug-921458 into lp:zorba has been updated. Status: Needs review => Approved For more details, see: https://code.launchpad.net/~zorba-coders/zorba/bug-921458/+merge/102058 -- https://code.launchpad.net/~zorba-coders/zorba/bug-921458/+merge/102058 Your team Zorba Coders is subscribed to branch lp:zorba. -- Mailing list: https://launchpad.net/~zorba-coders Post to : zorba-coders@lists.launchpad.net Unsubscribe : https://launchpad.net/~zorba-coders More help : https://help.launchpad.net/ListHelp
[Zorba-coders] [Merge] lp:~zorba-coders/zorba/bug-921458 into lp:zorba
The proposal to merge lp:~zorba-coders/zorba/bug-921458 into lp:zorba has been updated. Commit Message changed to: Fix for bug #921458 (file:read-text-lines() blocking) For more details, see: https://code.launchpad.net/~zorba-coders/zorba/bug-921458/+merge/102058 -- https://code.launchpad.net/~zorba-coders/zorba/bug-921458/+merge/102058 Your team Zorba Coders is subscribed to branch lp:zorba. -- Mailing list: https://launchpad.net/~zorba-coders Post to : zorba-coders@lists.launchpad.net Unsubscribe : https://launchpad.net/~zorba-coders More help : https://help.launchpad.net/ListHelp
[Zorba-coders] [Merge] lp:~zorba-coders/zorba/bug-921458 into lp:zorba
Matthias Brantner has proposed merging lp:~zorba-coders/zorba/bug-921458 into
lp:zorba.
Requested reviews:
William Candillon (wcandillon)
Paul J. Lucas (paul-lucas)
Related bugs:
Bug #921458 in Zorba: "file:read-text-lines() blocking"
https://bugs.launchpad.net/zorba/+bug/921458
For more details, see:
https://code.launchpad.net/~zorba-coders/zorba/bug-921458/+merge/102058
Fix for bug #921458 (file:read-text-lines() blocking)
The problem was that the string:split function isn't suitable for tokenizing
with newlines as separator.
I have provided a native implementation using C++'s getline.
--
https://code.launchpad.net/~zorba-coders/zorba/bug-921458/+merge/102058
Your team Zorba Coders is subscribed to branch lp:zorba.
=== modified file 'ChangeLog'
--- ChangeLog 2012-04-14 06:12:29 +
+++ ChangeLog 2012-04-16 07:08:21 +
@@ -21,6 +21,7 @@
* Fixed bug #872234 (prevent a rewritting to take place in case of sequential expr)
* Fixed bug #906494 (default compile with D_FILE_OFFSET_BITS=64)
* Fixed bug #912586, #912593 and #912722 (assertion failures with lax validation)
+ * Fixed bug #921458 (file:read-text-lines() blocking)
* Fixed bug #949910 (has-children may be invoked on all nodes). Internally, zorba::store::Item::getChildren() now returns NULL on node classes without offspring (instead of raising an error).
=== modified file 'modules/org/expath/ns/file.xq'
--- modules/org/expath/ns/file.xq 2012-04-14 01:36:33 +
+++ modules/org/expath/ns/file.xq 2012-04-16 07:08:21 +
@@ -422,11 +422,7 @@
declare %an:nondeterministic function file:read-text-lines(
$file as xs:string,
$encoding as xs:string
-) as xs:string*
-{
- let $content := file:read-text($file, $encoding)
- return fn:tokenize($content, "\n")
-};
+) as xs:string* external;
(:~
: This is an internal function that copies an entire source directory to an
=== modified file 'modules/org/expath/ns/file.xq.src/file.cpp'
--- modules/org/expath/ns/file.xq.src/file.cpp 2012-03-30 19:03:09 +
+++ modules/org/expath/ns/file.xq.src/file.cpp 2012-04-16 07:08:21 +
@@ -223,6 +223,124 @@
//*
+ReadTextLinesFunction::ReadTextLinesFunction(const FileModule* aModule)
+ : FileFunction(aModule)
+{
+}
+
+ItemSequence_t
+ReadTextLinesFunction::evaluate(
+ const ExternalFunction::Arguments_t& aArgs,
+ const StaticContext* aSctxCtx,
+ const DynamicContext* aDynCtx) const
+{
+ String lFileStr = getFilePathString(aArgs, 0);
+ File_t lFile = File::createFile(lFileStr.c_str());
+ String lEncoding("UTF-8");
+
+ // preconditions
+ if (!lFile->exists()) {
+raiseFileError("FOFL0001", "A file does not exist at this path", lFile->getFilePath());
+ }
+ if (lFile->isDirectory()) {
+raiseFileError("FOFL0004", "The given path points to a directory", lFile->getFilePath());
+ }
+
+ lEncoding = getEncodingArg(aArgs, 1);
+
+ return ItemSequence_t(new LinesItemSequence(lFile, lEncoding, this));
+}
+
+ReadTextLinesFunction::LinesItemSequence::LinesItemSequence(
+const File_t& aFile,
+const String& aEncoding,
+const ReadTextLinesFunction* aFunc)
+ : theFile(aFile),
+theEncoding(aEncoding),
+theFunc(aFunc)
+{
+}
+
+Iterator_t
+ReadTextLinesFunction::LinesItemSequence::getIterator()
+{
+ return new ReadTextLinesFunction::LinesItemSequence::LinesIterator(
+ theFile, theEncoding, theFunc
+);
+}
+
+ReadTextLinesFunction::LinesItemSequence::LinesIterator::LinesIterator(
+const File_t& aFile,
+const String& aEncoding,
+const ReadTextLinesFunction* aFunc)
+ : theFile(aFile),
+theEncoding(aEncoding),
+theFunc(aFunc),
+theStream(0)
+{
+}
+
+ReadTextLinesFunction::LinesItemSequence::LinesIterator::~LinesIterator()
+{
+ delete theStream;
+}
+
+void
+ReadTextLinesFunction::LinesItemSequence::LinesIterator::open()
+{
+ if ( transcode::is_necessary( theEncoding.c_str() ) )
+ {
+try
+{
+ theStream = new transcode::stream(theEncoding.c_str());
+}
+catch (std::invalid_argument const& e)
+{
+ theFunc->raiseFileError("FOFL0006", "Unsupported encoding", theEncoding.c_str());
+}
+ }
+ else
+ {
+theStream = new std::ifstream();
+ }
+ theFile->openInputStream(*theStream, false, true);
+}
+
+bool
+ReadTextLinesFunction::LinesItemSequence::LinesIterator::next(Item& aRes)
+{
+ if (!theStream || !theStream->good())
+return false;
+
+ std::string lStr;
+ getline(*theStream, lStr);
+
+ if (theStream->bad())
+ {
+return false;
+ }
+ else
+ {
+aRes = theFunc->theModule->getItemFactory()->createString(lStr);
+return true;
+ }
+}
+
+void
+ReadTextLinesFunction::LinesItemSequence::LinesIterator::close()
+{
+ delete theStream;
+ theStream = 0;
+}
+
+bool
+ReadTextLinesFunction::LinesItemSequence::LinesIterator::isOpen() const
+{
+ return theStream != 0;
+}
+
+//**
Re: [Zorba-coders] [Merge] lp:~zorba-coders/zorba/bug-921458 into lp:zorba
Review: Needs Fixing -- https://code.launchpad.net/~zorba-coders/zorba/bug-921458/+merge/101873 Your team Zorba Coders is subscribed to branch lp:zorba. -- Mailing list: https://launchpad.net/~zorba-coders Post to : zorba-coders@lists.launchpad.net Unsubscribe : https://launchpad.net/~zorba-coders More help : https://help.launchpad.net/ListHelp
Re: [Zorba-coders] [Merge] lp:~zorba-coders/zorba/bug-921458 into lp:zorba
Required: After your call to getline(), you should check the state of the stream again for badbit. Optional: you ought to use std::unique_ptr for the stream member so you don't have to delete it explicitly. -- https://code.launchpad.net/~zorba-coders/zorba/bug-921458/+merge/101873 Your team Zorba Coders is subscribed to branch lp:zorba. -- Mailing list: https://launchpad.net/~zorba-coders Post to : zorba-coders@lists.launchpad.net Unsubscribe : https://launchpad.net/~zorba-coders More help : https://help.launchpad.net/ListHelp
Re: [Zorba-coders] [Merge] lp:~zorba-coders/zorba/bug-921458 into lp:zorba
Review: Approve Works great. -- https://code.launchpad.net/~zorba-coders/zorba/bug-921458/+merge/101873 Your team Zorba Coders is subscribed to branch lp:zorba. -- Mailing list: https://launchpad.net/~zorba-coders Post to : zorba-coders@lists.launchpad.net Unsubscribe : https://launchpad.net/~zorba-coders More help : https://help.launchpad.net/ListHelp
[Zorba-coders] [Merge] lp:~zorba-coders/zorba/bug-921458 into lp:zorba
The proposal to merge lp:~zorba-coders/zorba/bug-921458 into lp:zorba has been updated. Status: Approved => Needs review For more details, see: https://code.launchpad.net/~zorba-coders/zorba/bug-921458/+merge/101873 -- https://code.launchpad.net/~zorba-coders/zorba/bug-921458/+merge/101873 Your team Zorba Coders is subscribed to branch lp:zorba. -- Mailing list: https://launchpad.net/~zorba-coders Post to : zorba-coders@lists.launchpad.net Unsubscribe : https://launchpad.net/~zorba-coders More help : https://help.launchpad.net/ListHelp
Re: [Zorba-coders] [Merge] lp:~zorba-coders/zorba/bug-921458 into lp:zorba
Voting does not meet specified criteria. Required: Approve > 1, Disapprove < 1, Needs Fixing < 1, Pending < 1. Got: 2 Pending. -- https://code.launchpad.net/~zorba-coders/zorba/bug-921458/+merge/101873 Your team Zorba Coders is subscribed to branch lp:zorba. -- Mailing list: https://launchpad.net/~zorba-coders Post to : zorba-coders@lists.launchpad.net Unsubscribe : https://launchpad.net/~zorba-coders More help : https://help.launchpad.net/ListHelp
[Zorba-coders] [Merge] lp:~zorba-coders/zorba/bug-921458 into lp:zorba
Validation queue job bug-921458-2012-04-13T09-53-08.237Z is finished. The final status was: All tests succeeded! -- https://code.launchpad.net/~zorba-coders/zorba/bug-921458/+merge/101873 Your team Zorba Coders is subscribed to branch lp:zorba. -- Mailing list: https://launchpad.net/~zorba-coders Post to : zorba-coders@lists.launchpad.net Unsubscribe : https://launchpad.net/~zorba-coders More help : https://help.launchpad.net/ListHelp
[Zorba-coders] [Merge] lp:~zorba-coders/zorba/bug-921458 into lp:zorba
Validation queue starting for merge proposal. Log at: http://zorbatest.lambda.nu:8080/remotequeue/bug-921458-2012-04-13T09-53-08.237Z/log.html -- https://code.launchpad.net/~zorba-coders/zorba/bug-921458/+merge/101873 Your team Zorba Coders is subscribed to branch lp:zorba. -- Mailing list: https://launchpad.net/~zorba-coders Post to : zorba-coders@lists.launchpad.net Unsubscribe : https://launchpad.net/~zorba-coders More help : https://help.launchpad.net/ListHelp
[Zorba-coders] [Merge] lp:~zorba-coders/zorba/bug-921458 into lp:zorba
The proposal to merge lp:~zorba-coders/zorba/bug-921458 into lp:zorba has been updated. Status: Needs review => Approved For more details, see: https://code.launchpad.net/~zorba-coders/zorba/bug-921458/+merge/101873 -- https://code.launchpad.net/~zorba-coders/zorba/bug-921458/+merge/101873 Your team Zorba Coders is subscribed to branch lp:zorba. -- Mailing list: https://launchpad.net/~zorba-coders Post to : zorba-coders@lists.launchpad.net Unsubscribe : https://launchpad.net/~zorba-coders More help : https://help.launchpad.net/ListHelp
[Zorba-coders] [Merge] lp:~zorba-coders/zorba/bug-921458 into lp:zorba
Matthias Brantner has proposed merging lp:~zorba-coders/zorba/bug-921458 into
lp:zorba.
Requested reviews:
Paul J. Lucas (paul-lucas)
William Candillon (wcandillon)
Related bugs:
Bug #921458 in Zorba: "file:read-text-lines() blocking"
https://bugs.launchpad.net/zorba/+bug/921458
For more details, see:
https://code.launchpad.net/~zorba-coders/zorba/bug-921458/+merge/101873
Fix for bug #921458 (file:read-text-lines() blocking)
The problem was that the string:split function isn't suitable for tokenizing
with newlines as separator.
I have provided a native implementation using C++'s getline.
--
https://code.launchpad.net/~zorba-coders/zorba/bug-921458/+merge/101873
Your team Zorba Coders is subscribed to branch lp:zorba.
=== modified file 'ChangeLog'
--- ChangeLog 2012-04-12 09:32:55 +
+++ ChangeLog 2012-04-13 09:06:22 +
@@ -18,6 +18,7 @@
* Fixed bug #872234 (prevent a rewritting to take place in case of sequential expr)
* Fixed bug #906494 (default compile with D_FILE_OFFSET_BITS=64)
* Fixed bug #912586, #912593 and #912722 (assertion failures with lax validation)
+ * Fixed bug #921458 (file:read-text-lines() blocking)
* Fixed bug #949910 (has-children may be invoked on all nodes). Internally, zorba::store::Item::getChildren() now returns NULL on node classes without offspring (instead of raising an error).
=== modified file 'modules/org/expath/ns/file.xq'
--- modules/org/expath/ns/file.xq 2012-04-11 09:14:41 +
+++ modules/org/expath/ns/file.xq 2012-04-13 09:06:22 +
@@ -422,11 +422,7 @@
declare %an:nondeterministic function file:read-text-lines(
$file as xs:string,
$encoding as xs:string
-) as xs:string*
-{
- let $content := file:read-text($file, $encoding)
- return fn:tokenize($content, "\n")
-};
+) as xs:string* external;
(:~
: This is an internal function that copies an entire source directory to an
=== modified file 'modules/org/expath/ns/file.xq.src/file.cpp'
--- modules/org/expath/ns/file.xq.src/file.cpp 2012-03-28 05:19:57 +
+++ modules/org/expath/ns/file.xq.src/file.cpp 2012-04-13 09:06:22 +
@@ -223,6 +223,118 @@
//*
+ReadTextLinesFunction::ReadTextLinesFunction(const FileModule* aModule)
+ : FileFunction(aModule)
+{
+}
+
+ItemSequence_t
+ReadTextLinesFunction::evaluate(
+ const ExternalFunction::Arguments_t& aArgs,
+ const StaticContext* aSctxCtx,
+ const DynamicContext* aDynCtx) const
+{
+ String lFileStr = getFilePathString(aArgs, 0);
+ File_t lFile = File::createFile(lFileStr.c_str());
+ String lEncoding("UTF-8");
+
+ // preconditions
+ if (!lFile->exists()) {
+raiseFileError("FOFL0001", "A file does not exist at this path", lFile->getFilePath());
+ }
+ if (lFile->isDirectory()) {
+raiseFileError("FOFL0004", "The given path points to a directory", lFile->getFilePath());
+ }
+
+ lEncoding = getEncodingArg(aArgs, 1);
+
+ return ItemSequence_t(new LinesItemSequence(lFile, lEncoding, this));
+}
+
+ReadTextLinesFunction::LinesItemSequence::LinesItemSequence(
+const File_t& aFile,
+const String& aEncoding,
+const ReadTextLinesFunction* aFunc)
+ : theFile(aFile),
+theEncoding(aEncoding),
+theFunc(aFunc)
+{
+}
+
+Iterator_t
+ReadTextLinesFunction::LinesItemSequence::getIterator()
+{
+ return new ReadTextLinesFunction::LinesItemSequence::LinesIterator(
+ theFile, theEncoding, theFunc
+);
+}
+
+ReadTextLinesFunction::LinesItemSequence::LinesIterator::LinesIterator(
+const File_t& aFile,
+const String& aEncoding,
+const ReadTextLinesFunction* aFunc)
+ : theFile(aFile),
+theEncoding(aEncoding),
+theFunc(aFunc),
+theStream(0)
+{
+}
+
+ReadTextLinesFunction::LinesItemSequence::LinesIterator::~LinesIterator()
+{
+ delete theStream;
+}
+
+void
+ReadTextLinesFunction::LinesItemSequence::LinesIterator::open()
+{
+ if ( transcode::is_necessary( theEncoding.c_str() ) )
+ {
+try
+{
+ theStream = new transcode::stream(theEncoding.c_str());
+}
+catch (std::invalid_argument const& e)
+{
+ theFunc->raiseFileError("FOFL0006", "Unsupported encoding", theEncoding.c_str());
+}
+ }
+ else
+ {
+theStream = new std::ifstream();
+ }
+ theFile->openInputStream(*theStream, false, true);
+}
+
+bool
+ReadTextLinesFunction::LinesItemSequence::LinesIterator::next(Item& aRes)
+{
+ if (!theStream || !theStream->good())
+return false;
+
+ std::string lStr;
+ getline(*theStream, lStr);
+
+ aRes = theFunc->theModule->getItemFactory()->createString(lStr);
+
+ return true;
+}
+
+void
+ReadTextLinesFunction::LinesItemSequence::LinesIterator::close()
+{
+ delete theStream;
+ theStream = 0;
+}
+
+bool
+ReadTextLinesFunction::LinesItemSequence::LinesIterator::isOpen() const
+{
+ return theStream != 0;
+}
+
+//*
+
ExistsFunction::ExistsFun
