[Oorexx-devel] Playing with the stream library in Rick's sandbox

2008-06-19 Thread Mark Miesfeld
This is mostly for Rick, posted here for other's that may be interested.

I worked my way through the new stream library about as far as I can go.  I
have a list of the things I found for when you get back to it.

Hopefully, each one will save you a few minutes of investigation.  grin

Many of the places I wasn't really sure of the proper fix, I just fixed it
enough to continue.

I was using the m_oodcls.rex, trying to get it complete.  The list is sort
of in the order I found them.

SysFile::flush()  There is the comment about invalidating the buffer, but no
code.

// update the real output position
filePointer += written;
// and invalidate the buffer
I added:
bufferPosition = 0;
bufferedInput = 0;

SysFile::getStreamTypeInfo  there is this test:

if ((fileInfo.st_mode  _S_IFREG) != 0)
{
device = true;
transient = true;
}
which marks persistent streams as transient.  I think you meant it to be
this test:

if ((fileInfo.st_mode  _S_IFCHR) != 0)
{
device = true;
transient = true;
}
Well, now I forget the order.

StreamInfo::checkEof()   There is this code:

  // if this is an eof condition, raise that not ready
if (fileInfo.atEof())
{
eof();
}
else
{
// must be an error, so raise that using the file error information
notreadyError();
}
which always generates an error and causes setPosition() to always fail.  I
took out the else.

StreamInfo::setPosition()  The position value is decremented, than also has
1 subtracted from it in the call to seek().

position--;  // convert to system position type
// seek to the target position, if possible.  The request position
// is a 1-based character number.  We need to convert this into
// a zero-based one before moving.
if (fileInfo.seek(position - 1, SEEK_SET, newPosition))
{
checkEof();
}
I just took out the position--; line

With the rest, I'm even less sure what a good fix is.

bool SysFile::getChar()

In several places in with loops using getChar(), the check is done for a
return of false to fall out of the loop.  But getChar() returns true when it
hits end of file.

 bool SysFile::getChar(char ch)
 {
size_t len;

return read(ch, 1, len);
 }
read() returns true at eof, and looks like it is intended to return true for
that condition.  So, I changed getChar() to only return true if it actually
got a new char:

 bool SysFile::getChar(char ch)
 {
size_t len = 0;

return (read(ch, 1, len)  len == 1);
 }

Then there is a basic problem with ~linein() in that the position variables
don't get updated so that linein() in a loop returns the first line over and
over.  This one I really kind of hacked.  Since the code path I was
producing was going through StreamInfo::readVariableLine(), I did the
position update there.

// If we have a new line character in the last position, we have
// a line.  The gets() function has translated crlf sequences into
// single lf characters.
if (buffer[bytesRead - 1] == '\n')
{
charReadPosition += currentLength + bytesRead + (collapsed ? 1 :
0);
lineReadPosition++;
lineReadCharPosition = charReadPosition;
last_op_was_read = true;
return context-NewString(buffer, currentLength + bytesRead -
1);
}

However, since gets() may have collapsed a /r/n into a single byte, I added
in an extra arg that would report if /r/n was collapsed into /n.

fileInfo.gets(readPosition, bufferSize - currentLength, bytesRead,
collapsed)

Not pretty, but it worked.  I really wasn't sure what you intended to do
with reads.

The last thing is eof checking when using buffered reads.  With a file where
the last bytes only partially fills the buffer, the file is at eof, but
there are still bytes in the buffer that need to be processed.

// hit end of file reading this?  This will be the entire line then
if (fileInfo.atEof())
{
...
}

The last read may have read 300 bytes into the buffer.  _eof() will return
true, so atEof() returns true.  But this line is not really the last line.
I hacked this into:

// hit end of file reading this?  This will be the entire line then
if (fileInfo.atEof()  !fileInfo.hasBufferedInput())
{
charReadPosition += currentLength + bytesRead + 1;
lineReadPosition++;
lineReadCharPosition = charReadPosition;
last_op_was_read = true;
return context-NewString(bufferAddress, currentLength +
bytesRead);
}

where I added to SysFile

inline bool hasBufferedInput() { return bufferedInput  bufferPosition;
}

As I said, I didn't mean these to be the correct fixes, just to point out
the areas that need something.

All of the above got m_oodcls.rex working 

Re: [Oorexx-devel] Playing with the stream library in Rick's sandbox

2008-06-19 Thread Rick McGuire
Mark,

If you get a chance, any changes to the Windows version of SysFile
might also be needed in the unix version as well.  I've not gotten to
the point of debugging anything on linux yet, but no sense in
debugging some of these problems twice.

Rick

On Thu, Jun 19, 2008 at 12:34 PM, Mark Miesfeld [EMAIL PROTECTED] wrote:

 On Thu, Jun 19, 2008 at 8:07 AM, Rick McGuire [EMAIL PROTECTED] wrote:

 See my comments below.

 On Thu, Jun 19, 2008 at 10:08 AM, Mark Miesfeld [EMAIL PROTECTED]
 wrote:
  ...   I just took out the position--; line
 

 All of those look reasonable.  Thanks for digging in to this!  Go
 ahead and check these in.

 Okay, I will.  grin


  read() returns true at eof, and looks like it is intended to return true
  for
  that condition.  So, I changed getChar() to only return true if it
  actually
  got a new char:

 Well, the problem is in read.  Those functions are supposed to return
 true if they read data, and false if they didn't return data for any
 reason.

 I'll see about fixing that in read then.


  I had similar problems with linein() trying to get Rexxtry to
 work.  I did a quick search for other places where I'd made the
 mistake, but obviously I missed at least one.

  However, since gets() may have collapsed a /r/n into a single byte, I
  added
  in an extra arg that would report if /r/n was collapsed into /n.

 Hmmmprobably the appropriate thing would be to ask the SysFile
 object what the current read position is rather than trying to infer
 it from the length that was returned.

 Yes, I redid that this morning to use SysFile.getPosition() and cleaned it
 up.


  So it looks like the bufferAddress member of StreamInfo has gotten
  overwritten.  I still get baffled trying to figure those out, so that's
  where I quit.

 These are actually pretty easy to figure out in Visual Studio.  Set a
 break point at the place where the buffer pointer is allocated, then
 set a data breakpoing on this-bufferAddress.  This will catch the
 overlay for you when it happens.

 I'll try that, again.  You've pointed out this technique to me before.
 grin   I think the problem I had this time was that I was trying to set
 the data breakpoint directly on bufferAddress and couldn't get the debugger
 to resolve it.  I didn't think to use 'this'.


 This is terrific progress!  Thanks for taking the time to look at
 these.  Don't be too hesitant about checking in fixes for these
 problems in the future.  If you have, a fix, and it fixes the problem,
 go ahead and commit it.  If you're not certain about the fix, ask for
 a review of the commit.  If there's another course that should be
 taken, I can correct it there.

 I'm slowly getting less shy.

 --
 Mark Miesfeld

 -
 Check out the new SourceForge.net Marketplace.
 It's the best place to buy or sell services for
 just about anything Open Source.
 http://sourceforge.net/services/buy/index.php
 ___
 Oorexx-devel mailing list
 Oorexx-devel@lists.sourceforge.net
 https://lists.sourceforge.net/lists/listinfo/oorexx-devel



-
Check out the new SourceForge.net Marketplace.
It's the best place to buy or sell services for
just about anything Open Source.
http://sourceforge.net/services/buy/index.php
___
Oorexx-devel mailing list
Oorexx-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/oorexx-devel


Re: [Oorexx-devel] Playing with the stream library in Rick's sandbox

2008-06-19 Thread Mark Miesfeld
On Thu, Jun 19, 2008 at 11:45 AM, Rick McGuire [EMAIL PROTECTED]
wrote:

 Mark,

 If you get a chance, any changes to the Windows version of SysFile
 might also be needed in the unix version as well.  I've not gotten to
 the point of debugging anything on linux yet, but no sense in
 debugging some of these problems twice.


Okay, I'll look at the unix version.

--
Mark Miesfeld
-
Check out the new SourceForge.net Marketplace.
It's the best place to buy or sell services for
just about anything Open Source.
http://sourceforge.net/services/buy/index.php___
Oorexx-devel mailing list
Oorexx-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/oorexx-devel