Re: [Metakit] c4_Bytes.Modify() mangling data

[artost]

--- Brian Kelley [EMAIL PROTECTED] wrote:
 It looks like you are adding the data at the end of
 the buffer.  If you make the following substitution
 
 pos = random.randrange(len(v[0].b)-1)
 
 then everything seems to work.  I'm not sure why
 this is but jcw might have a pointer.

Hmmm, it dosn't seem to make any difference for me. Do
you only see the problem come up when the data is
added
to the end (pos == len)?

Which version of metakit did you test it with?

Best Regards,
  Arto Stimms




__
Do you Yahoo!?
Yahoo! Mail - Helps protect you from nasty viruses.
http://promotions.yahoo.com/new_mail
_
Metakit mailing list  -  [EMAIL PROTECTED]
http://www.equi4.com/mailman/listinfo/metakit

Re: [Metakit] c4_Bytes.Modify() mangling data

[Brian Kelley]

I was inserting strings of length 2 which was why it worked for me. 
Yours were larger.  It turns out that you can't cross the end boundary
when using modify.  So if you are inserting a string of length 10, you
can't insert it into a string of length 9.  Also, if you are inserting
it into a string of length 12, you can only insert at 0 or 1!

Using this, I wrote a safe modify function that handles this boundary
case.  Let me know if it works for you:


import Mk4py
import random

s = Mk4py.storage()
v = s.getas(v[b:B])
p = Mk4py.property(B, b)
v.append()
#
# view.modify(byteprop, rownum, string, offset, diff)
#  Store (partial) byte property contents. A non-zero value of diff
removes (0) or inserts (0) bytes.

def modify(v, property, rownum, string, offset, diff):
# safe modify
val = getattr(v[rownum], property.name)
l = len(val)
ls = len(string)
if  l  ls and offset  l-ls:
return v.modify(property, rownum, string, offset, diff)

while string:
chunksize = max(l-offset, 1)
s1, string = string[:chunksize], string[chunksize:]
v.modify(property, rownum, s1, offset, len(s1))
offset += chunksize

pystring = v[0].b = test

for i in range(2000):
   # Create a random range string
   x = random.randrange(10)
   randstring = str(x)*x

   # insert it at a random position in the text
   pos = max(1, random.randrange(len(v[0].b)) - len(randstring) + 2)
   #v.modify(p, 0, randstring, pos, len(randstring))
   modify(v, p, 0, randstring, pos, len(randstring))
   # Keep a matching python string
   pystring = pystring[:pos] + randstring + pystring[pos:]

   if pystring != v[0].b:
   print v[0].b
   print pystring
   print len(randstring)
   raise ERROR: string mismatch
_
Metakit mailing list  -  [EMAIL PROTECTED]
http://www.equi4.com/mailman/listinfo/metakit

Re: [Metakit] c4_Bytes.Modify() mangling data

[Jean-Claude Wippler]

Brian Kelley wrote:
I was inserting strings of length 2 which was why it worked for me.
Yours were larger.  It turns out that you can't cross the end boundary
when using modify.  So if you are inserting a string of length 10, you
can't insert it into a string of length 9.  Also, if you are inserting
it into a string of length 12, you can only insert at 0 or 1!
Whoa.  Good catch and analysis!
The attached change ought to fix this issue.  I'll verify this later, 
but it'll let you proceed for now (or you can use the Python 
workaround, of course).

-jcw
Index: viewx.cpp
===
RCS file: /home/cvs/metakit/src/viewx.cpp,v
retrieving revision 1.11
diff -u -p -u -r1.11 viewx.cpp
--- viewx.cpp   23 Nov 2003 01:42:51 -  1.11
+++ viewx.cpp   23 Sep 2004 17:49:15 -
@@ -581,21 +581,15 @@ bool c4_BytesRef::Modify(const c4_Bytes
 c4_Handler h = _cursor._seq-NthHandler(colNum);
 const int n = buf_.Size();
 const t4_i32 limit = off_ + n; // past changed bytes
-const t4_i32 overshoot = limit - h.ItemSize(_cursor._index);
-
-if (diff_  overshoot)
-  diff_ = overshoot;
+  // get rid of an optimization, it was wrong  (2004-09-23)
 c4_Column* col = h.GetNthMemoCol(_cursor._index, true);
 if (col != 0)
 {
   if (diff_  0)
 col-Shrink(limit, - diff_);
   else if (diff_  0)
-  // insert bytes in the highest possible spot
-  // if a gap is created, it will contain garbage
-col-Grow(overshoot  0 ? col-ColSize() :
-   diff_  n ? off_ : limit - diff_, diff_);
+col-Grow(off_, diff_);
   col-StoreBytes(off_, buf_);
 }
_
Metakit mailing list  -  [EMAIL PROTECTED]
http://www.equi4.com/mailman/listinfo/metakit