Re: [PHP-DEV] bug in zend_API.c (with patch)

2003-03-07 Thread Eric Lambart 
On Fri, 2003-03-07 at 14:24, Brad LaFountain wrote:
 Why can't you just simply name your classes all lowercase.

Because then I wouldn't have discovered the bug! =)

 All of the extensions currently do this and it works fine.

Apparently that is the case--that's probably why I had the pleasure of
discovering this bug myself.  Although we are not currently developing
this extension for public release, it may be open-sourced in the future,
and for that reason I have followed the CODING_STANDARDS document fairly
closely.

We could, of course, give all classes and functions lower case names,
but as I explained in my previous post, we are porting an existing
application that exists in C++ and Java and the developers of those
original versions chose to use CamelCaps.  AFAIC, the less changes
needed to the code, the better.

Personally, I'd rather have case-sensitive class and function names in
PHP, but the whole point is, if the Zend engine is going to try and look
them up them in lowercase, IMHO it'd be really nice if it'd store it the
same way :)

BTW, since my first post I have discovered the  README.SUBMITTING_PATCH
file.  I still don't have the time now to deal with CVS, though if
pushed I will do so, sometime in the future.  Nonetheless, I am
re-submitting the patch here as requested in the form of a MIME
attachment.  FWIW I also performed a make test with no FAILs, though I
did not create a new test case for this bug.

Let me know if there's anything else I can do to help.  If this patch
will not be accepted SOLEY due to the fact that it's not done on CVS,
please let me know and I will take care of that when I have time, and
resubmit it.

For now, I have spent enough time on this problem already.

Regards,
Eric

--- Zend/OLD_zend_API.c Wed Oct  9 07:17:52 2002
+++ Zend/zend_API.c Fri Mar  7 14:49:36 2003
@@ -1039,6 +1039,12 @@
internal_function-type = ZEND_INTERNAL_FUNCTION;

while (ptr-fname) {
+   /* store all function names in lower case so they will always be found 
by
+* call_user_function_ex() */
+   size_t fname_len = strlen(ptr-fname);
+   char *lowercase_name = zend_strndup(ptr-fname, fname_len);
+   zend_str_tolower(lowercase_name, fname_len);
+
internal_function-handler = ptr-handler;
internal_function-arg_types = ptr-func_arg_types;
internal_function-function_name = ptr-fname;
@@ -1047,7 +1053,7 @@
zend_unregister_functions(functions, count, 
target_function_table TSRMLS_CC);
return FAILURE;
}
-   if (zend_hash_add(target_function_table, ptr-fname, 
strlen(ptr-fname)+1, function, sizeof(zend_function), NULL) == FAILURE) {
+   if (zend_hash_add(target_function_table, lowercase_name, fname_len+1, 
function, sizeof(zend_function), NULL) == FAILURE) {
unload=1;
break;
}

-- 
PHP Development Mailing List http://www.php.net/
To unsubscribe, visit: http://www.php.net/unsub.php

Re: [PHP-DEV] repost: return_value getting corrupted in extension

2003-02-14 Thread Eric Lambart 
On Thu, 13 Feb 2003 09:02:44 -0800, Eric Lambart wrote:

 Second of all, I have narrowed the problem down a bit.  It has nothing
 to do with parentheses or single-quotes (surprise!).  Seriously... I
 just observed it mangling another string, but what seemed stranger yet,
 it changed \037N^VESOBJ(4) to \037N^VESOBJ(3) (this is output from
 gdb, so that's ASCII octal 37, which isn't a printable character and
 didn't show up in my previous console output).
 
 So what does this have in common with changing \037N^VESoDMN(107) to
 \037N^VESoDMN'107)?  Simple: it is the 11th character of the string
 that is getting changed.  Moreover, it is getting decremented by one.
 Just as 3 precedes 4 in the ASCII table, ' precedes ).  These strings
 that are getting mangled are not the same variables... not the same
 zvals or memory locations or anything.

OK, another update, although I fear I am just adding noise to the list.
Not sure if anyone is reading this thread!

Further investigation on my part has revealed a workaround which fixes
this ONE problem:  if I use erealloc() to make my string just ONE byte
longer, the string does not get corrupted.

I first suspected this had nothing to do with actually extending the 
string, but that the pointer was getting assigned to a completely new
address; but I have disproved that by printing the pointer address before
and after the erealloc(), which can remain unchanged and the problem
nevertheless goes away.

If this solved the problem, this would be an acceptable workaround for me
at this point.  But the problem of the other object field getting
corrupted remains:

php_var_dump() from C:
object(eo_table)(10) {
  [serverId]=
  string(15) N^VESoDMN(192)
  [rows]=
  int(0)
  [cols]=
  int(0)
  [namedCols]=
  bool(false)
  [errorText]=
  string(0) 
  [errorItem]=
  string(0) 
  [errorId]=
  int(0)
  [flags]=
  int(0)
  [colNames]=
  NULL
  [table]=
  NULL
}

var_dump() from PHP script immediately afterwards:
object(eo_table)(10) {
  [serverId]=
  string(15) N^VESoDMN(192)
  [rows]=
  UNKNOWN:0
  [cols]=
  int(0)
  [namedCols]=
  bool(false)
  [errorText]=
  string(0) 
  [errorItem]=
  string(0) 
  [errorId]=
  int(0)
  [flags]=
  int(0)
  [colNames]=
  NULL
  [table]=
  NULL
}

As you can see, the rows field is becoming undefined.  Again, here I
have found another clue.
If I repeat the add_property_long() for rows, like this:
add_property_long(return_value, rows, rows);
add_property_long(return_value, rows, rows);

rows is OK, but the UNKNOWN:0 moves to the next field (cols)

If I repeat both, exactly like this,
add_property_long(return_value, rows, rows);
add_property_long(return_value, rows, rows);
add_property_long(return_value, cols, cols);
add_property_long(return_value, cols, cols);

...no data is getting corrupted... AT THE MOMENT.

I don't trust this.  I shouldn't have to trust this.  It's not the
ugliest hack, but it's very much a hack.  I really think something in 
either zend_hash.c is mishandling the object's hash table...
or in the case of the string getting corrupted, perhaps it's something in
zend_alloc.c or somewhere within the memory manager/garbage collector.

At this point, can anyone give me any reason to doubt that the problem
lies within the Zend engine?  Please refer to my earlier posts on this
thread for more (though less-informed and -investigated) details.

This is all very strange to me, ESPECIALLY the 11th-character thing.

Again, I am compiling this extension into PHP 4.3.0.
gcc version 2.96 2731 (Red Hat Linux 7.2 2.96-112.7.1)
Intel P4 CPU

Thanks for your time--and for Wez and others on the dev team, thanks for
your work on Zend/PHP!

Eric

-- 
PHP Development Mailing List http://www.php.net/
To unsubscribe, visit: http://www.php.net/unsub.php

Re: [PHP-DEV] repost: return_value getting corrupted in extension

2003-02-13 Thread Eric Lambart 
On Wed, 12 Feb 2003 13:40:09 -0800, Eric Lambart wrote:

UPDATE...
If you ask me, this is stranger still.  If you recall from my original
post--

 I am trying to return a serverId object member which contains a value
 such as N^VESoDMN(107).  Again, right before my PHP_FUNCTION returns
 control to the script, this value is perfect.  When I DON'T use
 --enable-debug, and the serverId is NOT getting completely clobbered,
 the value that is returned by $table-serverId is not N^VESoDMN(107).
 It is N^VESoDMN'107)

 This is not a random thing.  It is very consistent--I have several
 PHP_FUNCTIONs that return serverIds, either with RETVAL_STRING or as a
 member of a more complex object, and server IDs are always in a similar
 format, a string ending with a parenthesized integer.  Yet EVERY time,
 the ( is replaced by a single quote.

OK, here's an update.  I was not entirely correct about the ( becoming '
being some sort of simple character substitution.

First of all, I have recompiled my extension under PHP 4.3.0 and the
problem persists; exactly the same thing happens.  It is therefore not an 
issue with 4.2.3.

I am no longer using --enable-debug since it was not revealing any memory 
leaks anyway, and the problem with the single-character substitution seemed a
little less abstract than the strings getting completely clobbered, as
with debug enabled.

Second of all, I have narrowed the problem down a bit.  It has nothing to
do with parentheses or single-quotes (surprise!).  Seriously... I just
observed it mangling another string, but what seemed stranger yet, it
changed \037N^VESOBJ(4) to \037N^VESOBJ(3) (this is output from gdb,
so that's ASCII octal 37, which isn't a printable character and
didn't show up in my previous console output).

So what does this have in common with changing \037N^VESoDMN(107) to
\037N^VESoDMN'107)?  Simple: it is the 11th character of the string that
is getting changed.  Moreover, it is getting decremented by one.  Just as 3
precedes 4 in the ASCII table, ' precedes ).  These strings that are
getting mangled are not the same variables... not the same zvals or memory
locations or anything.

And in the case of 4 becoming 3, the circumstances are completely
different. It has nothing (at least directly) to do with my extension 
functions.  The only thing they have in common is the location: 
((my zval*)-value-str.val) + 11

Check out this code from my test script:

  $arrayClass = eo_invoke_method($conn, $objPtr, $method, 0, $params);
  echo arrayClass='$arrayClass'\n;
  $method = Interface;
  echo arrayClass='$arrayClass'\n;
  $params = array();
  echo arrayClass='$arrayClass'\n;

Which produces this console output:

arrayClass='N^VESOBJ(4)'
arrayClass='N^VESOBJ(4)'
arrayClass='N^VESOBJ(3)'

Now to me, this is really baffling.  Right before my variable changes, I am 
initializing the variable $params to be a new array() object (fwiw, $params 
already _was_ an array() containing one value; I am just re-using the variable).
So, in the process of initializing a standard PHP array(), something in the 
Zend engine is mangling my seemingly unrelated variable, $arrayClass... and I 
swear I'm not telling it to!  $arrayClass, coincidentally, refers to an object 
stored on a remote database server, and never contains, nor is ever contained 
IN, a PHP array.  It is just a simple PHP string.

Running trusty (?) old gdb, I set read and write watches on 
$arrayClass-value-str.val, but can't seem to catch where it's changing.  
The read watch is only triggered by PHP processing each echo statement which 
displays $arrayClass's value, the write watchpoint is never caught.

But what could be doing this?  Does the number 11 hold any special significance?
And why is the 11th character being decremented?  Could Zend be trying to 
decrement the refcount for something and pointing to the wrong part of memory?  
I suppose it could be decrementing just about anything, but why the 11th 
character of several different string-containing zvals?  This is obviously 
not completely random.

Please help me before I lose what's left of my mind!

TIA
Eric

-- 
PHP Development Mailing List http://www.php.net/
To unsubscribe, visit: http://www.php.net/unsub.php

Re: [PHP-DEV] repost: return_value getting corrupted in extension

2003-02-12 Thread Eric Lambart 
On Tue, 2003-02-11 at 16:00, Wez Furlong wrote: 
 No offense, but this is really quite a useless problem report.
 You are not showing any of the zend API that you use to create the
 object, so how can we help you?

No offense taken.  I hoped that showing the php_var_dump() would be
evidence enough that things are being properly initialized, and didn't
want to post such a long question that no one would bother to read it. 

It seemed to me that if I followed the instructions (as in the very slim
and out-of-date zend API docs, and with the help of those who have
posted comments therein) to initialize and setup the desired
return_value, that it should be returned to my script.

Here, then, is how I am initializing the given data.  This is the object
constructor.  You will see below how it is used.

/* {{{ eo_table class declaration */

/* zend_function entry list resides in eo_gateway.c */

PHP_FUNCTION(eo_table)
{
  /* define and initialize the object members */
  printf(\tin eo_table constructor now!\n);
  add_property_string(this_ptr, serverId, , TRUE);

  add_property_long(this_ptr, rows, 0);
  add_property_long(this_ptr, cols, 0);

  add_property_bool(this_ptr, namedCols, FALSE);

  add_property_string(this_ptr, errorText, , TRUE);
  add_property_string(this_ptr, errorItem, , TRUE);
  add_property_long(this_ptr, errorId, 0);

  add_property_long(this_ptr, flags, 0);

  add_property_null(this_ptr, colNames);
  add_property_null(this_ptr, table);
} /* end eo_table */

/* }}} */

Here are the bits from eo_gateway.c.  I hope I've pasted them all:

zend_class_entry *eo_table_class_entry_ptr_global;

static zend_class_entry *eo_table_class_entry_ptr;

static zend_function_entry eo_table_class_functions[] =
{
  ZEND_FE(eo_table, NULL)
  ZEND_FE(eo_table_error_dump, NULL)
  ZEND_FALIAS(error_dump, eo_table_error_dump, NULL)
  {NULL, NULL, NULL}
};

/* {{{ PHP_MINIT_FUNCTION  */
PHP_MINIT_FUNCTION(eo_gateway)
{
  zend_class_entry eo_table_class_entry;

  INIT_CLASS_ENTRY(eo_table_class_entry, eo_table, eo_table_class_functions);

  eo_table_class_entry_ptr =
zend_register_internal_class(eo_table_class_entry TSRMLS_CC);

  return SUCCESS;
} /* end PHP_MINIT_FUNCTION */
/* }}} */

/* {{{ PHP_RINIT_FUNCTION
 */
PHP_RINIT_FUNCTION(eo_gateway)
{
  eo_table_class_entry_ptr_global = eo_table_class_entry_ptr;
  return SUCCESS;
}
/* }}} */

In the source file in which I am actually creating and intializing my 
object, I have declared:

extern zend_class_entry *eo_table_class_entry_ptr_global;

Here, within a function called getData(), which is called by the
getDataReply() function mentioned in my initial post, we create the
object:

/* initialize return_value to be a table object */
printf(\n\tCreating the return_value object (eo_table)\n);
object_init_ex(return_value, eo_table_class_entry_ptr_global);

/* call the table constructor */
call_object_constructor(return_value, eo_table);
php_var_dump(return_value, 1 TSRMLS_CC);

Here, from another source file, is the definition of
call_object_constructor():

/* {{{ void call_object_constructor() */
void call_object_constructor(zval* object, char* object_name)
{
  /* constructor function name */
  zval* ctor_fn;
  /* constructor return value (required but not used) */
  zval* ctor_retval;

  printf(\tcalling constructor for: '%s'\n, object_name);

  /* specify the constructor/object name  */
  MAKE_STD_ZVAL(ctor_fn);
  ZVAL_STRING(ctor_fn, object_name, TRUE);

  MAKE_STD_ZVAL(ctor_retval);

  /* call the constructor */
  if ( call_user_function(NULL, object, ctor_fn, ctor_retval, 0, NULL)
  != SUCCESS )
  {
php_error(E_ERROR, Could not call object constructor '%s', object_name);
  }

  printf(\tctor_retval is %s\n, zend_zval_type_name(ctor_retval));
  /* clean up! */
  zval_dtor(ctor_fn);
  efree(ctor_fn);
  efree(ctor_retval);

} // end call_object_constructor()
/* }}} */

I believe that is all the relevant code, and though I have removed some
stuff for clarity and privacy, none of it directly affects any Zend
objects or variables.

 Hints: make sure that your zvals are correctly initialized (there are
 some big differences between ALLOC_ZVAL and MAKE_STD_ZVAL).

One resets the refcount and the is_ref flag, the other doesn't.  All the
variables I'm dealing with are fresh, not references to pre-existing
zvals.

Note also that the variables which are getting corrupted are simple
strings, not even zvals.  Here is how I'm setting the value of serverId,
which is one of those getting corrupted:

add_property_string(return_value, serverId, serverId, TRUE);
efree(serverId);

I don't believe I'm ever running MAKE_STD_ZVAL on return_value, but I
don't think the examples didn't do that either.  It's already
initialized by default, correct?

 Other things to try are looking at other extensions that create objects.
 In the 4.3 release, the user-space streams creates objects and call their
 constructors, so take a look at

Re: [PHP-DEV] repost: return_value getting corrupted in extension

2003-02-12 Thread Eric Lambart 
Wez Furlong said:

   /* call the constructor */
   if ( call_user_function(NULL, object, ctor_fn, ctor_retval, 0,
 NULL)

 Looks correct too.  However, I would use call_user_function_ex, which
 relieves you of the burden of creating a retval (among other things).

The API for call_user_function... changed (new parameter was added) so the
instructions in the docs are totally borked.  Someone else figured out how
to make it work and added a helpful note to the page for using
call_user_function() (not _ex()) so I just followed their hints without
using my brain and reading the source.  It seems to work.

Thanks for the tip though, I'll look into it once I overcome this bigger
nastiness.

 Note also that the variables which are getting corrupted are simple
 strings, not even zvals.  Here is how I'm setting the value of
 serverId, which is one of those getting corrupted:

 add_property_string(return_value, serverId, serverId, TRUE);
 efree(serverId);

 In this case, it is better to do this:

 add_property_string(return_value, serverId, serverId, FALSE);
 and not use the serverId again.

Thanks for your very clear explanation (which I've snipped out).  That's more
or less as I understood it, but now I feel more confident.   I doubt you
expected that to be related to my bug, but I changed all such instances
anyway, and of course the problem persists.

  Did you --enable-debug in your development build?  If not, I strongly
 recommend that you do, as it will help detect overruns in emalloc'd
 memory.  This is a great way to track down the lines of code that are at
 fault.

Yes I've been developing with debug enabled, it's a real lifesaver
if--like me--you're not perfect.  It also helped me to understand how Zend
manages memory, since it would tell me if I neededed to efree() something
or if it was taken care of by Zend.

But this raises an interesting point...  I accidentally re-ran configure
_without_ --enable-debug, did a make clean and make, and suddenly my main
bug went away!  return_value is no longer getting corrupted in the same
way, and I am not getting a segfault when trying to access my object's
member variable!  But let's not get too excited here...

I say no longer getting corrupted in the same way because it is getting
corrupted... but in a very strange way which I encountered several months
ago, and used an ugly hack to get around, rather than properly solving the
issue.

If you ask me, this is stranger yet.  If you recall from my original post,
I am trying to return a serverId object member which contains a value
such as N^VESoDMN(107).  Again, right before my PHP_FUNCTION returns
control to the script, this value is perfect.  When I DON'T use
--enable-debug, and the serverId is NOT getting completely clobbered, the
value that is returned by $table-serverId is not N^VESoDMN(107).  It is
N^VESoDMN'107)

This is not a random thing.  It is very consistent--I have several
PHP_FUNCTIONs that return serverIds, either with RETVAL_STRING or as a
member of a more complex object, and server IDs are always in a similar
format, a string ending with a parenthesized integer.  Yet EVERY time, the
( is replaced by a single quote.

Does this make ANY sense to anyone out there?  Is there some hidden hack
within Zend that is being triggered by this string pattern (perhaps the ^
followed by ( in a string?) and which causes Zend to replace a
open-paren with a single-quote?

To be perfectly clear, no other character is ever changed, and the ( is
always changed in the same way.  Aside from that, the serverId is valid
and if I
replace the ' with a (, the server recognizes the ID as valid when I send
it back.

 The other thing to try is valgrind - it is possible that some of your
 non-zend-api code is misbehaving, and valgrind will help you find that
 problem code.

Another excellent suggestion, and I tried it this morning, even with the
-v option.  Result: nothing.  No warnings except to tell me I was about to
have a segfault.  I haven't read the whole valgrind manual, but unless you
can think of some command-line option that could be helpful, I don't think
that's going to get me anywhere.

I don't know if the  ( becomes '  glitch offers any clue as to what
could be happening, but if not, I guess I am doomed to step interminably
through php/Zend in gdb, and become far more intimately familiar with the
inner workings of Zend than I would really like.

And to be clear, once I re-enabled debug in configure, my data is being
completely corrupted just as before.  Does this offer any clues?  I have
come across some #IFDEFs etc. in the source, where the debugging code is
activated, but before I go and look at every such instance for clues,
maybe someone else has an idea?

Thanks again for your assistance.

Bamboozled,
Eric




-- 
PHP Development Mailing List http://www.php.net/
To unsubscribe, visit: http://www.php.net/unsub.php

[PHP-DEV] repost: return_value getting corrupted in extension

2003-02-11 Thread Eric Lambart 
I originally posted this about 20 hours ago, and it has still not
appeared on the newsgroup.  I assume something went wrong, and apologize
if it appears twice.

In my original message, I neglected to mention that I am compiling code
this with PHP 4.2.3, though I have had similar problems with an earlier
version of this extension (on another dev server) with 4.3.0.

---

Hello all,
I have scoured the php.dev group, reading most of the messages I could
find regarding extensions and modules, but could not find any other
reports of this problem anywhere.  I have a feeling there is an obvious
answer.

I have a PHP_FUNCTION() defined within my extension, which is in turn 
calling a standard C function (called getDataReply()) which is passed 
the standard return_value zval*.  My getDataReply() function initializes 
a Zend object (which I have defined) using object_init_ex(), then calls 
the object's constructor, and then assigns some values to the members of 
the object.

That is all well and good.  Everything works great.  I can use a
php_var_dump() and everything looks as it should.  I even do another
php_var_dump(return_value, 1 TSRMLS_CC) within my PHP_FUNCTION(), and
sure enough... return_value still contains the proper data structure (see
below).

The problem occurs as soon as control returns to the script.  In my test
script, I do a var_dump() and the data is corrupted!  I have looked at
the Zend internals but forgive me, I don't have the time or patience to 
to step line-by-line through code (using gdb) that has few comments and 
which I don't understand very well.

Can anyone please give me some ideas as to what I could be doing wrong
here?  NOTE that I have discovered that if I perform a deep copy using 
zend_copy_ctor(return_value) before returning to the script, my data is
undamaged... but then I have a big memory leak.  I also anticipate having
very large data sets within my objects, and deep-copying all of that 
every time would be very expensive.

Here is a how I am calling my C function from within my PHP function:

  PHP_FUNCTION(eo_create_object)
  {
..
.. do stuff
..

/* let's see what the server has to say */
temp = getDataReply(conn, return_value);

if ( temp )
{
  .. in this case return_value is set based on the value of temp, a
  .. fairly simple data structure (but this code is NOT being reached;
  .. see below.  return_value is only being modified by the above 
  .. function.
}
else
{
  printf(temp is NULL\n);
}

printf(done with getDataReply()\n);

/* this php_var_dump() returns uncorrupted data */

php_var_dump(return_value, 1 TSRMLS_CC);

  } // end eo_create_object()

I don't see why it should make any difference, but in another source file,
getDataReply() is declared like this:

  variant *getDataReply(eo_connection *conn, zval *return_value)
  {
.. do stuff, init return_value as eo_table class, init it, and fill w/data
  }

The following are the exact results of the php_var_dump() from within 
zif_eo_create_object(), _immediately_ before returning control to the 
script.
---
  temp is NULL
  done with getDataReply()
  object(eo_table)(10) {
[serverId]=
string(14) N^VESoDMN(94)
[rows]=
int(0)
[cols]=
int(0)
[namedCols]=
bool(false)
[errorText]=
string(0) 
[errorItem]=
string(0) 
[errorId]=
int(0)
[flags]=
int(0)
[colNames]=
NULL
[table]=
NULL
  }

Those data are correct, everything looks PERFECT.

-

Here is what I am doing in the test script (PHP code):

  ?php

  /* $conn is a resource, $o is an associative array */
  $table = eo_create_object($conn, '...table', 0, $o, NULL, NULL, NULL);
  var_dump($table);

  ?

And here are the results of the last var_dump():
  object(eo_table)(10) {
[serverId]=
UNKNOWN:0
[rows]=
UNKNOWN:0
[cols]=
int(0)
[namedCols]=
bool(false)
[errorText]=
string(0) 
[errorItem]=
string(0) 
[errorId]=
int(0)
[flags]=
int(0)
[colNames]=
NULL
[table]=
NULL
  }

...as you can see, some of the values have been corrupted.  As soon as I try 
to access $table-serverId, PHP segfaults in erealloc().  I will not include 
the backtrace here as it seems irrelevant; serverId no longer has a legitimate 
value, so of course it cannot be accessed.

This is a simple example.  The member variable table, when it has a value, 
contains an associative array.  Sometimes those values are OK, sometimes they
are corrupted.  I have had other strange corruptions in returned values, and 
I do not understand why.

Thanks for any help you can offer,
Eric

-- 
This message was created in a marvelously Microsoft-free computing environment.

To email me, ADD spam before the bo

-- 
PHP Development Mailing List http://www.php.net/
To unsubscribe, visit: http://www.php.net/unsub.php