From: [EMAIL PROTECTED]
Operating system: n/a
PHP version: 5.0.5
PHP Bug Type: Scripting Engine problem
Bug description: tokenizer ext drops final comment
Description:
------------
The tokenizer extension is ignoring a final comment when tokenizing a
script that does not contain a closing ?>
Reproduce code:
---------------
var_dump(token_get_all("<?php print 'foo'; # you'll see it
print 'bar'; # but not this one"));
Expected result:
----------------
array(13) {
[0]=>
array(2) {
[0]=>
int(366)
[1]=>
string(6) "<?php "
}
[1]=>
array(2) {
[0]=>
int(266)
[1]=>
string(5) "print"
}
[2]=>
array(2) {
[0]=>
int(369)
[1]=>
string(1) " "
}
[3]=>
array(2) {
[0]=>
int(315)
[1]=>
string(5) "'foo'"
}
[4]=>
string(1) ";"
[5]=>
array(2) {
[0]=>
int(369)
[1]=>
string(1) " "
}
[6]=>
array(2) {
[0]=>
int(364)
[1]=>
string(16) "# you'll see it
"
}
[7]=>
array(2) {
[0]=>
int(369)
[1]=>
string(1) " "
}
[8]=>
array(2) {
[0]=>
int(266)
[1]=>
string(5) "print"
}
[9]=>
array(2) {
[0]=>
int(369)
[1]=>
string(1) " "
}
[10]=>
array(2) {
[0]=>
int(315)
[1]=>
string(5) "'bar'"
}
[11]=>
string(1) ";"
[12]=>
array(2) {
[0]=>
int(369)
[1]=>
string(1) " "
}
[13]=>
array(2) {
[0]=>
int(364)
[1]=>
string(18) "# but not this one"
}
Actual result:
--------------
array(13) {
[0]=>
array(2) {
[0]=>
int(366)
[1]=>
string(6) "<?php "
}
[1]=>
array(2) {
[0]=>
int(266)
[1]=>
string(5) "print"
}
[2]=>
array(2) {
[0]=>
int(369)
[1]=>
string(1) " "
}
[3]=>
array(2) {
[0]=>
int(315)
[1]=>
string(5) "'foo'"
}
[4]=>
string(1) ";"
[5]=>
array(2) {
[0]=>
int(369)
[1]=>
string(1) " "
}
[6]=>
array(2) {
[0]=>
int(364)
[1]=>
string(16) "# you'll see it
"
}
[7]=>
array(2) {
[0]=>
int(369)
[1]=>
string(1) " "
}
[8]=>
array(2) {
[0]=>
int(266)
[1]=>
string(5) "print"
}
[9]=>
array(2) {
[0]=>
int(369)
[1]=>
string(1) " "
}
[10]=>
array(2) {
[0]=>
int(315)
[1]=>
string(5) "'bar'"
}
[11]=>
string(1) ";"
[12]=>
array(2) {
[0]=>
int(369)
[1]=>
string(1) " "
}
}
--
Edit bug report at http://bugs.php.net/?id=35286&edit=1
--
Try a CVS snapshot (php4): http://bugs.php.net/fix.php?id=35286&r=trysnapshot4
Try a CVS snapshot (php5.0):
http://bugs.php.net/fix.php?id=35286&r=trysnapshot50
Try a CVS snapshot (php5.1):
http://bugs.php.net/fix.php?id=35286&r=trysnapshot51
Fixed in CVS: http://bugs.php.net/fix.php?id=35286&r=fixedcvs
Fixed in release: http://bugs.php.net/fix.php?id=35286&r=alreadyfixed
Need backtrace: http://bugs.php.net/fix.php?id=35286&r=needtrace
Need Reproduce Script: http://bugs.php.net/fix.php?id=35286&r=needscript
Try newer version: http://bugs.php.net/fix.php?id=35286&r=oldversion
Not developer issue: http://bugs.php.net/fix.php?id=35286&r=support
Expected behavior: http://bugs.php.net/fix.php?id=35286&r=notwrong
Not enough info:
http://bugs.php.net/fix.php?id=35286&r=notenoughinfo
Submitted twice:
http://bugs.php.net/fix.php?id=35286&r=submittedtwice
register_globals: http://bugs.php.net/fix.php?id=35286&r=globals
PHP 3 support discontinued: http://bugs.php.net/fix.php?id=35286&r=php3
Daylight Savings: http://bugs.php.net/fix.php?id=35286&r=dst
IIS Stability: http://bugs.php.net/fix.php?id=35286&r=isapi
Install GNU Sed: http://bugs.php.net/fix.php?id=35286&r=gnused
Floating point limitations: http://bugs.php.net/fix.php?id=35286&r=float
No Zend Extensions: http://bugs.php.net/fix.php?id=35286&r=nozend
MySQL Configuration Error: http://bugs.php.net/fix.php?id=35286&r=mysqlcfg
