ID:               26463
 Updated by:       [EMAIL PROTECTED]
 Reported By:      andrey at midphase dot com
-Status:           Closed
+Status:           Open
 Bug Type:         Scripting Engine problem
 Operating System: Windows XP
-PHP Version:      4.3.4
+PHP Version:      PHP 5 CVS HEAD
 Assigned To:      iliaa
 New Comment:

$a = token_get_all('<?php return <<<EOF

test me

EOF; ?>');

var_dump($a);

foreach ($a as $token) {

    @var_dump(token_name($token[0]));

}



generates



array(10) {

  [0]=>

  array(2) {

    [0]=>

    int(366)

    [1]=>

    string(6) "<?php "

  }

  [1]=>

  array(2) {

    [0]=>

    int(335)

    [1]=>

    string(6) "return"

  }

  [2]=>

  array(2) {

    [0]=>

    int(369)

    [1]=>

    string(1) " "

  }

  [3]=>

  array(2) {

    [0]=>

    int(370)

    [1]=>

    string(8) "<<<EOF

"

  }

  [4]=>

  array(2) {

    [0]=>

    int(307)

    [1]=>

    string(4) "test"

  }

  [5]=>

  array(2) {

    [0]=>

    int(314)

    [1]=>

    string(1) " "

  }

  [6]=>

  array(2) {

    [0]=>

    int(307)

    [1]=>

    string(2) "me"

  }

  [7]=>

  array(2) {

    [0]=>

    int(314)

    [1]=>

    string(2) "

"

  }

  [8]=>

  array(2) {

    [0]=>

    int(307)

    [1]=>

    string(3) "EOF"

  }

  [9]=>

  array(2) {

    [0]=>

    int(314)

    [1]=>

    string(4) "; ?>"

  }

}

string(10) "T_OPEN_TAG"

string(8) "T_RETURN"

string(12) "T_WHITESPACE"

string(15) "T_START_HEREDOC"

string(8) "T_STRING"

string(25) "T_ENCAPSED_AND_WHITESPACE"

string(8) "T_STRING"

string(25) "T_ENCAPSED_AND_WHITESPACE"

string(8) "T_STRING"

string(25) "T_ENCAPSED_AND_WHITESPACE"



This last token is a T_END_HEREDOC, T_WHITESPACE and T_CLOSE_TAG all
combined incorrectly into a T_ENCAPSED_AND_WHITESPACE.  If any other
statements follow the T_END_HEREDOC, output is as expected.



Greg


Previous Comments:
------------------------------------------------------------------------

[2003-11-29 14:04:12] [EMAIL PROTECTED]

This bug has been fixed in CVS.

In case this was a PHP problem, snapshots of the sources are packaged
every three hours; this change will be in the next snapshot. You can
grab the snapshot at http://snaps.php.net/.
 
In case this was a documentation problem, the fix will show up soon at
http://www.php.net/manual/.

In case this was a PHP.net website problem, the change will show
up on the PHP.net site and on the mirror sites in short time.
 
Thank you for the report, and for helping us make PHP better.



------------------------------------------------------------------------

[2003-11-29 10:10:59] andrey at midphase dot com

Sorry, actual content of test.php is



<?php

  $tokens=token_get_all(file_get_contents("code.php"));

  print_r($tokens);

?>

------------------------------------------------------------------------

[2003-11-29 10:08:04] andrey at midphase dot com

Description:
------------
When calling token_get_all with a string containing heredoc strings,
T_END_HEREDOC element contains two semicolons separated by newline and
is  followed by semicolon character.

This does not happen if heredoc string is followed by concatenation
operatior



Reproduce code:
---------------
=====================

code.php

=====================

<?php

      $x=<<<DD

  jhdsjkfhjdsh

DD

."";

      $a=<<<DD

  jhdsjkfhjdsh

DD;

  echo $a;

?>



=====================

test.php

=====================

<?php

  $tokens=token_get_all(file_get_contents("test.php"));

  print_r($tokens);

?>

Expected result:
----------------
Array

(

    [0] => Array

        (

            [0] => 354

            [1] => <?php



        )



    [1] => Array

        (

            [0] => 357

            [1] =>       

        )



    [2] => Array

        (

            [0] => 307

            [1] => $x

        )



    [3] => =

    [4] => Array

        (

            [0] => 358

            [1] => <<<DD



        )



    [5] => Array

        (

            [0] => 312

            [1] =>   

        )



    [6] => Array

        (

            [0] => 305

            [1] => jhdsjkfhjdsh

        )



    [7] => Array

        (

            [0] => 312

            [1] => 



        )



    [8] => Array

        (

            [0] => 359

            [1] => DD



        )



    [9] => .

    [10] => Array

        (

            [0] => 313

            [1] => ""

        )



    [11] => ;

    [12] => Array

        (

            [0] => 357

            [1] => 

      

        )



    [13] => Array

        (

            [0] => 307

            [1] => $a

        )



    [14] => =

    [15] => Array

        (

            [0] => 358

            [1] => <<<DD



        )



    [16] => Array

        (

            [0] => 312

            [1] =>   

        )



    [17] => Array

        (

            [0] => 305

            [1] => jhdsjkfhjdsh

        )



    [18] => Array

        (

            [0] => 312

            [1] => 



        )



    [19] => Array

        (

            [0] => 359

            [1] => DD;



        )



/*    [20] => ;  deleted */

    [21] => Array

        (

            [0] => 357

            [1] =>   

        )



    [22] => Array

        (

            [0] => 314

            [1] => echo

        )



    [23] => Array

        (

            [0] => 357

            [1] =>  

        )



    [24] => Array

        (

            [0] => 307

            [1] => $a

        )



    [25] => ;

    [26] => Array

        (

            [0] => 357

            [1] => 



        )



    [27] => Array

        (

            [0] => 356

            [1] => ?>

        )



)



Actual result:
--------------
Array

(

    [0] => Array

        (

            [0] => 354

            [1] => <?php



        )



    [1] => Array

        (

            [0] => 357

            [1] =>       

        )



    [2] => Array

        (

            [0] => 307

            [1] => $x

        )



    [3] => =

    [4] => Array

        (

            [0] => 358

            [1] => <<<DD



        )



    [5] => Array

        (

            [0] => 312

            [1] =>   

        )



    [6] => Array

        (

            [0] => 305

            [1] => jhdsjkfhjdsh

        )



    [7] => Array

        (

            [0] => 312

            [1] => 



        )



    [8] => Array

        (

            [0] => 359

            [1] => DD



        )



    [9] => .

    [10] => Array

        (

            [0] => 313

            [1] => ""

        )



    [11] => ;

    [12] => Array

        (

            [0] => 357

            [1] => 

      

        )



    [13] => Array

        (

            [0] => 307

            [1] => $a

        )



    [14] => =

    [15] => Array

        (

            [0] => 358

            [1] => <<<DD



        )



    [16] => Array

        (

            [0] => 312

            [1] =>   

        )



    [17] => Array

        (

            [0] => 305

            [1] => jhdsjkfhjdsh

        )



    [18] => Array

        (

            [0] => 312

            [1] => 



        )



    [19] => Array

        (

            [0] => 359

            [1] => DD;

;

        )



    [20] => ;

    [21] => Array

        (

            [0] => 357

            [1] =>   

        )



    [22] => Array

        (

            [0] => 314

            [1] => echo

        )



    [23] => Array

        (

            [0] => 357

            [1] =>  

        )



    [24] => Array

        (

            [0] => 307

            [1] => $a

        )



    [25] => ;

    [26] => Array

        (

            [0] => 357

            [1] => 



        )



    [27] => Array

        (

            [0] => 356

            [1] => ?>

        )



)




------------------------------------------------------------------------


-- 
Edit this bug report at http://bugs.php.net/?id=26463&edit=1

Reply via email to