Bug#911415: flex: yyinput() return value is backward incompatible

[Manoj Srivastava]

On Tue, May 26 2020, Bill Allombert wrote:

> On Fri, May 22, 2020 at 03:16:56AM -0700, Manoj Srivastava wrote:
>> 
>> On 12:19 Thursday, 21 May 2020 +0200, Bill Allombert wrote
>>  > On Wed, May 20, 2020 at 10:03:08PM -0700, Manoj Srivastava wrote:
>>  > > While it is true that the change was incompatible wwith what we
>>  > >  had befire, the change was made almost four and a half years ago. I
>>  > >  suspect we have gotten used to it now; and changing it back would just
>>  > >  cause issues.
>>  > 
>>  > Is the new behaviour documented now ?
>>  > This is needed to use yyinput() properly.
>> 
>>  See below. The yyinput usage is demonstrated in an example in
>>  the info node about generating C++ scanners.

> Well but this is not proper documentation, this is incidental
> documentation at best.  Someone writing a C scanner will not read this
> example.  Is it documented somewhere that the yyinput API has changed
> ?

Why should a writer of a C scanner care?

,[ 8 Actions ]
| (Note that if the scanner is compiled using 'C++', then 'input()' is
| instead referred to as yyinput(), in order to avoid a name clash with
| the 'C++' stream by the name of 'input'.)
`

People writing C++ scanners are likely to see it, and they are
 the only ones to care. One could create a github documentation issue
 about it if one feels strongly enough.


> Note that the time should be computed between the time the new flex is
> released and the time I report the problem, not between the git commit
> and now.

As far as upstream is concerned when they commited it is when
 the feature went out. At thi point, just reverting it is another change
 in behaviour. Again, a github issue could be an option if you feel this
 strongly about it.

Manoj

--
I don't care where I sit as long as I get fed.  -- Calvin Trillin
Manoj Srivastava    
4096R/C5779A1C E37E 5EC5 2A01 DA25 AD20  05B6 CF48 9438 C577 9A1C


smime.p7s
Description: S/MIME cryptographic signature

Bug#911415: flex: yyinput() return value is backward incompatible

[Bill Allombert]

On Fri, May 22, 2020 at 03:16:56AM -0700, Manoj Srivastava wrote:
> 
> On 12:19 Thursday, 21 May 2020 +0200, Bill Allombert wrote
>  > On Wed, May 20, 2020 at 10:03:08PM -0700, Manoj Srivastava wrote:
>  > > While it is true that the change was incompatible wwith what we
>  > >  had befire, the change was made almost four and a half years ago. I
>  > >  suspect we have gotten used to it now; and changing it back would just
>  > >  cause issues.
>  > 
>  > Is the new behaviour documented now ?
>  > This is needed to use yyinput() properly.
> 
>   See below. The yyinput usage is demonstrated in an example in
>  the info node about generating C++ scanners.

Well but this is not proper documentation, this is incidental documentation at
best.  Someone writing a C scanner will not read this example.
Is it documented somewhere that the yyinput API has changed ?

> > The commit message does give any rationale or information about this.
>  > 
>  > > I guess this is the new normal?
>  > 
>  > Do you mean that sacarstically ?
> 
>   What make you think that? After 4.5 years of this being the
>  default, stating that yyinput returning NULL is normal seems like a
>  statement of fact.

Sorry, I always understood the expression 'the new normal' to be sarcastic in
nature.

Note that the time should be computed between the time the new flex is
released and the time I report the problem, not between the git commit
and now.

Cheers,
Bill.

Bug#911415: flex: yyinput() return value is backward incompatible

[Manoj Srivastava]

On 12:19 Thursday, 21 May 2020 +0200, Bill Allombert wrote
 > On Wed, May 20, 2020 at 10:03:08PM -0700, Manoj Srivastava wrote:
 > > While it is true that the change was incompatible wwith what we
 > >  had befire, the change was made almost four and a half years ago. I
 > >  suspect we have gotten used to it now; and changing it back would just
 > >  cause issues.
 > 
 > Is the new behaviour documented now ?
 > This is needed to use yyinput() properly.

See below. The yyinput usage is demonstrated in an example in
 the info node about generating C++ scanners.


> The commit message does give any rationale or information about this.
 > 
 > > I guess this is the new normal?
 > 
 > Do you mean that sacarstically ?

What make you think that? After 4.5 years of this being the
 default, stating that yyinput returning NULL is normal seems like a
 statement of fact.

Manoj

In Node: 18 Generating C++ Scanners
===
   Here is an example of a simple C++ scanner:

  // An example of using the flex C++ scanner class.
 
 %{
 #include 
 using namespace std;
 int mylineno = 0;
 %}
 
 %option noyywrap c++
 
 string  \"[^\n"]+\"
 
 ws  [ \t]+
 
 alpha   [A-Za-z]
 dig [0-9]
 name({alpha}|{dig}|\$)({alpha}|{dig}|[_.\-/$])*
 num1[-+]?{dig}+\.?([eE][-+]?{dig}+)?
 num2[-+]?{dig}*\.{dig}+([eE][-+]?{dig}+)?
 number  {num1}|{num2}
 
 %%
 
 {ws}/* skip blanks and tabs */
 
 "/*"{
 int c;
 
 while((c = yyinput()) != 0)
 {
 if(c == '\n')
 ++mylineno;
 
 else if(c == '*')
 {
 if((c = yyinput()) == '/')
 break;
 else
 unput(c);
 }
 }
 }
 
 {number}  cout << "number " << YYText() << '\n';
 
 \nmylineno++;
 
 {name}cout << "name " << YYText() << '\n';
 
 {string}  cout << "string " << YYText() << '\n';
 
 %%
 
// This include is required if main() is an another source file.
//#include 
 
 int main( int /* argc */, char** /* argv */ )
 {
 FlexLexer* lexer = new yyFlexLexer;
 while(lexer->yylex() != 0)
 ;
 return 0;
 }

--
Two wrights don't make a rong, they make an airplane.  Or bicycles.
Manoj Srivastava  
4096R/C5779A1C E37E 5EC5 2A01 DA25 AD20  05B6 CF48 9438 C577 9A1C

Bug#911415: flex: yyinput() return value is backward incompatible

[Bill Allombert]

On Wed, May 20, 2020 at 10:03:08PM -0700, Manoj Srivastava wrote:
> Hi,
> 
> While it is true that the change was incompatible wwith what we
>  had befire, the change was made almost four and a half years ago. I
>  suspect we have gotten used to it now; and changing it back would just
>  cause issues.

Hello Manoj,
Is the new behaviour documented now ?
This is needed to use yyinput() properly.
The commit message does give any rationale or information about this.

> I guess this is the new normal?

Do you mean that sacarstically ?

Cheers,
Bill.

Bug#911415: flex: yyinput() return value is backward incompatible

[Bill Allombert]

Package: flex
Version: 2.6.1-1.3
Severity: normal

Hello Manoj,
In flex 2.6 input() return 0 instead of EOF in the EOB_ACT_END_OF_FILE
case:
case EOB_ACT_END_OF_FILE:
{
-   if ( yywrap( ) )
-   return EOF;
+   if ( yywrap(  ) )
+   return 0;

Since any user of input() needs to check its return value, this change
breaks backward compatibility with pre-2.6 flex and is not documented
anywhere I can see. Also what happens if the input stream includes NULs ?

Cheers,
-- 
Bill. 

Imagine a large red swirl here.