Re: [Cocci] Replacing one (specific!) type with another

[Julia Lawall]

> > The name is illustrated in Advanced SmPL:
> > http://coccinelle.lip6.fr/papers/cocciwk4_talk2.pdf
>
> Call me stupid, but I just went through it again and I still don't see
> it. There are lots of examples with "@ ..stuff..@" (the first on slide
> 4), but as far as I can tell it never explains what this means.

OK, perhaps it is mentioned in the video:
http://faultlinux.lip6.fr/videos/workshop_videos_2011/Julia_Lawall_introduction_part2.f4v

>
> Actually, the frequent occurence of @script:python@ makes me think that
> this actually has semantic signficance and is much more than a label
> that I can choose for my own convenience...?

script:python indicates that the rule has python code rather than pattern
matching.  The name of a rule that you can choose freely should not
contain :

>
>
> > A variety of metavariable types are listed in slide 11 of the Linux
> > oriented tutorial: http://coccinelle.lip6.fr/papers/tutorial.pdf
>
> Yes, I saw that. But I deliberately wrote "explained" rather than just
> "listed" :-).

In the video, perhaps...

The following paper is quite old, but may also be helpful:

Tutorial paper: Semantic Patches, Documenting and Automating Collateral
Evolutions in Linux Device Drivers
Yoann Padioleau, Julia L. Lawall, and Gilles Muller
Ottawa Linux Symposium (OLS 2007), June 2007.

julia
___
Cocci mailing list
Cocci@systeme.lip6.fr
https://systeme.lip6.fr/mailman/listinfo/cocci

Re: [Cocci] Replacing one (specific!) type with another

[Julia Lawall]

On Sat, 8 Oct 2016, Nikolaus Rath wrote:

> On Oct 08 2016, Julia Lawall  wrote:
> > On Fri, 7 Oct 2016, Nikolaus Rath wrote:
> >
> >> On Oct 05 2016, Julia Lawall  wrote:
> >> >>
> >> >> 2. ..and how would I go about if instead of the type, I want to replace
> >> >>a variable name? (my_type *ptr --> my_type *pointer).
> >> >
> >> > I'm not completely sure what the issue is here.  Do you specifically want
> >> > to convert ptr to pointer?  Is the type important?  To make exactly what
> >> > you have written, you could put:
> >> >
> >> > @@
> >> > typedef my_type;
> >> > idexpression mytype * p1;
> >> > @@
> >> >
> >> > - ptr@p1
> >> > + pointer
> >> >
> >> > This checks for the word ptr, and also checks that it is an identifier of
> >> > the right type.  I haven't tested it, so let me know if there is any
> >> > problem.
> >>
> >> It workes somewhat... but not completely.
> >>
> >> Here's what I wanted to do: I merged two structures (struct fuse_session
> >> and struct fuse_ll) into one (struct fuse_session). I've first replaced
> >> all the type names, and then manually fixed the cases where this
> >> resulted in bogus/redundant code (typically in functions that used to
> >> work with both structs).
> >>
> >> Now my project compiles and runs fine, but I the variable naming is
> >> inconsistent: in some cases the struct fuse_session pointer is called
> >> *se (these were the variables that were always of type struct
> >> fuse_session), and in other cases the pointer is called *f (these were
> >> the variables that were previously of type struct fuse_ll).
> >>
> >> I'd like to fix this too, and always refer call fuse_session pointers
> >> "se" (except where this name is already used for something else, but
> >> I'll just fix this up by hand afterwards). I tried the following patch:
> >>
> >> $ cat se-name.cocci
> >> @@
> >> idexpression struct fuse_session *p1;
> >> @@
> >> - f@p1
> >> + se
> >>
> >> but it resulted in these changes:
> >>
> >>struct fuse_session *f = req->se;
> >> -  struct cuse_data *cd = f->cuse_data;
> >> -  size_t bufsize = f->bufsize;
> >> +  struct cuse_data *cd = se->cuse_data;
> >> +  size_t bufsize = se->bufsize;
> >>
> >> So it seems to replace the variable where its used, but not where it's
> >> defined.
> >>
> >> Is there a way to catch the definitions too?
> >
> > Write separate rules for that.  You would need one case for a local
> > variable and one case fora parameter.  You can actually probably just drop
> > the rule you have currently.  I would imagine something like the
> > following:
> >
> > @@
> > symbol f, se; // avoid unneeded warnings from Coccinelle
> > @@
> >
> > struct fuse_session *
> > -f
> > +se
> >  ;
> > <...
> > -f
> > +se
> > ...>
>
> Could you explain how this works (in particular the effect of the angle
> brackets)? I also can't resist to point out that "symbol" is not
> included in the list of metavariable types from the tutorial slides :-).

<... P ...> is called a nest.  It means that the pattern can appear 0 or
more times.  You can put a transformation on the pattern.  It will happen
as many times as the thing appears.

The list on the slides is not exhaustive.  The list in the language
grammar should be exhaustive:
http://coccinelle.lip6.fr/docs/index.html

Symbol also just exists to quiet a warning message; it's not essential.

>
> > @@
> > identifier fn;
> > @@
> >
> > fn(...,struct fuse_session *f,...) { <...
> > -f
> > +se
> > ...> }
> >
> > I think that the symbol declaration has effect in the rest of the semantic
> > patch, and does not have to be repeated.  If you get warnings for the
> > second rule, just copy it down.
>
> Not sure what you mean with "copy it down".

Put the symbol declaration in both rules.

> I don't get Coccinelle
> warnings, but if I just use the two rules as you gave them, then
> it looks as if the second one isn't working:
>
> @@ -584,9 +584,9 @@ static struct fuse_ll_pipe *fuse_ll_get_
>
>  static void fuse_ll_clear_pipe(struct fuse_session *f)
>  {
> - struct fuse_ll_pipe *llp = pthread_getspecific(f->pipe_key);
> + struct fuse_ll_pipe *llp = pthread_getspecific(se->pipe_key);
>   if (llp) {
> - pthread_setspecific(f->pipe_key, NULL);
> + pthread_setspecific(se->pipe_key, NULL);
>   fuse_ll_pipe_free(llp);
>   }
>  }
>
>
> Is the problem that "...," does not match nothing, i.e. *f must not be
> the first argument of the function?

It should match nothing.  What version of Coccinelle do you have?

julia

>
> Best,
> -Nikolaus
>
> --
> GPG encrypted emails preferred. Key id: 0xD113FCAC3C4E599F
> Fingerprint: ED31 791B 2C5C 1613 AF38 8B8A D113 FCAC 3C4E 599F
>
>  »Time flies like an arrow, fruit flies like a Banana.«
> ___
> Cocci mailing list
> Cocci@systeme.lip6.fr
> https://systeme.lip6.fr/mailman/listinfo/cocci

Re: [Cocci] Replacing one (specific!) type with another

[Nikolaus Rath]

On Oct 08 2016, Julia Lawall  wrote:
> On Fri, 7 Oct 2016, Nikolaus Rath wrote:
>
>> On Oct 05 2016, Julia Lawall  wrote:
>> >>
>> >> 2. ..and how would I go about if instead of the type, I want to replace
>> >>a variable name? (my_type *ptr --> my_type *pointer).
>> >
>> > I'm not completely sure what the issue is here.  Do you specifically want
>> > to convert ptr to pointer?  Is the type important?  To make exactly what
>> > you have written, you could put:
>> >
>> > @@
>> > typedef my_type;
>> > idexpression mytype * p1;
>> > @@
>> >
>> > - ptr@p1
>> > + pointer
>> >
>> > This checks for the word ptr, and also checks that it is an identifier of
>> > the right type.  I haven't tested it, so let me know if there is any
>> > problem.
>>
>> It workes somewhat... but not completely.
>>
>> Here's what I wanted to do: I merged two structures (struct fuse_session
>> and struct fuse_ll) into one (struct fuse_session). I've first replaced
>> all the type names, and then manually fixed the cases where this
>> resulted in bogus/redundant code (typically in functions that used to
>> work with both structs).
>>
>> Now my project compiles and runs fine, but I the variable naming is
>> inconsistent: in some cases the struct fuse_session pointer is called
>> *se (these were the variables that were always of type struct
>> fuse_session), and in other cases the pointer is called *f (these were
>> the variables that were previously of type struct fuse_ll).
>>
>> I'd like to fix this too, and always refer call fuse_session pointers
>> "se" (except where this name is already used for something else, but
>> I'll just fix this up by hand afterwards). I tried the following patch:
>>
>> $ cat se-name.cocci
>> @@
>> idexpression struct fuse_session *p1;
>> @@
>> - f@p1
>> + se
>>
>> but it resulted in these changes:
>>
>>  struct fuse_session *f = req->se;
>> -struct cuse_data *cd = f->cuse_data;
>> -size_t bufsize = f->bufsize;
>> +struct cuse_data *cd = se->cuse_data;
>> +size_t bufsize = se->bufsize;
>>
>> So it seems to replace the variable where its used, but not where it's
>> defined.
>>
>> Is there a way to catch the definitions too?
>
> Write separate rules for that.  You would need one case for a local
> variable and one case fora parameter.  You can actually probably just drop
> the rule you have currently.  I would imagine something like the
> following:
>
> @@
> symbol f, se; // avoid unneeded warnings from Coccinelle
> @@
>
> struct fuse_session *
> -f
> +se
>  ;
> <...
> -f
> +se
> ...>

Could you explain how this works (in particular the effect of the angle
brackets)? I also can't resist to point out that "symbol" is not
included in the list of metavariable types from the tutorial slides :-).

> @@
> identifier fn;
> @@
>
> fn(...,struct fuse_session *f,...) { <...
> -f
> +se
> ...> }
>
> I think that the symbol declaration has effect in the rest of the semantic
> patch, and does not have to be repeated.  If you get warnings for the
> second rule, just copy it down.

Not sure what you mean with "copy it down". I don't get Coccinelle
warnings, but if I just use the two rules as you gave them, then
it looks as if the second one isn't working:

@@ -584,9 +584,9 @@ static struct fuse_ll_pipe *fuse_ll_get_
 
 static void fuse_ll_clear_pipe(struct fuse_session *f)
 {
-   struct fuse_ll_pipe *llp = pthread_getspecific(f->pipe_key);
+   struct fuse_ll_pipe *llp = pthread_getspecific(se->pipe_key);
if (llp) {
-   pthread_setspecific(f->pipe_key, NULL);
+   pthread_setspecific(se->pipe_key, NULL);
fuse_ll_pipe_free(llp);
}
 }


Is the problem that "...," does not match nothing, i.e. *f must not be
the first argument of the function?


Best,
-Nikolaus

-- 
GPG encrypted emails preferred. Key id: 0xD113FCAC3C4E599F
Fingerprint: ED31 791B 2C5C 1613 AF38 8B8A D113 FCAC 3C4E 599F

 »Time flies like an arrow, fruit flies like a Banana.«
___
Cocci mailing list
Cocci@systeme.lip6.fr
https://systeme.lip6.fr/mailman/listinfo/cocci

Re: [Cocci] Replacing one (specific!) type with another

[Nikolaus Rath]

On Oct 08 2016, Julia Lawall  wrote:
> On Fri, 7 Oct 2016, Nikolaus Rath wrote:
>
>> On Oct 06 2016, Julia Lawall  wrote:
>> >> Somehow I'm having a really hard time grasping the fundamentals. The
>> >> first two examples make sense - I could adapt them for similar
>> >> situations. But then, I still have absolutely no idea how I would come
>> >> up with the third example, or how to adapt it. What does "idexpression"
>> >> mean? What does "ptr@p1" mean? Is this documented anywhere?
>> >
>> > idexpression is an expression that is resricted to be an identifier.  It
>> > allows you to put a type on an expression that has the form of an
>> > identifier.  You can also say identifier x;  But that is just a name.  It
>> > has no type.  It could be an expression, a field name, a parameter name,
>> > etc.
>>
>> Hm. Based on your last two sentences I'd conclude that 'idexpression'
>> matches variable names for variables of a specific type. But that
>> doesn't seem to be what you describe in your first two sentences. Could
>> you explain what you meant iwth "expression that has the form of an
>> identifier"?
>>
>> To me, an identifier is something that's written literally into the
>> source code and cannot be meaningfully taken apart, e.g. a function
>> name, variable name, or the member of a struct. An expression, on the
>> other hand, is something that can be meaningfully split into
>> sub-components. Is that also how you use these terms?
>
> An expression is something that has a value.  So an idexpression would be
> something that cannot be taken apart, but it is also used in a context
> where it has a value.  Unlike in the case of a parameter name or field
> name, where it is a name for something that will get a value in the
> future.

Got it, thanks!

>> > @ connects patterns that match the same term.  So match a term against the
>> > explicit name ptr and also match it against an identifier expression that
>> > has a particular type.  This is not exactly a beginner example.
>>
>> Okay, I'll just ignore that for now.
>>
>> >> I'd hate to waste your time asking tons of such trivial questions on the
>> >> mailing list, but I just can't find any helpful documentation at all...
>> >
>> > If you look on the web page in the papers and slides section, at the top
>> > there are several tutorials and overview talks, some with video.
>>
>> I think I looked at everything that is not a video. But none of it
>> mentioned that you can put an arbitrary name between the @@ or explained
>> what the different metavariable types (idexpression, expression, etc)
>> are.
>
> The name is illustrated in Advanced SmPL:
> http://coccinelle.lip6.fr/papers/cocciwk4_talk2.pdf

Call me stupid, but I just went through it again and I still don't see
it. There are lots of examples with "@ ..stuff..@" (the first on slide
4), but as far as I can tell it never explains what this means.

Actually, the frequent occurence of @script:python@ makes me think that
this actually has semantic signficance and is much more than a label
that I can choose for my own convenience...?
 

> A variety of metavariable types are listed in slide 11 of the Linux
> oriented tutorial: http://coccinelle.lip6.fr/papers/tutorial.pdf

Yes, I saw that. But I deliberately wrote "explained" rather than just
"listed" :-).


Best,
-Nikolaus

-- 
GPG encrypted emails preferred. Key id: 0xD113FCAC3C4E599F
Fingerprint: ED31 791B 2C5C 1613 AF38 8B8A D113 FCAC 3C4E 599F

 »Time flies like an arrow, fruit flies like a Banana.«
___
Cocci mailing list
Cocci@systeme.lip6.fr
https://systeme.lip6.fr/mailman/listinfo/cocci

Re: [Cocci] Usage of "expressions" and "identifiers" with SmPL

[Nikolaus Rath]

On Oct 08 2016, SF Markus Elfring  wrote:
>> Someone who has to read so much text and in the end has no information
>> about the question he was asking will not likely get a good impression
>> about the software he is trying to use.
>
> A beginner should usually read some text for the desired learning experience.
> Can it be that you worry a bit too much about the potential for
> bad impressions around your software?

Sorry to be the bringer of bad news, but Julia is spot on. The only
effect your emails had on me was a big "WTF!?". Had Julia not responded
to my messages at almost the same time, I would have left this list and
Coccinelle for good.

> Would you like to start another marketing project?

What's the point of all these random questions that you bring up in
response to every sentence? Do you think they are helpful?

> I find that this mailing list gets only low message traffic so far.

Do you think others share this impression? Do you think this is a good
thing or a bad thing? How do you think will this situation evolve?

> So I would imagine that most well-intended discussion contributions could
> be useful. Is the mixture of presented topics reasonable?

What do you mean with discussion contributions? Do you think that asking
if the topic is reasonable is equivalent to asking if the specific
contributions are reasonable?


> We came along different views around the usage of "expressions"
> in previous discussions, didn't we?

Whom do you mean with "we"? Is it important in this context?

> You repeated the explanation "An expression is something that has a value."
> which is reasonable to some degree. I would like to know then:
> Which "value" is provided by the software construct "idexpression"
> in the semantic patch language?

Do you think you will be able to find out the answer to this question?
How could we best assist you in that?


Best,
-Nikolaus

-- 
GPG encrypted emails preferred. Key id: 0xD113FCAC3C4E599F
Fingerprint: ED31 791B 2C5C 1613 AF38 8B8A D113 FCAC 3C4E 599F

 »Time flies like an arrow, fruit flies like a Banana.«
___
Cocci mailing list
Cocci@systeme.lip6.fr
https://systeme.lip6.fr/mailman/listinfo/cocci

Re: [Cocci] Usage of "expressions" and "identifiers" with SmPL

[Julia Lawall]

On Sat, 8 Oct 2016, SF Markus Elfring wrote:

> > I'm not angry at all,
>
> Thanks for your feedback that there are other emotions involved.
>
>
> > only tired of so much incorrect
>
> Which details from my description approach are inappropriate in your view?

The whole numbered list was pointless.  He was trying to understand what
is meant by "idexpression".  This is a concept particular to Coccinelle,
that was introduced because we discovered that sometimes one wants to
focus only on identifiers, but to also specify their types.  I couldn't
tell from your answer what question it was intended to be relevant to, but
I don't think it is the same as the one he was asking.  It seems to be a
generic overview of what Coccinelle does.  Someone who has to read so much
text and in the end has no information about the question he was asking
will not likely get a good impression about the software he is trying to
use.

julia
___
Cocci mailing list
Cocci@systeme.lip6.fr
https://systeme.lip6.fr/mailman/listinfo/cocci

Re: [Cocci] Usage of "expressions" and "identifiers" with SmPL

[SF Markus Elfring]

> I'm not angry at all,

Thanks for your feedback that there are other emotions involved.


> only tired of so much incorrect

Which details from my description approach are inappropriate in your view?


> or unnecessary information being provided to new users.

I got the impression somehow that the clarification request by Nikolaus Rath 
evolved into a direction where further technical background information
would be useful.
He can also decide on his own if and how he would eventually like to
continue this software development and user support discussion.

Regards,
Markus
___
Cocci mailing list
Cocci@systeme.lip6.fr
https://systeme.lip6.fr/mailman/listinfo/cocci

Re: [Cocci] Usage of "expressions" and "identifiers" with SmPL

[Julia Lawall]

> > The next time you answer a question from a newcomer where your answer
> > either includes a question or goes beyond say 3 lines of text, I will
> > adjust your subscription to the mailing list such that you no longer
> > receive mail.
>
> Did you really get so angry that you reconsider our collaboration in such
> a drastic way?

I'm not angry at all, only tired of so much incorrect or unnecessary
information being provided to new users.  It's not practical to write to
each new user and say "Please ignore this message".  I would prefer that
they just stop in the first place.

julia
___
Cocci mailing list
Cocci@systeme.lip6.fr
https://systeme.lip6.fr/mailman/listinfo/cocci

Re: [Cocci] Usage of "expressions" and "identifiers" with SmPL

[SF Markus Elfring]

>>* Syntax constructs from the supported source languages are matched with
>>  "key words" that are often similar (or even identical) to the host 
>> language
>>  (within SmPL scripts).
> 
> This is an enormous amount of verbiage,

Interesting view …


> and I don't see the relevance to the question.

I am trying to explain the matching from syntax elements of the
C programming language to the means of the semantic patch language
from my view.


> Furthermore, I would very much appreciate it if you would stop responding
> to any questions from new users.

I find this kind of response surprising.

Would you eventually like to benefit from additional views which evolved
during collaboration of some years?


> You are not the developer or maintainer of Coccinelle.

This information is appropriate to some degree.

How do you think about the evolution around roles like "intensive SmPL user",
"system tester", "bug reporter" and "supporter"?

* I am trying to contribute a bit also to your software.

* I am active as another ordinary free software for years.


> You don't represent the tool in any way.

I do not represent it "officially". - But I can imagine that my software
development activities have got some useful "side effects" from
which a kind of "image" or "representation" could have evolved already.


> A very large portion of the information that you provide is incorrect

Would you like to clarify this a bit more?


> or unnecessary.

I got an other impression about the necessity.


> I am very concerned that you will frighten people away,

I can understand your concern a bit.


> especially if you ask them to do things that go beyond their
> intended involvement, such as your "Do you see further chances to
> reduce..." comment below.

The involvement can eventually increase, can't it?


> The next time you answer a question from a newcomer where your answer
> either includes a question or goes beyond say 3 lines of text, I will
> adjust your subscription to the mailing list such that you no longer
> receive mail.

Did you really get so angry that you reconsider our collaboration in such
a drastic way?


> I'm sure you consider that you are helping out,

I hope so.

Did my small script examples help a bit in the requested clarification?


> but you are not,

I find this feedback interesting and surprising.


> and it needs to stop immediately.

How can software development discussions be continued after such a response?

Regards,
Markus
___
Cocci mailing list
Cocci@systeme.lip6.fr
https://systeme.lip6.fr/mailman/listinfo/cocci

Re: [Cocci] Usage of "expressions" and "identifiers" with SmPL

[SF Markus Elfring]

> To me, an identifier is something that's written literally into the
> source code and cannot be meaningfully taken apart, e.g. a function
> name, variable name, or the member of a struct. An expression, on the
> other hand, is something that can be meaningfully split into
> sub-components. Is that also how you use these terms?

Yes, in principle.

The Coccinelle software was designed for the generation of semantic patches.

1. Its tool "spatch" expects some input and will usually produce
   corresponding output.

2. One kind of such input are source files for which data processing
   is also directly supported if they were mostly written in a programming
   language like "C".

3. Another kind of required input is the specification of source code search
   or transformation patterns in the semantic patch language.

4. Special data processing is also possible just because the programming
   languages "OCaml" and "Python" can be used in SmPL script rules already.

5. There are several languages involved. SmPL script developers need to be
   careful about the relevant syntax context.

   * So while you are looking from a view of "C source", you might tend
 to think about "C identifiers". The semantic patch language provides
 metavariables which can get the data type "identifier".

   * But what was an "item" in the source language can become an other
 in the Coccinelle technology.
 One example is the use of a metavariable with the type "idexpression"
 in your case. At which places would you start to think about a code
 situation by the means of "expressions"?

   * Syntax constructs from the supported source languages are matched with
 "key words" that are often similar (or even identical) to the host language
 (within SmPL scripts).


>> @ connects patterns that match the same term.  So match a term against the
>> explicit name ptr and also match it against an identifier expression that
>> has a particular type.  This is not exactly a beginner example.
> 
> Okay, I'll just ignore that for now.
> 
>>> I'd hate to waste your time asking tons of such trivial questions on the
>>> mailing list, but I just can't find any helpful documentation at all...
>>
>> If you look on the web page in the papers and slides section, at the top
>> there are several tutorials and overview talks, some with video.
> 
> I think I looked at everything that is not a video.

Thanks for your feedback.


> But none of it mentioned that you can put an arbitrary name between the @@

I got an other impression from the available documentation.


> or explained what the different metavariable types (idexpression,
> expression, etc) are.

I see some improvement possibilities there, too.


> Anyway, enough whining. Coccinelle seems like a really useful tool,

Yes, of course.


> even if I'm having an impedance mismatch with its documentation.

Do you see further chances to reduce this mismatch for following
software developers?

Regards,
Markus
___
Cocci mailing list
Cocci@systeme.lip6.fr
https://systeme.lip6.fr/mailman/listinfo/cocci