[Cocci] parsing of C code
Hello,
I have tried to improve the parsing of C code recently. The main changes,
currently available on github, are as follows:
1. More aggressive inclusion of header files, combined with caching of
header files. Now if there is only one occurrence of a header file with a
given name in the provided include paths, it will take that one, even if
there is no obvious connection between the location of the .c file and the
location of the header file. This compensates for the lact of parsing of
Makefiles to extract -I options. More header files will likely now be
included, particularly with options like --all-includes or
--recursive-includes. But caching of previously parsed header files has
been reinstated, which improves performance. This had been removed
because it wasn't doing nested includes, even if the --recursive-includes
option was provided, but that issue has been addressed.
2. If there is a parse error within the arguments of a function call, the
arguments are ignored, but not the entire enclosing functions definition,
as was done previously. For the Linux kernel, this seemed to allow
thousands of extra lines of code to be parsed and matched by Coccinelle.
I have only tested this on the Linux kernel. If you are using some other
software, you can run the following semantic patch on your software using
your current and the new versions of Coccinelle:
@r@
identifier f;
position p;
@@
f@p(...) { ... }
@script:ocaml@
f << r.f;
p << r.p;
@@
Printf.printf "%s:%d: %s\n" (List.hd p).file (List.hd p).line f;
flush stdout
If some functions are missing in the output when using the new version, as
compared to the output when using the old version, and if these functions
are things you might want to process in some way, thenlet me know about
the problem.
thanks,
julia
___
Cocci mailing list
Cocci@systeme.lip6.fr
https://systeme.lip6.fr/mailman/listinfo/cocci
Re: [Cocci] Comparing statement lists with SmPL
>>> The pattern matched in two different functions? >> >> Please look once more. >> >> Yes. - This is one of the reasons why I ask here again. >> >> I would like to choose if a code analysis should happen only within a single >> function >> or on interesting parts from the complete source file. > > Feel free to make a tool of your own if you are not satisfied with the one > offered you. This is also possible. - But I wonder more about specific test results based on the currently provided software. I got matches for places where I would expect that they should not match. > Coccinelle applies a rule to one function at a time. This is also fine for a while. > That is not going to change. I imagine that the situation could evolve more if additional development resources will appear somehow. > If you want to match things in different functions, make two rules. Additional SmPL rules will search for other details. Regards, Markus ___ Cocci mailing list Cocci@systeme.lip6.fr https://systeme.lip6.fr/mailman/listinfo/cocci
Re: [Cocci] Comparing statement lists with SmPL
>> How do the presented functions “usb6fire_fw_ezusb_upload” and >> “usb6fire_fw_fpga_upload” >> fit to this information? > > The pattern matched in two different functions? Please look once more. Yes. - This is one of the reasons why I ask here again. I would like to choose if a code analysis should happen only within a single function or on interesting parts from the complete source file. Regards, Markus ___ Cocci mailing list Cocci@systeme.lip6.fr https://systeme.lip6.fr/mailman/listinfo/cocci
Re: [Cocci] Comparing statement lists with SmPL
On Tue, 22 Aug 2017, SF Markus Elfring wrote: > >> * A test result was shown from two function implementations. > >> It can be nice to compare several functions. > >> Is it also possible that the comparison will be only performed within > >> the same function bodies? > > > > If you make a single rule then it will only applied within individual > > functions. > > How do the presented functions “usb6fire_fw_ezusb_upload” and > “usb6fire_fw_fpga_upload” > fit to this information? The pattern matched in two different functions? julia___ Cocci mailing list Cocci@systeme.lip6.fr https://systeme.lip6.fr/mailman/listinfo/cocci
Re: [Cocci] Comparing statement lists with SmPL
>> * A test result was shown from two function implementations. >> It can be nice to compare several functions. >> Is it also possible that the comparison will be only performed within >> the same function bodies? > > If you make a single rule then it will only applied within individual > functions. How do the presented functions “usb6fire_fw_ezusb_upload” and “usb6fire_fw_fpga_upload” fit to this information? Regards, Markus ___ Cocci mailing list Cocci@systeme.lip6.fr https://systeme.lip6.fr/mailman/listinfo/cocci
Re: [Cocci] Comparing statement lists with SmPL
>> How would you like to treat variations in such log messages? > > They look fine as they are. I have got other development opinions there depending on the preferred design goals. * The mentioned small SmPL script can also help to find special differences. * These log statements are similar (and not identical) in the code structure. I wonder also about other marked code places how they would fit to the search pattern. * A test result was shown from two function implementations. It can be nice to compare several functions. Is it also possible that the comparison will be only performed within the same function bodies? Regards, Markus ___ Cocci mailing list Cocci@systeme.lip6.fr https://systeme.lip6.fr/mailman/listinfo/cocci
Re: [Cocci] Comparing statement lists with SmPL
On Tue, 22 Aug 2017, SF Markus Elfring wrote:
> >> * Are there further development challenges to consider for the safe
> >> identification
> >> of unique statements by metavariables?
> >
> > If you want to ensure that two metavariables match things
> > in different places, then put a position variable on each match
> > and use apython rule afterwards to discard the occurrences that
> > are both in the same position.
>
> I imagine that such an approach can only matter if there will be
> several SmPL rules involved.
>
>
> I have tried another variant out for a source code analysis.
>
> @duplicated_code@
> identifier work;
> statement s1, s2;
> type T;
> @@
> T work(...)
> {
> <+...
> *if ((...) < 0)
> *{
> ...
> * s1
> * s2
> *}
> ...+>
> <+...
> *if ((...) < 0)
> *{
> ...
> * s1
> * s2
> *}
> ...+>
> }
>
>
> elfring@Sonne:~/Projekte/Linux/next-patched> XX=$(date) && spatch.opt
> -timeout 34 -j 4 --chunksize 1 --sp-file
> ~/Projekte/Coccinelle/janitor/show_same_statements3.cocci --dir sound >
> ~/Projekte/Bau/Linux/scripts/Coccinelle/tuning1/next/20170803/same_statements3+.diff
> 2>
> ~/Projekte/Bau/Linux/scripts/Coccinelle/tuning1/next/20170803/same_statements3+-errors.txt;
> YY=$(date) && echo "$XX | $YY"
> Di 22. Aug 10:24:47 CEST 2017 | Di 22. Aug 10:26:04 CEST 2017
>
>
> Now I find that a test result like the following is worth for further
> development considerations.
>
> …
> --- sound/usb/6fire/firmware.c
> +++ /tmp/nothing/usb/6fire/firmware.c
> @@ -246,14 +246,8 @@ static int usb6fire_fw_ezusb_upload(
> while (usb6fire_fw_ihex_next_record(rec)) { /* write firmware */
> ret = usb6fire_fw_ezusb_write(device, 0xa0, rec->address,
> rec->data, rec->len);
> - if (ret < 0) {
> kfree(rec);
> release_firmware(fw);
> - dev_err(>dev,
> - "unable to upload ezusb firmware %s: data
> urb.\n",
> - fwname);
> - return ret;
> - }
> }
>
> release_firmware(fw);
> @@ -319,13 +313,8 @@ static int usb6fire_fw_fpga_upload(
> buffer[i] = bitrev8((u8)*c);
>
> ret = usb6fire_fw_fpga_write(device, buffer, i);
> - if (ret < 0) {
> release_firmware(fw);
> kfree(buffer);
> - dev_err(>dev,
> - "unable to upload fpga firmware: fw urb.\n");
> - return ret;
> - }
> }
> release_firmware(fw);
> kfree(buffer);
> …
>
>
> How would you like to treat variations in such log messages?
They look fine as they are.
julia___
Cocci mailing list
Cocci@systeme.lip6.fr
https://systeme.lip6.fr/mailman/listinfo/cocci
Re: [Cocci] Comparing statement lists with SmPL
>> * Are there further development challenges to consider for the safe
>> identification
>> of unique statements by metavariables?
>
> If you want to ensure that two metavariables match things
> in different places, then put a position variable on each match
> and use apython rule afterwards to discard the occurrences that
> are both in the same position.
I imagine that such an approach can only matter if there will be
several SmPL rules involved.
I have tried another variant out for a source code analysis.
@duplicated_code@
identifier work;
statement s1, s2;
type T;
@@
T work(...)
{
<+...
*if ((...) < 0)
*{
...
* s1
* s2
*}
...+>
<+...
*if ((...) < 0)
*{
...
* s1
* s2
*}
...+>
}
elfring@Sonne:~/Projekte/Linux/next-patched> XX=$(date) && spatch.opt -timeout
34 -j 4 --chunksize 1 --sp-file
~/Projekte/Coccinelle/janitor/show_same_statements3.cocci --dir sound >
~/Projekte/Bau/Linux/scripts/Coccinelle/tuning1/next/20170803/same_statements3+.diff
2>
~/Projekte/Bau/Linux/scripts/Coccinelle/tuning1/next/20170803/same_statements3+-errors.txt;
YY=$(date) && echo "$XX | $YY"
Di 22. Aug 10:24:47 CEST 2017 | Di 22. Aug 10:26:04 CEST 2017
Now I find that a test result like the following is worth for further
development considerations.
…
--- sound/usb/6fire/firmware.c
+++ /tmp/nothing/usb/6fire/firmware.c
@@ -246,14 +246,8 @@ static int usb6fire_fw_ezusb_upload(
while (usb6fire_fw_ihex_next_record(rec)) { /* write firmware */
ret = usb6fire_fw_ezusb_write(device, 0xa0, rec->address,
rec->data, rec->len);
- if (ret < 0) {
kfree(rec);
release_firmware(fw);
- dev_err(>dev,
- "unable to upload ezusb firmware %s: data
urb.\n",
- fwname);
- return ret;
- }
}
release_firmware(fw);
@@ -319,13 +313,8 @@ static int usb6fire_fw_fpga_upload(
buffer[i] = bitrev8((u8)*c);
ret = usb6fire_fw_fpga_write(device, buffer, i);
- if (ret < 0) {
release_firmware(fw);
kfree(buffer);
- dev_err(>dev,
- "unable to upload fpga firmware: fw urb.\n");
- return ret;
- }
}
release_firmware(fw);
kfree(buffer);
…
How would you like to treat variations in such log messages?
Regards,
Markus
___
Cocci mailing list
Cocci@systeme.lip6.fr
https://systeme.lip6.fr/mailman/listinfo/cocci
