Re: [Cocci] Comparing statement lists with SmPL
> Unfortunately, the software version “1.0.6-00186-g0acd38ee” does not like this
> SmPL specification.
I overlooked to add parentheses in my evolving SmPL script.
@duplicated_code@
identifier work;
statement s1, s2;
statement list sl;
type T;
@@
T work(...)
{
... when any
(
*if ((...) < 0) {
* s1
* s2
...
*}
&{
sl
}
)
...
(
*if ((...) < 0) {
* s1
* s2
...
*}
&{
sl
}
)
...
}
The following command does not show the expected source code analysis results
at the moment.
elfring@Sonne:~/Projekte/Coccinelle/janitor> spatch.opt
show_same_statements2.cocci ../Probe/hdsp-excerpt1.c
Is there any more clarification needed for the use of SmPL conjunctions?
Regards,
Markus
___
Cocci mailing list
Cocci@systeme.lip6.fr
https://systeme.lip6.fr/mailman/listinfo/cocci
Re: [Cocci] Comparing statement lists with SmPL
>> Can the search for duplicated source code be improved by the means of the
>> semantic patch language?
>
> For two statements at least you could do:
>
> (
> {
> s1
> s2
> ...
> }
> &
> {
> sl
> }
> )
I constructed another SmPL script based on this suggestion as follows.
@duplicated_code@
identifier work;
statement s1, s2;
statement list sl;
type T;
@@
T work(...)
{
... when any
*if ((...) < 0) {
* s1
* s2
...
*}
&{
sl
}
...
*if ((...) < 0) {
* s1
* s2
...
*}
&{
sl
}
...
}
Unfortunately, the software version “1.0.6-00186-g0acd38ee” does not like this
SmPL specification.
elfring@Sonne:~/Projekte/Coccinelle/janitor> spatch.opt --parse-cocci
show_same_statements2.cocci
…
minus: parse error:
File "show_same_statements2.cocci", line 15, column 0, charpos = 156
around = '&',
whole content = &{
How can my understanding become better also for the use of SmPL conjunctions?
Regards,
Markus
___
Cocci mailing list
Cocci@systeme.lip6.fr
https://systeme.lip6.fr/mailman/listinfo/cocci
Re: [Cocci] Comparing statement lists with SmPL
>> Now I am looking for a way to express the constraint that the statement list
>> metavariable should match only source code with two statements at least.
>> Can the search for duplicated source code be improved by the means of the
>> semantic patch language?
>
> For two statements at least you could do:
>
> (
> {
> s1
> s2
> ...
> }
> &
> {
> sl
> }
> )
Thanks for another example.
I admit that I am unused to such a variable combination with a SmPL conjunction.
Will this programming interface evolve any further?
Regards,
Markus
___
Cocci mailing list
Cocci@systeme.lip6.fr
https://systeme.lip6.fr/mailman/listinfo/cocci
Re: [Cocci] Comparing statement lists with SmPL
> The source code analysis results look promising by this command.
I have just noticed that an other source file points details out for further
development considerations around the mentioned functionality from the
Coccinelle software.
elfring@Sonne:~/Projekte/Linux/next-patched> git checkout next-20170803 &&
spatch.opt ~/Projekte/Coccinelle/janitor/show_same_statements1.cocci
sound/pci/rme9652/hdspm.c
…
(ONCE) already tagged but only removed, so safe
diff =
…
@@ -6497,9 +6497,6 @@ static int snd_hdspm_create_alsa_devices
i = 0;
while (i < hdspm->midiPorts) {
err = snd_hdspm_create_midi(card, hdspm, i);
- if (err < 0) {
- return err;
- }
i++;
}
@@ -6955,10 +6952,6 @@ static int snd_hdspm_probe(struct pci_de
hdspm->pci = pci;
err = snd_hdspm_create(card, hdspm);
- if (err < 0) {
- snd_card_free(card);
- return err;
- }
if (hdspm->io_type != MADIface) {
snprintf(card->shortname, sizeof(card->shortname), "%s_%x",
…
Now I am looking for a way to express the constraint that the statement list
metavariable should match only source code with two statements at least.
Can the search for duplicated source code be improved by the means of the
semantic patch language?
Regards,
Markus
___
Cocci mailing list
Cocci@systeme.lip6.fr
https://systeme.lip6.fr/mailman/listinfo/cocci
Re: [Cocci] Comparing statement lists with SmPL
>> * How can be determined how many code is equivalent between two statement >> list variables? > > I guess you can make special cases, for one statement, two statements, etc. I guess that we would like to avoid such an approach for some use cases. I hope that there are safer programming possibilities available. Regards, Markus ___ Cocci mailing list Cocci@systeme.lip6.fr https://systeme.lip6.fr/mailman/listinfo/cocci
Re: [Cocci] Comparing statement lists with SmPL
>> Is this programming interface usable to determine if any code could be
>> combined
>> to some degree?
>
> I don't really understand the questions.
I try again to describe the use case I became interested in a bit more.
> Statement list metavariables can be used once they are bound like any other
> metavariables.
Can it be that they will need any special computing power occasionally?
I have tried the following small SmPL script out.
@duplicated_code@
identifier work;
statement list sl;
type T;
@@
T work(...)
{
... when any
*if ((...) < 0) {
* sl
*}
...
*if ((...) < 0) {
* sl
*}
...
}
The source code analysis results look promising by this command.
elfring@Sonne:~/Projekte/Linux/next-patched> git checkout next-20170803 &&
spatch.opt ~/Projekte/Coccinelle/janitor/show_same_statements1.cocci
sound/pci/rme9652/hdsp.c
…
@@ -793,11 +793,6 @@ static int hdsp_get_iobox_version (struc
…
@@ -5382,19 +5372,11 @@ static int snd_hdsp_probe(struct pci_dev
…
How should such update candidates be transformed further?
> However they can only be bound to the complete sequence of statements in a
> block. So
>
> @@
> statement list sl;
> @@
> if (x) {
> sl
> }
>
> is ok, but
>
> @@
> statement list sl;
> @@
> if (x) {
> one();
> sl
> two();
> }
>
> is not.
Thanks for your example.
* I do not like the mentioned software restriction at the moment.
* How can be determined how many code is equivalent between two statement list
variables?
Regards,
Markus
___
Cocci mailing list
Cocci@systeme.lip6.fr
https://systeme.lip6.fr/mailman/listinfo/cocci
[Cocci] Comparing statement lists with SmPL
Hello, The Coccinelle software supports metavariables with the type “statement list”. Can statement lists be compared if they refer to the same source code structures? Is this programming interface usable to determine if any code could be combined to some degree? Regards, Markus ___ Cocci mailing list Cocci@systeme.lip6.fr https://systeme.lip6.fr/mailman/listinfo/cocci
Re: [Cocci] Coccinelle: add atomic_as_refcounter script
> Elena, please don't follow Markus's suggestion. I suggest to consider additional software development possibilities. We have got recurring different opinions around the application of advanced regular expressions. > Coccinelle doesn't interpret regular expressions when selecting code to work > on, This software behaviour can be fine. > and thus the resulting rule will be less efficient. I do not come to the same conclusion for some use cases. > Markus, if you see this somehow, Yes. - But I can read your response only by the web interface for your mailing list archive at the moment. https://systeme.lip6.fr/pipermail/cocci/2017-August/004308.html > at least yesterday all mail sent to you was bouncing. Thanks for this information. - I was not informed about such message exchange problems so far. I hope that communication difficulties will be fixed somehow. Regards, Markus ___ Cocci mailing list Cocci@systeme.lip6.fr https://systeme.lip6.fr/mailman/listinfo/cocci
Re: [Cocci] Coccinelle: add atomic_as_refcounter script
Dear Elena, Would you like to take another look at my development suggestion for your approach? https://systeme.lip6.fr/pipermail/cocci/2017-August/004300.html How do you think about to refactor another rule like the following? @r3 exists@ expression E; identifier I=~"^atomic(?:64|_long)?_add_return$"; position P; @@ E = I@P(-1, ...); Can the specification of SmPL constraints be occasionally be more succinct with the use of regular expressions? Regards, Markus ___ Cocci mailing list Cocci@systeme.lip6.fr https://systeme.lip6.fr/mailman/listinfo/cocci
Re: [Cocci] add atomic_as_refcounter script
Dear Elena, I have also taken another look at your script. The SmPL rule “r1” specifies six metavariables for identifiers. It seems that the names “fname3” till “fname6” do not care for different function parameters in the SmPL disjunction at the end. How do you think about to reduce this disjunction by using a regular expression with an alternation like “.*(?:call_rcu|del|queue_work|schedule_work).*” for a constraint? Will it help to refactor another rule a bit? @r3 exists@ identifier x; position p1; @@ x = ( atomic_add_return at p1(-1, ...); | atomic_long_add_return at p1(-1, ...); | atomic64_add_return at p1(-1, ...); ) Regards, Markus ___ Cocci mailing list Cocci@systeme.lip6.fr https://systeme.lip6.fr/mailman/listinfo/cocci
Re: [Cocci] Replacing suffix rules in make scripts?
> So I get the impression that an other build approach will be safer there. > How do you think about to add a dependency specification so that interface > descriptions will be always compiled before the corresponding OCaml code? I showed change possibilities which can improve the software situation to some degree. * Improve several dependency specifications in make rules https://github.com/coccinelle/coccinelle/pull/108 * Fix specifications for build dependencies between shared components https://github.com/coccinelle/coccinelle/issues/110 * Support references for the top-level source file directory in build scripts https://github.com/coccinelle/coccinelle/issues/104 Which ideas would you like to integrate from the available development approaches? Regards, Markus ___ Cocci mailing list Cocci@systeme.lip6.fr https://systeme.lip6.fr/mailman/listinfo/cocci
Re: [Cocci] Replacing suffix rules in make scripts?
> Why do I get such an error message for a parallel software build attempt > when the specified dependencies worked as expected for the non-parallel > build try before? How do you think about the corresponding update suggestion “Improve some dependency specifications in make rules”? https://github.com/elfring/Coccinelle-20160205/commit/9597598779f3604272621f6d42ead79eaafad2a8 Regards, Markus ___ Cocci mailing list Cocci@systeme.lip6.fr https://systeme.lip6.fr/mailman/listinfo/cocci
Re: [Cocci] Replacing suffix rules in make scripts?
> The OCaml makefile specifies an order in which the files should be > compiled in the project. If you are sidestepping that ordering, then you > are not doing things in the intended manner. I hope that some details were clarified to some degree in the meantime. > There is only an intent to support running make or make opt in the > subdirectories or at the root. Perhaps Thierry will put a new make file > that has some other properties (I don't know which ones) in the future. > But there is no intent to support anything else at the moment. I suggest to take also another look around the discussed build scripts. Example: https://github.com/coccinelle/coccinelle/blob/cd539cd12ad0674d5a8f4c33f6c21ae00aec8e57/globals/Makefile#L50 Now I wonder about the following test result for my current “developer rules”. elfring@Sonne:~/Projekte/Coccinelle/20160205/globals> make clean && LANG=C make --no-builtin-rules V=1 && make clean && LANG=C make --no-builtin-rules -j4 V=1 rm -f … /usr/bin/ocamldep -I ../commons -I /home/elfring/Projekte/Coccinelle/20160205/bundles/pcre/ *.mli *.ml > .depend … /usr/bin/ocamlc.opt -unsafe -I ../commons -I /home/elfring/Projekte/Coccinelle/20160205/bundles/pcre/ -c flag.ml … /usr/bin/ocamlc.opt -unsafe -I ../commons -I /home/elfring/Projekte/Coccinelle/20160205/bundles/pcre/ -a -o globals.cma config.cmo flag.cmo regexp_pcre.cmo regexp.cmo iteration.cmo rm -f … /usr/bin/ocamldep -I ../commons -I /home/elfring/Projekte/Coccinelle/20160205/bundles/pcre/ *.mli *.ml > .depend … /usr/bin/ocamlc.opt -unsafe -I ../commons -I /home/elfring/Projekte/Coccinelle/20160205/bundles/pcre/ -c flag.ml … /usr/bin/ocamlc.opt -unsafe -I ../commons -I /home/elfring/Projekte/Coccinelle/20160205/bundles/pcre/ -c iteration.mli make: *** No rule to make target 'flag.cmi', needed by 'iteration.cmo'. Stop. … Why do I get such an error message for a parallel software build attempt when the specified dependencies worked as expected for the non-parallel build try before? Do you get any more ideas on how to explain such a test surprise? Regards, Markus ___ Cocci mailing list Cocci@systeme.lip6.fr https://systeme.lip6.fr/mailman/listinfo/cocci
Re: [Cocci] Replacing suffix rules in make scripts?
> OK, I don't undertand what command you are doing to get the problem, Did we achieve a better common understanding in the meantime? > so I can't reproduce it, so I can't determine whether it is a real issue. * Are you really keen to achieve this also in your own test environments? * How much does my build difficulty fit into your development attention and software change capacity? > The OCaml makefile specifies an order in which the files should be > compiled in the project. If you are sidestepping that ordering, then you > are not doing things in the intended manner. I am just curious to fix the dependency checking (without unexpected side effects) for build prerequisites somehow in the involved software components. Regards, Markus ___ Cocci mailing list Cocci@systeme.lip6.fr https://systeme.lip6.fr/mailman/listinfo/cocci
Re: [Cocci] Replacing suffix rules in make scripts?
> The command “ocamldep *.mli *.ml > .depend” was performed for > relevant subdirectories. > Unfortunately, this approach seems to be insufficient for the desired > safe integration of dependency information so far. I have got another idea. The adjustment results were more promising for my build script changes in other subdirectories. Small make functions construct extra rules there. If the used make rules from the ocamldep call do not produce the desired effects for checks of prerequisites so far, I could try to convert the provided data to an other style of dependency specification. Such generated make rules would finally contain one useful command in each recipe. Regards, Markus ___ Cocci mailing list Cocci@systeme.lip6.fr https://systeme.lip6.fr/mailman/listinfo/cocci
Re: [Cocci] Replacing suffix rules in make scripts?
>> Can you get the impression from the mentioned error message examples >> that the desired software build prerequisites failed anyhow? > > Something fails in your situation, but without knowing the arguments to > the compiler, I don't know what. You could know also from the reduced information. Is it really easier for you to see the following data again? elfring@Sonne:~/Projekte/Coccinelle/20160205/parsing_c> LANG=C make V=1 /usr/bin/ocamlc.opt -unsafe -I ../commons -I ../commons/ocamlextra -I ../globals -I ../parsing_cocci -c includes.mli File "includes.mli", line 38, characters 46-60: Error: Unbound module Ast_c make: *** [/home/elfring/Projekte/Coccinelle/20160205/common_rules.make:6: includes.cmi] Error 2 Regards, Markus ___ Cocci mailing list Cocci@systeme.lip6.fr https://systeme.lip6.fr/mailman/listinfo/cocci
Re: [Cocci] Replacing suffix rules in make scripts?
> The command “ocamldep *.mli *.ml > .depend” was performed for > relevant subdirectories. > Unfortunately, this approach seems to be insufficient for the desired > safe integration of dependency information so far. How do you think about to work any more with dedicated files for each source file by pattern rules like it is described in the manual section “Generating prerequisites automatically”? https://www.gnu.org/software/make/manual/html_node/Automatic-Prerequisites.html Would it make sense to split the data from the command “ocamldep” into smaller units? Do I stumble on unexpected software behaviour by the tool “GNU Make 4.2.1-1.7” on my openSUSE Tumbleweed system? Regards, Markus ___ Cocci mailing list Cocci@systeme.lip6.fr https://systeme.lip6.fr/mailman/listinfo/cocci
Re: [Cocci] Replacing suffix rules in make scripts?
>> I showed how I could stumble on open issues. > > No you didn't. I try then once more to help you to interpret the mentioned error message examples. The command “ocamldep *.mli *.ml > .depend” was performed for relevant subdirectories. Unfortunately, this approach seems to be insufficient for the desired safe integration of dependency information so far. > I don't know what are the arguments to the ocaml compiler. If you think in terms of the build system, you would know that this tool needs input files. Unfortunately, the current build scripts did not trigger the generation of the desired prerequisites for the command examples at the end. > What file are you trying to compile? The attempted data processing steps can not succeed when required files were not built before. If these dependencies will be fixed, then we can worry if an error message like “Unbound module Ast_c” can occur because of other reasons. Regards, Markus ___ Cocci mailing list Cocci@systeme.lip6.fr https://systeme.lip6.fr/mailman/listinfo/cocci
Re: [Cocci] Replacing suffix rules in make scripts?
>> I am curious on how a dependency for a software interface like >> “Visitor_ast0_types” >> should be expressed for so many source files in a consistent way. > > What problem are you trying to solve? I would like to achieve something more for the topics which were also registered in the issue tracker. > Why do the dependency specifications need improving? Examples: elfring@Sonne:~/Projekte/Coccinelle/20160205/parsing_cocci> LANG=C make V=1 /usr/bin/ocamldep *.mli *.ml > .depend /usr/bin/ocamlc.opt … File "_none_", line 1: Error: Cannot find file flag_parsing_cocci.cmo make: *** [Makefile:67: cocci_parser.cma] Error 2 elfring@Sonne:~/Projekte/Coccinelle/20160205/parsing_c> LANG=C make V=1 /usr/bin/ocamlc.opt … File "includes.mli", line 38, characters 46-60: Error: Unbound module Ast_c make: *** [/home/elfring/Projekte/Coccinelle/20160205/common_rules.make:6: includes.cmi] Error 2 These messages indicate reasons why I am also fiddling with the build scripts for this software again. Regards, Markus ___ Cocci mailing list Cocci@systeme.lip6.fr https://systeme.lip6.fr/mailman/listinfo/cocci
Re: [Cocci] Replacing suffix rules in make scripts?
> If you have changed things, then perhaps you have caused the problem? I have found an approach for the adjustment of affected build scripts where I can improve the involved dependency specifications in a few subdirectories on my own to some degree. But I guess that a directory like “parsing_cocci” provides bigger development challenges to make the dependencies safer there. I am curious on how a dependency for a software interface like “Visitor_ast0_types” should be expressed for so many source files in a consistent way. Regards, Markus ___ Cocci mailing list Cocci@systeme.lip6.fr https://systeme.lip6.fr/mailman/listinfo/cocci
Re: [Cocci] Replacing suffix rules in make scripts?
> If you have changed things, then perhaps you have caused the problem? This is also another possibility in principle. It seems that I wonder again why some source files should be compiled without interface description files there. An other script variant showed the following result. elfring@Sonne:~/Projekte/Coccinelle/20160205/globals> LANG=C make iteration.cmx V=1 My test command: /usr/bin/ocamlopt.opt -unsafe -I ../commons -I /home/elfring/Projekte/Coccinelle/20160205/bundles/pcre/ -c flag.ml File "flag.ml", line 24, characters 17-36: Error: Unbound module Commands make: *** [Makefile:62: flag.cmx] Error 2 Can it be that the corresponding dependency specifications were incomplete anyhow in this make script for the direct use at such a place? Would you like to help any more to get these dependencies more consistent? Regards, Markus ___ Cocci mailing list Cocci@systeme.lip6.fr https://systeme.lip6.fr/mailman/listinfo/cocci
Re: [Cocci] Replacing suffix rules in make scripts?
> So I get the impression that an other build approach will be safer there. > How do you think about to add a dependency specification so that interface > descriptions will be always compiled before the corresponding OCaml code? Hello, I have achieved another bit of progress with my attempt to replace suffix rules by pattern rules in some make scripts. I stumbled on the next software development challenge as follows. elfring@Sonne:~/Projekte/Coccinelle/20160205> LANG=C make V=1 … make[6]: Entering directory '/home/elfring/Projekte/Coccinelle/20160205/globals' /usr/bin/ocamlopt.opt -unsafe -I ../commons -I /home/elfring/Projekte/Coccinelle/20160205/bundles/pcre/ -c iteration.ml File "iteration.ml", line 36, characters 4-28: Error: Unbound module Flag make[6]: *** [/home/elfring/Projekte/Coccinelle/20160205/common_rules.make:12: iteration.cmx] Error 2 … * Do I overlook an “obvious” software dependency once more in this use case? * How should the relevant details be resolved here? Regards, Markus ___ Cocci mailing list Cocci@systeme.lip6.fr https://systeme.lip6.fr/mailman/listinfo/cocci
[Cocci] Replacing suffix rules in make scripts?
Hello, I have achieved a bit of progress with one experiment for build script adjustments. I would like to try further changes out. Unfortunately, I stumbled on an error message like the following from the program “GNU Make 4.2.1-1.7” then. … /usr/bin/ocamlopt.opt … File "cocci.ml", line 1: Error: Could not find the .cmi file for interface cocci.mli. make[3]: *** [Makefile:533: cocci.cmx] Error 2 … It seems that the following code is relevant in this use case. … .ml.cmo: $(OCAMLC_CMD) -c $< .mli.cmi: $(OCAMLC_CMD) -c $< .ml.cmx: $(OCAMLOPT_CMD) -c $< … Other documentation provides this information: “… Suffix rules cannot have any prerequisites of their own. …” So I get the impression that an other build approach will be safer there. How do you think about to add a dependency specification so that interface descriptions will be always compiled before the corresponding OCaml code? Regards, Markus ___ Cocci mailing list Cocci@systeme.lip6.fr https://systeme.lip6.fr/mailman/listinfo/cocci
Re: [Cocci] Easier search for assignment targets with SmPL
>> Can it become safer and more convenient to handle assignments here? > > I have no idea what you are asking for. I wonder about such a kind of feedback. > In any case, an expression metavariable already matches a variable. This information can be appropriate in some source code situations. Do you care for any more “special use cases”? > I don't know what a designated initializer is. I find this aspect strange when it was already discussed several times. http://en.cppreference.com/w/c/language/struct_initialization How would you prefer to handle various software development challenges around specifications for variable definitions? Regards, Markus ___ Cocci mailing list Cocci@systeme.lip6.fr https://systeme.lip6.fr/mailman/listinfo/cocci
Re: [Cocci] Easier search for assignment targets with SmPL
> var = action(...); This specification looks also simple for the semantic patch language at first glance. I tried to express that a specific value is assigned to something. But this “something” could be a different item depending on the position in the source code. 1. Variable 2. Expression 3. Element for a designated initialiser I would appreciate if I do not need to repeat specifications for these cases in SmPL code. How do you think about the possibility to support the desired source code search by an additional metavariable type there? Can it become safer and more convenient to handle assignments here? Regards, Markus ___ Cocci mailing list Cocci@systeme.lip6.fr https://systeme.lip6.fr/mailman/listinfo/cocci
Re: [Cocci] Checking failure predicates with SmPL
> var = action(...); > if (failed(var)) This SmPL specification contains a few further software development challenges. 1. It could be taken into account that a function call and a corresponding check for the return value are not directly next to each other. A SmPL ellipsis could be added there. But I guess that special restrictions will be needed then for a safe “when specification”. 2. The check which should appropriately be performed for a return code depends on the called function. * Each failure predicate could be written as a separate SmPL rule. This approach would result into code duplication to some degree. * I would prefer to keep the selection and data processing for the desired predicate at a single place. But I am unsure so far how good such a variant is supported by the current Coccinelle software. Regards, Markus ___ Cocci mailing list Cocci@systeme.lip6.fr https://systeme.lip6.fr/mailman/listinfo/cocci
Re: [Cocci] Safe determination of function properties for SmPL scripts
> How can the Coccinelle software help any more in such an use case? This small SmPL script example demonstrates two special selections of functions by their name. They belong to separate categories then. * The primary actions should usually provide a return value somehow. * A call of a function which will not return from its final task is one possibility for an error reaction. Have you got any preferences for the determination and integration of name lists for functions with the desired properties into the semantic patch language in a safe way? Would you dare to query a database table (or view) for such a purpose? Regards, Markus ___ Cocci mailing list Cocci@systeme.lip6.fr https://systeme.lip6.fr/mailman/listinfo/cocci
Re: [Cocci] Checking source code (including optional parts) with SmPL
I would like to clarify the possibilities for corresponding software extensions a bit more. The Coccinelle software provides also the search operator “question mark” which denotes code on a specific line as optional. https://github.com/coccinelle/coccinelle/blob/9b5e00862900b0daadf1819a6894d31544a3ba34/docs/manual/cocci_syntax.tex#L1067 The scope of the special code there is only the marked line so that I see some software development challenges to pack desired search filters into one line. * How often do you check the handling for optional parts? - Do you find it relevant to check for the assignment of an error code to an other variable? - Are special constraints needed for additional function calls? * How are the chances to improve readability and maintainability for SmPL code? Regards, Markus ___ Cocci mailing list Cocci@systeme.lip6.fr https://systeme.lip6.fr/mailman/listinfo/cocci
[Cocci] Checking exception handling implementations with SmPL
Hello,
I would like to check the implementation of exception handling in various
source files. The following script for the semantic patch language tries to
express a few possibilities for such a source code search pattern.
@show_special_reactions@
expression error_code, var;
identifier action =~ "^(?:malloc|strdup)$",
no_return =~ "[Aa]bort",
label;
statement es;
@@
var = action(...);
if (failed(var))
{
(
* return error_code;
|
* goto label;
|
* no_return();
)
}
else
es
I find this approach too simple and therefore incomplete. Three well-known
reactions are shown. But there can be more source code before these statements
like further function calls and variable assignments.
We do eventually not know the extra code there and it can also be
generally optional.
How can the Coccinelle software help any more in such an use case?
Regards,
Markus
___
Cocci mailing list
Cocci@systeme.lip6.fr
https://systeme.lip6.fr/mailman/listinfo/cocci
Re: [Cocci] Easy search for elements besides binary operators with SmPL
>> I find it just strange that extra parentheses are needed in such an use case >> at the moment. > > ... is used in a number of contexts in SmPL, and sometimes its use has to > be restricted to avoid parsing ambiguities. I tried to use extra parentheses for other source code search patterns. I left them “accidentally” out with undesired consequences this time. > I'm a bit surprised also that it is a problem in this case. I find such a feedback interesting then if it will influence corresponding software evolution. Regards, Markus ___ Cocci mailing list Cocci@systeme.lip6.fr https://systeme.lip6.fr/mailman/listinfo/cocci
Re: [Cocci] Easy search for elements besides binary operators with SmPL
>>> Maybe put parentheses around the second argument. An isomorphism will >>> drop them and you will avoid whatever parsing issue is being encountered. >> >> I do not observe a desired improvement after such a SmPL code adjustment. > > Your change is not what I suggested either. The second argument starts to > the right of one comma and ends to the left of the next one. I interpreted the “second one” as belonging to the ellipsis in the binary operator specification (for a moment). Now I observe that the following SmPL script variants work as desired in principle. @find_last_option@ expression target; identifier action; @@ target = action(..., ( ... *| __GFP_NOFAIL ) ); @find_flag@ expression target; identifier action; @@ target = action(..., ( ... *| __GFP_NOFAIL | ... ), ...); I find it just strange that extra parentheses are needed in such an use case at the moment. Regards, Markus ___ Cocci mailing list Cocci@systeme.lip6.fr https://systeme.lip6.fr/mailman/listinfo/cocci
[Cocci] Easy search for elements besides binary operators with SmPL
Hello, I would like to check also the application of elements which are used together with binary operators like the following. @find@ expression target; identifier action; @@ target = action(..., ... *| __GFP_NOFAIL | ..., ...); elfring@Sonne:~/Projekte/Linux/next-patched> spatch.opt drivers/md/dm-region-hash.c ~/Projekte/Coccinelle/janitor/show_use_of_nofail1.cocci … minus: parse error: File "/home/elfring/Projekte/Coccinelle/janitor/show_use_of_nofail1.cocci", line 6, column 17, charpos = 87 around = '...', whole content = ... How should I improve my search pattern? A metavariable can already be used for the search of a function parameter at an undetermined position within a bigger list. Can similar functionality work also for binary operators? Regards, Markus ___ Cocci mailing list Cocci@systeme.lip6.fr https://systeme.lip6.fr/mailman/listinfo/cocci
Re: [Cocci] [PATCH v3] Coccinelle: Script to replace allocate and memset with zalloc functions
> This script finds instances of allocate and memset which can be > replaced with a direct call to zalloc equivalent of a function. What is the software development status for this SmPL script in comparison to an other directory for source code transformations? https://git.kernel.org/cgit/linux/kernel/git/next/linux-next.git/tree/scripts/coccinelle/api/alloc Regards, Markus ___ Cocci mailing list Cocci@systeme.lip6.fr https://systeme.lip6.fr/mailman/listinfo/cocci
Re: [Cocci] Replacing combination of kcalloc() and copy_from_user() by memdup_user() with SmPL?
> The comm_assoc isomorphism allows A || ... to match any disjunction where > A appears at the top level, not just in the far left position. Thanks for your explanation. Are there further software development challenges to consider around the handling of items like position variables and “mixed modifs”? > It also matches the case where A is not present at all. Will the bug report “Safer source code analysis by "memdup_user.cocci"” get another look? https://github.com/coccinelle/coccinelle/issues/78 Regards, Markus ___ Cocci mailing list Cocci@systeme.lip6.fr https://systeme.lip6.fr/mailman/listinfo/cocci
Re: [Cocci] Replacing combination of kcalloc() and copy_from_user() by memdup_user() with SmPL?
> elfring@Sonne:~/Projekte/Coccinelle/janitor> spatch.opt
> replace_kcalloc_by_memdup_user-draft1.cocci ../Probe/user_sdma-excerpt1.c
> …
> refactoring: position variables or mixed modifs interfere with comm_assoc iso
> bool (bool !to
> >>> IS_ERR(to)
> || ...)
> …
Would you like to explain this message a bit more while the following SmPL
approach seems to work
as expected?
@refactoring@
expression code, count, from, to, size;
identifier label, log;
@@
to =
- kcalloc(count, size, GFP_KERNEL)
+ memdup_user(from, size * count)
;
if (
-!to
+IS_ERR(to)
) {
- code = -ENOMEM;
- goto label;
-}
-code = copy_from_user(to, from, (size) * (count));
-if (code) {
...
+ code = PTR_ERR(to);
log(..., code, ...);
- code = -EFAULT;
goto label;
}
elfring@Sonne:~/Projekte/Coccinelle/janitor> spatch.opt
replace_kcalloc_by_memdup_user-draft2.cocci ../Probe/user_sdma-excerpt1.c
init_defs_builtins: /usr/local/lib64/coccinelle/standard.h
HANDLING: ../Probe/user_sdma-excerpt1.c
diff =
--- ../Probe/user_sdma-excerpt1.c
+++ /tmp/cocci-output-8749-f5da06-user_sdma-excerpt1.c
@@ -11,20 +11,12 @@ int hfi1_user_sdma_process_request(struc
ret = -EINVAL;
goto free_req;
}
- req->tids = kcalloc(ntids, sizeof(*req->tids), GFP_KERNEL);
- if (!req->tids) {
- ret = -ENOMEM;
- goto free_req;
- }
- /*
-* ...
-*/
- ret = copy_from_user(req->tids, iovec[idx].iov_base,
-ntids * sizeof(*req->tids));
- if (ret) {
+ req->tids = memdup_user(iovec[idx].iov_base,
+ sizeof(*req->tids) * ntids);
+ if (IS_ERR(req->tids)) {
+ ret = PTR_ERR(req->tids);
SDMA_DBG(req, "Failed to copy %d TIDs (%d)",
ntids, ret);
- ret = -EFAULT;
goto free_req;
}
req->n_tids = ntids;
Regards,
Markus
___
Cocci mailing list
Cocci@systeme.lip6.fr
https://systeme.lip6.fr/mailman/listinfo/cocci
[Cocci] Replacing combination of kcalloc() and copy_from_user() by memdup_user() with SmPL?
Hello,
I came along another function implementation where I got the impression that
the function “memdup_user” could also be used there in a way which is similar
to an existing source code transformation approach by a small script for
the semantic patch language.
SmPL script:
https://git.kernel.org/cgit/linux/kernel/git/next/linux-next.git/tree/scripts/coccinelle/api/memdup_user.cocci?id=632571b1bee00494aef749512d9f3290dfba0ead#n1
@refactoring@
expression code, count, from, to, size;
identifier label, log;
@@
to =
- kcalloc(count, size, GFP_KERNEL)
+ memdup_user(from, size * count)
;
if (
-!to
+IS_ERR(to)
|| ...) {
- code = -ENOMEM;
- goto label;
-}
-code = copy_from_user(to, from, (size) * (count));
-if (code) {
...
log(...,
- code
+ PTR_ERR(to)
, ...);
code = -ENOMEM;
goto label;
}
Source file:
https://git.kernel.org/cgit/linux/kernel/git/next/linux-next.git/tree/drivers/infiniband/hw/hfi1/user_sdma.c?id=632571b1bee00494aef749512d9f3290dfba0ead#n718
int hfi1_user_sdma_process_request(struct file *fp, struct iovec *iovec,
unsigned long dim, unsigned long *count)
{
/*
* ...
*/
if (req_opcode(req->info.ctrl) == EXPECTED) {
u16 ntids = iovec[idx].iov_len / sizeof(*req->tids);
if (!ntids || ntids > MAX_TID_PAIR_ENTRIES) {
ret = -EINVAL;
goto free_req;
}
req->tids = kcalloc(ntids, sizeof(*req->tids), GFP_KERNEL);
if (!req->tids) {
ret = -ENOMEM;
goto free_req;
}
/*
* ...
*/
ret = copy_from_user(req->tids, iovec[idx].iov_base,
ntids * sizeof(*req->tids));
if (ret) {
SDMA_DBG(req, "Failed to copy %d TIDs (%d)",
ntids, ret);
ret = -EFAULT;
goto free_req;
}
req->n_tids = ntids;
idx++;
}
/*
* ...
*/
return 0;
free_req:
user_sdma_free_request(req, true);
if (req_queued)
pq_update(pq);
set_comp_state(pq, cq, info.comp_idx, ERROR, req->status);
return ret;
}
elfring@Sonne:~/Projekte/Coccinelle/janitor> spatch.opt
replace_kcalloc_by_memdup_user-draft1.cocci ../Probe/user_sdma-excerpt1.c
…
refactoring: position variables or mixed modifs interfere with comm_assoc iso
bool (bool !to
>>> IS_ERR(to)
|| ...)
…
The software combination “Coccinelle 1.0.6-00093-ge1776d7b (OCaml 4.03.0)”
does not like this change attempt so far.
Which details should I reconsider next?
Regards,
Markus
___
Cocci mailing list
Cocci@systeme.lip6.fr
https://systeme.lip6.fr/mailman/listinfo/cocci
[Cocci] Moving setting of error codes into if branches with SmPL?
Hello, I have tried the following small script out for the semantic patch language. @show_assignments@ expression target; identifier code =~ "^E[A-Z]+$"; statement is, es; @@ *target = -(code); if (...) is else es elfring@Sonne:~/Projekte/Linux/next-patched> spatch.opt -timeout 23 -j 4 --chunksize 1 -sp-file ~/Projekte/Coccinelle/janitor/show_misplaced_error_code_assignments1.cocci -dir security/selinux 2> ~/Projekte/Bau/Linux/scripts/Coccinelle/show_misplaced_error_code_assignments1-errors.txt Some update candidates were displayed. Is such a simple source code search pattern worth to be integrated into the Coccinelle script collection? I see further development challenges for the specification of a corresponding source code transformation. How hard can it become to move the found variable assignments before condition checks to the place before the last goto statement within an if branch automatically? Regards, Markus ___ Cocci mailing list Cocci@systeme.lip6.fr https://systeme.lip6.fr/mailman/listinfo/cocci
Re: [Cocci] … adding missing NULL checks
> Is there a clever way of conditionally making this change? The answer depends on your desire how “clever” (or sophisticated) an other approach should become. You could try to work with more disjunctions and rules in the semantic patch language. This transformation might trigger a few target conflicts and further software development challenges as you might become also aware of special needs and corresponding advancements in technology. How many software developers and source code reviewers care really about unused (or ignored) return values from various function calls? Regards, Markus ___ Cocci mailing list Cocci@systeme.lip6.fr https://systeme.lip6.fr/mailman/listinfo/cocci
Re: [Cocci] … adding missing NULL checks
>> need to put another copy of that at the end of the rule with ? in front >> of it. So that it stops when it comes to that again. > > And now it's working just fine. I became also curious on how the discussed source code transformation can really work. I imagine that there are still some software development challenges left over for further considerations. * A filter criterium like “T = strdup(...);” was specified. How do you think about to use a SmPL disjunction (or an advanced SmPL constraint on a metavariable with the type “identifier” instead)? * Can the function call “pkg_emit_errno” become a bit more generic by passing a string from previously matched data? * Would you like to check any more cases so that the technology “data flow analysis” will be also needed here? Regards, Markus ___ Cocci mailing list Cocci@systeme.lip6.fr https://systeme.lip6.fr/mailman/listinfo/cocci
Re: [Cocci] Improved filtering for a big function collection with SmPL?
> Why were you looking for such functions in the first place? These functions provide various return values. I am curious on how often the returned values will actually be checked and used (or not). > If you want non-static non-void functions and you find them, > then what is the issue? NetBSD contains a lot of components from various software. I tried to get an impression from the size of the corresponding function collection. The mentioned concrete size will show further software development challenges if meaningful source code transformations should be performed with such data, won't it? Regards, Markus ___ Cocci mailing list Cocci@systeme.lip6.fr https://systeme.lip6.fr/mailman/listinfo/cocci
[Cocci] Improved filtering for a big function collection with SmPL?
Hello, I tried the Coccinelle software out for another source code analysis approach on source files from NetBSD 7.0.2. A small SmPL script worked mostly so good that 136978 non-static function implementations were finally found with a non-void return type there. * Can such an analysis result trigger further software development considerations? * I imagine that this function collection can only be used for some benefit if it will be grouped and filtered to specific needs. * Would you like to suppress any functions from this data extraction? Regards, Markus ___ Cocci mailing list Cocci@systeme.lip6.fr https://systeme.lip6.fr/mailman/listinfo/cocci
Re: [Cocci] Checking incidence for SmPL initialisation rules after parallel script execution
>> Can you generally clarify current consequences from usage of a SmPL rule >> “initialize:…” >> together with the parameter “--jobs” for the program “spatch”? > > It should happen only once. Thanks for this information. I would like to know if the script initialisation should happen for each background process and/or after the start of the program “spatch”. > Thus without your semantic patch and your .c file, I can't tell > why it doesn't behave as expected. You got the corresponding data from my source code analysis try for NetBSD. How often do you observe a simple text line output from initialisation code after parallel execution of one from your test scripts? Regards, Markus ___ Cocci mailing list Cocci@systeme.lip6.fr https://systeme.lip6.fr/mailman/listinfo/cocci
Re: [Cocci] Checking incidence for SmPL initialisation rules after parallel script execution
> Without having the semantic patch and the .c file, ideally with both > cut down to a minimal version that shows the problem, > I can't answer this question. Can you generally clarify current consequences from usage of a SmPL rule “initialize:…” together with the parameter “--jobs” for the program “spatch”? Regards, Markus ___ Cocci mailing list Cocci@systeme.lip6.fr https://systeme.lip6.fr/mailman/listinfo/cocci
Re: [Cocci] Checking incidence for SmPL initialisation rules after parallel script execution
>> Is it really appropriate that this header appears so often in the shown >> output file? > > Without having the semantic patch and the .c file, ideally with both > cut down to a minimal version that shows the problem, > I can't answer this question. Does the follow display look a bit better? elfring@Sonne:~/Projekte/Coccinelle/janitor> grep 'function|"data type"|"source file"|line|column' ~/Projekte/Bau/NetBSD/Coccinelle/functions1/API2.txt | wc -l 5 (Four jobs plus one) I am looking for another update on the consequences from the parallel execution of scripts for the semantic patch language. How often should the specified initialisation be performed for the embedded programming language scripts? Regards, Markus ___ Cocci mailing list Cocci@systeme.lip6.fr https://systeme.lip6.fr/mailman/listinfo/cocci
[Cocci] Checking incidence for SmPL initialisation rules after parallel script execution
Hello, I noticed some software development progress for an issue like “SmPL constraints are broken for type metavariables.”. https://github.com/coccinelle/coccinelle/issues/32 Now I am checking further test results from a command like the following for the software combination “Coccinelle 1.0.6-00033-g23cca0a2 (OCaml 4.03.0)”. spatch.opt -timeout 23 -j 4 --chunksize 1 -sp-file ~/Projekte/Coccinelle/janitor/list_non-void_and_non-static_functions2.cocci -dir ~/geladen/Freeware/Quellen/NetBSD/7.0.2 > ~/Projekte/Bau/NetBSD/Coccinelle/functions1/API2.txt 2> ~/Projekte/Bau/NetBSD/Coccinelle/functions1/API2-errors.txt This script contains also a SmPL rule “initialize:python” which should output a header line (usually once?). elfring@Sonne:~/Projekte/Coccinelle/janitor> ack 'function|"data type"|"source file"|line|column' ~/Projekte/Bau/NetBSD/Coccinelle/functions1/API2.txt | wc -l 2022 Is it really appropriate that this header appears so often in the shown output file? Regards, Markus ___ Cocci mailing list Cocci@systeme.lip6.fr https://systeme.lip6.fr/mailman/listinfo/cocci
Re: [Cocci] non-static data structure selection by SmPL
> disable optional_storage Thanks for your hint. * How do you think about to expose more information around this special setting for the handling of “annotations” by the semantic patch language together with “Coccilib”? * Would it make sense to specify any SmPL constraints for storage-class specifiers? Regards, Markus ___ Cocci mailing list Cocci@systeme.lip6.fr https://systeme.lip6.fr/mailman/listinfo/cocci
Re: [Cocci] non-static data structure selection by SmPL
> I have no idea what you are talking about. I got the impression that recent extensions for the Coccinelle software could increase the chances to update the support for special constraints a bit more. > What is wrong with the rule that is shown? This SmPL specification is still working for a while. I became curious once more if there are further possibilities to select in a single SmPL rule if a storage-class specifier like “static” was not (!) specified. How do you think about to reduce the dependency on position variables (or internal database tables) for such source code checking? Regards, Markus ___ Cocci mailing list Cocci@systeme.lip6.fr https://systeme.lip6.fr/mailman/listinfo/cocci
[Cocci] Determination of static data types by SmPL constraints?
Hello,
I am using the following bit of code for semantic patch language scripts.
…
@find_static depends on safety_check@
identifier safety_check.work;
position safety_check.pos;
@@
static void work@pos(...)
{
...
}
…
The Coccinelle software was extended for the handling of special constraints
by the execution of embedded scripts in supported programming languages
to some degree.
I got the idea that this functionality could also be used now to check
the usage of a storage-class specifier like “static” in data types for further
source code analysis approaches.
* How should such a specifier be determined by the programming interfaces
of “Coccilib”?
* Would you like to compare run time characteristics for the execution
of the shown SmPL rule and a similar constraint implemented as
an OCaml (or Python) script?
Regards,
Markus
___
Cocci mailing list
Cocci@systeme.lip6.fr
https://systeme.lip6.fr/mailman/listinfo/cocci
Re: [Cocci] Excluding specific assignment combinations with SmPL
> ( > ex1 = action(...); > ex2 = <+...ex1...+>; > | > *ex1 = action(...); > ex2 = value; > ) > > is perhaps what you are looking for. I would prefer an approach where the expressions “ex1” and “value” could be compared somehow directly instead of matching an unwanted branch in a SmPL disjunction. Regards, Markus ___ Cocci mailing list Cocci@systeme.lip6.fr https://systeme.lip6.fr/mailman/listinfo/cocci
Re: [Cocci] Excluding specific assignment combinations with SmPL
> Why don't you think about it on your own and write back if you > don't figure anything out. I am still looking for ways to make a small SmPL specification like the following a bit safer. @find_…@ expression ex1, ex2, value; identifier action =~ …; @@ *ex1 = action(...); ex2 = value; I am unsure if such a source code search approach should eventually be split into more SmPL processing steps. Regards, Markus ___ Cocci mailing list Cocci@systeme.lip6.fr https://systeme.lip6.fr/mailman/listinfo/cocci
Re: [Cocci] Comparing SmPL script constraints with direct regular expression interface
> Why not just define a python function that you can call in a script to do
> your big regexp for you?
Would any more developers and reviewers like to extend a source code analysis
approach with a regular expression (or other data structure) like the following?
SmPL script example:
@initialize:python@
@@
import re
compiled = re.compile("""^
(?:
k(?:
[cmz]alloc
| (?:m(?:
alloc_array
| em(?:
_cache_alloc(?:_node)?
| dup
)
)
| zalloc_node
)
| str(?:
dup(?:_const)?
| ndup
)
)
| of_find_matching_node
# Alternation placeholder
)$""", re.VERBOSE)
def is_selected(id):
match = compiled.search(id)
if match:
return True
else:
return False
@find_too_late_checking@
expression assign1, assign2, ex1, ex2;
identifier action1: script:python() { is_selected(action1) },
action2: script:python() { is_selected(action2) },
work;
statement is, es;
type return_type;
@@
return_type work(...)
{
... when any
*ex1 = action1(...);
ex2 = action2(...);
... when any
when != (ex1 = assign1)
when != (ex2 = assign2)
if (
* \( !(ex1) \| (ex1) == NULL \)
|| \( !(ex2) \| (ex2) == NULL \)
|| ...)
is
else
es
... when any
}
elfring@Sonne:~/Projekte/Coccinelle/janitor> spatch.opt
show_too_late_checking-python-20161202.cocci ../Probe/clk-mstp-excerpt1.c
…
diff =
--- ../Probe/clk-mstp-excerpt1.c
+++ /tmp/cocci-output-20769-f189ac-clk-mstp-excerpt1.c
@@ -5,9 +5,7 @@ static void __init cpg_mstp_clocks_init(
struct clk **clks;
unsigned int i;
- group = kzalloc(sizeof(*group), GFP_KERNEL);
clks = kmalloc(MSTP_MAX_CLOCKS * sizeof(*clks), GFP_KERNEL);
- if (group == NULL || clks == NULL) {
kfree(group);
kfree(clks);
pr_err("%s: failed to allocate group\n", __func__);
Regards,
Markus
___
Cocci mailing list
Cocci@systeme.lip6.fr
https://systeme.lip6.fr/mailman/listinfo/cocci
Re: [Cocci] Comparing SmPL script constraints with direct regular expression interface
>> Do you find the documentation complete for this functionality at the moment? > > No idea. How can we achieve a better common understanding on this aspect for example? >> I imagine a need for advanced configuration possibilities around “regexp” >> engines. > > You imagine many things. Yes, of course. ;-) > Changes will only happen when there is a concrete need. I came along some development needs also together with software experiments around SmPL scripts. How do your needs look different over time? >> Is the current approach the official one already? > > Yes. Thanks for this information. >>> Perhaps the constraint scrpt syntax could be generalized to allow more >>> in practice than function calls. >> >> Which software development concerns can hinder progress in the way >> I imagine so far? > > Lack of time. > >> >>> That might happen some day, but it is an extremely low priority. >> >> I am curious if the involved dependencies can be clarified further. > > I doubt there are any dependencies. Our software development capacity is limited as usual. Will future software research activities improve the situation a bit more? >> https://github.com/coccinelle/coccinelle/blob/cbc751b30d9e02390d60ebed643c8e4a3fa0bb2b/tests/idcon_ocaml.cocci > > In the test case the braces are around a function call, not a function name. This detail is also clear for me to some degree. > No idea what could be relevant or irrelevant about it. I got further ideas for this software area after a delay. > That is the syntax that has been implemented. I wonder still why such a variant was chosen. > It may change when someone when someone motivated enough to work on the > implementation writes the code to make the change and tests the result. My motivation could eventually increase if the understanding of the corresponding OCaml source code could become better somehow besides other factors. > It won't change just because someone finds it conceptually inelegant > and sends lots of emails about it. So there is some usual change resistance. I dare to provide intensive feedback in the hope that details can be uncovered which can support more progress. > It's actually strange that you want to define regexps in other places, This detail is just another aspect for the involved software evolution. > but you don't want to define scripts in other places. I got also further ideas around the placement of script fragments. > Why not just define a python function that you can call in a script > to do your big regexp for you? This approach can only work safely if script constraints will be completely documented in the manual (besides one test case in OCaml and Python). Regards, Markus ___ Cocci mailing list Cocci@systeme.lip6.fr https://systeme.lip6.fr/mailman/listinfo/cocci
Re: [Cocci] Comparing SmPL script constraints with direct regular expression interface
> I don't think there is anything to be more explicit about. Do you find the documentation complete for this functionality at the moment? > I don't foresee any change in the regular expression syntax, I imagine a need for advanced configuration possibilities around “regexp” engines. > and I don't forsee any change in the constraint script syntax. Is the current approach the official one already? > Perhaps the constraint scrpt syntax could be generalized to allow more > in practice than function calls. Which software development concerns can hinder progress in the way I imagine so far? > That might happen some day, but it is an extremely low priority. I am curious if the involved dependencies can be clarified further. >> Have we got just different views about the relevance of curly brackets around >> the specification of a function name (instead of a complete function body)? > > No idea what this means. What curly braces? What function name? I find this kind of feedback strange. I hope that communication difficulties can be resolved better around the provided test case, can't it? https://github.com/coccinelle/coccinelle/blob/cbc751b30d9e02390d60ebed643c8e4a3fa0bb2b/tests/idcon_ocaml.cocci Regards, Markus ___ Cocci mailing list Cocci@systeme.lip6.fr https://systeme.lip6.fr/mailman/listinfo/cocci
Re: [Cocci] Comparing SmPL script constraints with direct regular expression interface
>> >> https://github.com/coccinelle/coccinelle/commit/cbc751b30d9e02390d60ebed643c8e4a3fa0bb2b >> >>But I am unsure about their current development and documentation status. >>Are there any more imaginations floating around for the notation and usage >>of named (or ad hoc) predicates? > > I don't anticipate any changes in user-facing functionality I would appreciate a bit more explicit information for open isses. > in either of these cases in the short term. Have we got just different views about the relevance of curly brackets around the specification of a function name (instead of a complete function body)? Would other delimiters help to make the corresponding data passing a bit shorter? Regards, Markus ___ Cocci mailing list Cocci@systeme.lip6.fr https://systeme.lip6.fr/mailman/listinfo/cocci
[Cocci] Comparing SmPL script constraints with direct regular expression interface
Hello,
I tried another source code search pattern out with the semantic patch language
from the software “Coccinelle 1.0.6-00022-g862f5b29 (OCaml 4.03)”.
SmPL script example:
@find_too_late_checking@
expression assign1, assign2, ex1, ex2;
identifier action1 =~ "^(?x)
(?:
k(?:[cmz]alloc|(?:m(?:alloc_array|em_cache_alloc(?:_node)?)|zalloc_node))
| of_find_matching_node
# Alternation placeholder
)$",
action2 =~ "^(?x)
(?:
k(?:[cmz]alloc|(?:m(?:alloc_array|em_cache_alloc(?:_node)?)|zalloc_node))
| of_find_matching_node
# Alternation placeholder
)$",
work;
statement is, es;
type return_type;
@@
return_type work(...)
{
... when any
*ex1 = action1(...);
ex2 = action2(...);
... when any
when != (ex1 = assign1)
when != (ex2 = assign2)
if (
* \( !(ex1) \| (ex1) == NULL \)
|| \( !(ex2) \| (ex2) == NULL \)
|| ...)
is
else
es
... when any
}
elfring@Sonne:~/Projekte/Coccinelle/janitor> spatch.opt
show_too_late_checking1.cocci ../Probe/clk-mstp-excerpt1.c
…
diff =
--- ../Probe/clk-mstp-excerpt1.c
+++ /tmp/cocci-output-23612-4723e1-clk-mstp-excerpt1.c
@@ -5,9 +5,7 @@ static void __init cpg_mstp_clocks_init(
struct clk **clks;
unsigned int i;
- group = kzalloc(sizeof(*group), GFP_KERNEL);
clks = kmalloc(MSTP_MAX_CLOCKS * sizeof(*clks), GFP_KERNEL);
- if (group == NULL || clks == NULL) {
kfree(group);
kfree(clks);
pr_err("%s: failed to allocate group\n", __func__);
This source code analysis approach is incomplete so far. It demonstrates a few
aspects where further software extensions might help.
1. The semantic patch language supports data processing with regular expressions
to some degree as direct constraints for metavariables.
A “regexp” can be specified for several metavariables. Such regular
expressions
can become very detailed and big so that I got the desire to avoid
code duplication there as much as possible.
So it would be nice if SmPL variables can share a specific compiled “regexp”
from a single specification place.
2. The support for script constraints was recently mentioned.
https://github.com/coccinelle/coccinelle/commit/cbc751b30d9e02390d60ebed643c8e4a3fa0bb2b
But I am unsure about their current development and documentation status.
Are there any more imaginations floating around for the notation and usage
of named (or ad hoc) predicates?
Regards,
Markus
___
Cocci mailing list
Cocci@systeme.lip6.fr
https://systeme.lip6.fr/mailman/listinfo/cocci
Re: [Cocci] Finding missing return value checks for some function calls with SmPL
>> But can it be that this assignment is checked a bit too late? > > Yes, Thanks that you can agree to such a view. > but no one will care. I am curious if further software development opinions will appear around similar situations. > The code is more concise as is. I have got an other opinion for such an implementation detail. > If the alloc fails, the machine is already in a very bad state, so one more > alloc > attempt doesn't matter. Would it be nicer to avoid a questionable function call (and a corresponding variable assignment) when it was determined that the desired task could not be completed for a moment? Regards, Markus ___ Cocci mailing list Cocci@systeme.lip6.fr https://systeme.lip6.fr/mailman/listinfo/cocci
Re: [Cocci] Finding missing return value checks for some function calls with SmPL
> Now I wonder why the first assignment is not marked together with the
> function call
> for further considerations by my source code analysis approach so far.
I should have adjusted one of the used regular expressions a bit more like the
following.
@find_too_late_checking@
expression ex1, ex2;
identifier action1 =~ "^(?x)
(?:
kzalloc
| kmalloc_array
)$",
action2 =~ "^(?x)
(?:
kzalloc
| kmalloc(?:_array)?
)$",
work;
type return_type;
@@
return_type work(...)
{
... when any
*ex1 = action1(...);
ex2 = action2(...);
... when any
}
elfring@Sonne:~/Projekte/Coccinelle/janitor> spatch.opt
show_too_late_checking2.cocci ../Probe/clk-mstp-excerpt1.c
init_defs_builtins: /usr/local/lib64/coccinelle/standard.h
HANDLING: ../Probe/clk-mstp-excerpt1.c
diff =
--- ../Probe/clk-mstp-excerpt1.c
+++ /tmp/cocci-output-20109-52f048-clk-mstp-excerpt1.c
@@ -5,7 +5,6 @@ static void __init cpg_mstp_clocks_init(
struct clk **clks;
unsigned int i;
- group = kzalloc(sizeof(*group), GFP_KERNEL);
clks = kmalloc(MSTP_MAX_CLOCKS * sizeof(*clks), GFP_KERNEL);
if (group == NULL || clks == NULL) {
kfree(group);
I imagine that there are further software improvements to consider around
such use cases.
Regards,
Markus
___
Cocci mailing list
Cocci@systeme.lip6.fr
https://systeme.lip6.fr/mailman/listinfo/cocci
Re: [Cocci] Inclusion of script constraints in SmPL
>> * Would such an inclusion approach trigger special requirements on the >> position for corresponding initialisation code? > > The constraint specification should be a single function call. Does the script constraint syntax allow more programming language code there? > You can set up your @initialize:python@ code to import whatever you want. I would prefer to keep initialisation code for the implementation of the predicate function closely together with the call of the included metavariable. Regards, Markus ___ Cocci mailing list Cocci@systeme.lip6.fr https://systeme.lip6.fr/mailman/listinfo/cocci
Re: [Cocci] Inclusion of script constraints in SmPL
> The idea with the script constraints on metavariables feature is that the > script code there should only contain a function call. The actual > function definition should be elsewhere; probably in an initialize rule. * Did you consider the use case already when such a constraint specification would be included from an external file as a component or together with a rule selection for the semantic patch language? * Would such an inclusion approach trigger special requirements on the position for corresponding initialisation code? Regards, Markus ___ Cocci mailing list Cocci@systeme.lip6.fr https://systeme.lip6.fr/mailman/listinfo/cocci
Re: [Cocci] Further developments around function callbacks for SmPL metavariables?
> The idea with the script constraints on metavariables feature is that the > script code there should only contain a function call. I would like to add another view to this software evolution for further considerations. The purpose of such a function call can be interpreted in different ways, can't it? 1. I get the impression from the word “constraint” that something should be restricted. Does this aspect mean that the functionality is finally mapped to a predicate? 2. Can it be also used as a data generator or content provider for any other non-boolean value? Regards, Markus ___ Cocci mailing list Cocci@systeme.lip6.fr https://systeme.lip6.fr/mailman/listinfo/cocci
Re: [Cocci] SmPL script construction with included files
> > The idea with the script constraints on metavariables feature is that the > script code there should only contain a function call. The actual > function definition should be elsewhere; probably in an initialize rule. This information can be reasonable to some degree. How will the software situation evolve if each script constraint will need another bit of initialisation code? Can it become an interesting software development challenge then if the constraint specification approach should be switchable for a few of them by a corresponding make script? Regards, Markus ___ Cocci mailing list Cocci@systeme.lip6.fr https://systeme.lip6.fr/mailman/listinfo/cocci
Re: [Cocci] SmPL script construction with included files
> In a script rule, the metavariable declarations are interpreted by Coccinelle, > not the scripting language, so an include would not be allowed there. This detail was clear for me in principle. > Script rules also don't have constraints on metavariables anyway, Can other SmPL rule types specify constraints in the code form of a regular expression or a programming language script? > so I'm not sure what you are asking about. I am looking for ways to avoid source code duplication by inclusion of other files. Would it make sense to allow the use of an include statement within the SmPL variable definition block of a rule? Regards, Markus ___ Cocci mailing list Cocci@systeme.lip6.fr https://systeme.lip6.fr/mailman/listinfo/cocci
Re: [Cocci] SmPL script construction with several initialisation parts
Can such initialisation code be constructed by the specification of several script parts there? >>> I don't understand the question. >> I suggest to consider corresponding design consequences again. >> >> >>> If the script language supports includes, then you can put includes in the >>> script code. >> I imagine that it will be occasionally nicer to avoid the indirect inclusion >> of another file there. >> When will it be more consistent to construct an initialisation script >> by your host programming language? > No idea what you mean. Is it easier for you to clarify the use case based on a concrete source code example from the recent development “Script constraints on meta-variables”? https://github.com/coccinelle/coccinelle/blob/cbc751b30d9e02390d60ebed643c8e4a3fa0bb2b/tests/idcon_python.cocci#L1 If I would like to switch between two constraint specifications there for a function name list selection in a bigger SmPL script template, I am looking for a convenient way to append another bit of initialisation code. Regards, Markus ___ Cocci mailing list Cocci@systeme.lip6.fr https://systeme.lip6.fr/mailman/listinfo/cocci
Re: [Cocci] SmPL script construction with several initialisation parts
>> Can such initialisation code be constructed by the specification of several >> script parts there? > > I don't understand the question. I suggest to consider corresponding design consequences again. > If the script language supports includes, then you can put includes in the > script code. I imagine that it will be occasionally nicer to avoid the indirect inclusion of another file there. When will it be more consistent to construct an initialisation script by your host programming language? Regards, Markus ___ Cocci mailing list Cocci@systeme.lip6.fr https://systeme.lip6.fr/mailman/listinfo/cocci
[Cocci] SmPL script construction with several initialisation parts
Hello, The semantic patch language supports also extra data processing by initialisation scripts. http://coccinelle.lip6.fr/docs/main_grammar003.html Can such initialisation code be constructed by the specification of several script parts there? Regards, Markus ___ Cocci mailing list Cocci@systeme.lip6.fr https://systeme.lip6.fr/mailman/listinfo/cocci
Re: [Cocci] Coccinelle: misc: Improve the matching of rules
>> I assumed that the reduced code repetition could result in a desired effect. > > Incorrect on all points. I wonder then why the Coccinelle software would not benefit from the specification of common parts in two similar SmPL disjunctions. Regards, Markus ___ Cocci mailing list Cocci@systeme.lip6.fr https://systeme.lip6.fr/mailman/listinfo/cocci
Re: [Cocci] [PATCH v2 1/3] Coccinelle: misc: Improve the matching of rules
> Sorry, I thought it was the original code, and didn't realize that it was > the suggestion.. > > But I don't think that the suggestion improves readability in any way. I hoped that my SmPL script variant could influence the execution characteristics in positive ways. > Coccinelle disjunctions also float up to the statement level anyway, I do not really understand this information at the moment. > so the semantic patch that is actually applied to the code is the same in > either case. I assumed that the reduced code repetition could result in a desired effect. +( + ret = +( + request_threaded_irq@p(irq, NULL, thread_fn, flags, ...) +| + devm_request_threaded_irq@p(dev, irq, NULL, thread_fn, flags, ...) +) + ; +| + return +( +request_threaded_irq@p(irq, NULL, thread_fn, flags, ...) +| +devm_request_threaded_irq@p(dev, irq, NULL, thread_fn, flags, ...) +) + ; +) Regards, Markus ___ Cocci mailing list Cocci@systeme.lip6.fr https://systeme.lip6.fr/mailman/listinfo/cocci
Re: [Cocci] [PATCH v2 1/3] Coccinelle: misc: Improve the matching of rules
> What do you mean by "a bit more fine-tuning"? If you have a concrete > suggestion, then make it. But don't make work for other people to guess > what you might mean. I hoped that the SmPL source code below would explain my update suggestion in a sufficient way to use two smaller SmPL disjunctions. >> +( >> + ret = >> +( >> + request_threaded_irq@p(irq, NULL, thread_fn, flags, ...) >> +| >> + devm_request_threaded_irq@p(dev, irq, NULL, thread_fn, flags, ...) >> +) >> + ; >> +| >> + return >> +( >> +request_threaded_irq@p(irq, NULL, thread_fn, flags, ...) >> +| >> +devm_request_threaded_irq@p(dev, irq, NULL, thread_fn, flags, ...) >> +) >> + ; >> +) Regards, Markus ___ Cocci mailing list Cocci@systeme.lip6.fr https://systeme.lip6.fr/mailman/listinfo/cocci
Re: [Cocci] Replacing calls of general functions by more specific ones with SmPL
>> How much would you like to increase the support for data processing around >> macros >> with your software? > > Ther are no particular plans in this direction. How does this kind of feedback fit to an information like “Add identifier list metavariables for #define parameter lists” from the recent software release? Regards, Markus ___ Cocci mailing list Cocci@systeme.lip6.fr https://systeme.lip6.fr/mailman/listinfo/cocci
Re: [Cocci] Replacing calls of general functions by more specific ones with SmPL
>> I have got a few concerns for further software development considerations. >> How do you think about to support also an advanced interface by which it >> could be >> determined if a preprocessor symbol would correspond to such a desired >> character >> (instead of fiddling with regular expressions within a rule for the embedded >> programming >> language script)? > > What C code are you concerned about exactly? I guess that this "story" affects some general aspects. After I got a better understanding of related software dependencies, I begin to think also about further possibilities to make such a source code search pattern safer and more efficient. How much would you like to increase the support for data processing around macros with your software? Regards, Markus ___ Cocci mailing list Cocci@systeme.lip6.fr https://systeme.lip6.fr/mailman/listinfo/cocci
Re: [Cocci] Replacing calls of general functions by more specific ones with SmPL
>> Will it occasionally matter then to distinguish better if a specific source >> code >> fragment or a complete source file should be excluded from an analysis run >> on demand? > > I have no idea what you mean by this. I assume that there are more chances. I am just unsure around the application of a function call like "cocci.include_match(False)" so far. > But since I know very little about this printing problem in general, It is not a "problem". - Is it just the beginning for another software development challenge (or adventure)? > I doubt that I can give an answer. I imagine that we can exchange some ideas to improve the affected software situation further, can't we? > Try it and see what requirements turn up. Which items will be picked up for further clarification? Regards, Markus ___ Cocci mailing list Cocci@systeme.lip6.fr https://systeme.lip6.fr/mailman/listinfo/cocci
Re: [Cocci] Replacing calls of general functions by more specific ones with SmPL
>> I do not see the way so far how desired data could be extracted from a >> function >> (or expression?) parameter. Would you like to make the introspection options >> a bit >> clearer for the SmPL interface? > > You can inherit and test properties of anything you like in python or ocaml > code. I have got an SmPL approach working. > If you inherit some kind of metavariable other than a position variable, > the term will be represented as a string. This software design aspect might look sufficient for a basic variant of the discussed source code search pattern. > But that should be fine for detecting eg if a string has only one character. I have got a few concerns for further software development considerations. How do you think about to support also an advanced interface by which it could be determined if a preprocessor symbol would correspond to such a desired character (instead of fiddling with regular expressions within a rule for the embedded programming language script)? Regards, Markus ___ Cocci mailing list Cocci@systeme.lip6.fr https://systeme.lip6.fr/mailman/listinfo/cocci
Re: [Cocci] Replacing calls of general functions by more specific ones with SmPL
> You can look at patches people have previously submitted and see to what > extent there is interest in this issue. I see another software development challenge here. The discussed source code search pattern builds on a "general function" like "seq_printf" as an update candidate from which each instance (probably with a different prefix) is connected with a few replacement possibilities. Did any other software developer fiddle with corresponding combinations of function names? Regards, Markus ___ Cocci mailing list Cocci@systeme.lip6.fr https://systeme.lip6.fr/mailman/listinfo/cocci
Re: [Cocci] Replacing calls of general functions by more specific ones with SmPL
> You can inherit and test properties of anything you like in python or
> ocaml code. If you inherit some kind of metavariable other than a
> position variable, the term will be represented as a string. But that
> should be fine for detecting eg if a string has only one character.
Your explanation sounds promising.
> If it has multiple characters, you can do the include match false call, so
> that
> in the last rule when yu inherit again the metavariable containing th
> string, it won't be seen.
I see another software development challenge then.
* I imagine that this "false call" will work for the exclusion of unwanted
source
code fragments.
* But I will also need a way to construct a source code change ("plus code" in
the semantic patch) with OCaml or Python scripting, won't I?
Regards,
Markus
___
Cocci mailing list
Cocci@systeme.lip6.fr
https://systeme.lip6.fr/mailman/listinfo/cocci
Re: [Cocci] Replacing calls of general functions by more specific ones with SmPL
>> With which metavariables and corresponding programming interfaces can >> strings that >> are passed to such functions be checked in more detail by the Coccinelle >> software? > > The simplest would be to use python or ocaml code to make the checks that > are needed. You can use cocci.include_match(False) in python or > Coccilib.include_match false in ocaml when your test detects that > something is not suitable. Thanks for your suggestion. > coccinelle/demos/first.cocci may be helpful. I have got the impression that this small SmPL script demonstrates the usage of the mentioned function from your software library only for a simple position variable. I do not see the way so far how desired data could be extracted from a function (or expression?) parameter. Would you like to make the introspection options a bit clearer for the SmPL interface? Regards, Markus ___ Cocci mailing list Cocci@systeme.lip6.fr https://systeme.lip6.fr/mailman/listinfo/cocci
[Cocci] Replacing calls of general functions by more specific ones with SmPL
Hello, I came during my software development activities along a few Linux source files where I got an advice by the script "checkpatch.pl" like the following. WARNING: Prefer seq_puts to seq_printf https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/tree/scripts/checkpatch.pl?id=29fbff8698fc0ac1a1d74584b258e0bf18b469f9#n5610 Joe Perches pointed out that the function "seq_putc" would be more appropriate at a specific source code place. https://patchwork.kernel.org/patch/9364455/ https://lkml.kernel.org/r/<1475771699.1914.10.ca...@perches.com> Now I am curious on how good the semantic patch language is applicable for advanced source code transformations in similar use cases. With which metavariables and corresponding programming interfaces can strings that are passed to such functions be checked in more detail by the Coccinelle software? Regards, Markus ___ Cocci mailing list Cocci@systeme.lip6.fr https://systeme.lip6.fr/mailman/listinfo/cocci
Re: [Cocci] Usage of "expressions" and "identifiers" with SmPL
>>> The only effect your emails had on me was a big "WTF!?". >> >> All of them (including the provided small SmPL script examples on >> 2016-10-05)? > > You wrote three emails on that day. I assume you mean this one? There are further possibilities to improve your learning experience around the replacement of data types and variable names, aren't there? > , > | > 2. ..and how would I go about if instead of the type, I want to replace > | >a variable name? (my_type *ptr --> my_type *pointer). > | > | Would you like to try another small SmPL script out like the following? > | > | > | @name_replacement@ > | @@ > | my_type * > | -ptr > | +pointer > | ; > | > | > | How will your software development experiments evolve for the desired > | application of the semantic patch language? > ` > > I'm afraid the answer is yes. Interesting … > Even though the snippet was relevant, Nice. - Thanks for this kind of feedback. > its presentation triggered the same WTF effect. Are you really at the beginning of a learning process where almost every new information can make you upset anyhow? * How would this fit to your academic education? * Did you start reading the Coccinelle manual? >>> What's the point of all these random questions that you bring up in >>> response to every sentence? >> >> Did you inspect any items from the GitHub issue tracker? > [...] > > Well, I tried to make a point here., but it quite obviously failed utterly. I find that a constructive discussion consists of several questions and some corresponding answers, doesn't it? I am wondering about the learning style that you seem to present here in comparison to some information which was published in your blog. Are there any further challenges to clarify around a term like "mental capacity"? Can any additional techniques or tools help to avoid unwanted communication difficulties so that another software development "fusion" would be achievable? Regards, Markus ___ Cocci mailing list Cocci@systeme.lip6.fr https://systeme.lip6.fr/mailman/listinfo/cocci
Re: [Cocci] Usage of "expressions" and "identifiers" with SmPL
>>> Someone who has to read so much text and in the end has no information >>> about the question he was asking will not likely get a good impression >>> about the software he is trying to use. >> >> A beginner should usually read some text for the desired learning experience. >> Can it be that you worry a bit too much about the potential for >> bad impressions around your software? > > Sorry to be the bringer of bad news, but Julia is spot on. Thanks for your interesting feedback. It seems to indicate also special details about your learning approach. > The only effect your emails had on me was a big "WTF!?". All of them (including the provided small SmPL script examples on 2016-10-05)? Do you care for the consequences around the usage of embedded (programming) languages within the semantic patch language as "a host"? Can such a distinction be useful also for you? > Had Julia not responded to my messages at almost the same time, Here software support is very nice, isn't it? > I would have left this list and Coccinelle for good. I wonder that you would really run away so quickly just because a few alternative description approaches became a bit longer. >> Would you like to start another marketing project? > > What's the point of all these random questions that you bring up in > response to every sentence? Did you inspect any items from the GitHub issue tracker? > Do you think they are helpful? I hope so. >> I find that this mailing list gets only low message traffic so far. > > Do you think others share this impression? Yes. - I guess so. (I do not know concrete numbers for my view.) > Do you think this is a good thing or a bad thing? It might be easier to get started than in other areas you could get involved in depending on your desire. I hope that the varying topic mixture is interesting enough as another valuable information source. > How do you think will this situation evolve? I imagine that the Coccinelle software will be improved in various directions. I hope that the attractiveness of the provided tools will increase accordingly. Will such an evolution mean more users with a higher message exchange rate at various places? >> So I would imagine that most well-intended discussion contributions could >> be useful. Is the mixture of presented topics reasonable? > > What do you mean with discussion contributions? Did I (and the other contributors) publish any messages on this mailing list before that could fit to your way of looking for helpful information already? > Do you think that asking if the topic is reasonable is equivalent to asking > if the specific contributions are reasonable? No. - Would you like to discuss this aspect any further? >> We came along different views around the usage of "expressions" >> in previous discussions, didn't we? > > Whom do you mean with "we"? I guess that it could be primarily interpreted as Julia and me. > Is it important in this context? I find the software development history somewhat relevant also in this use case. >> You repeated the explanation "An expression is something that has a value." >> which is reasonable to some degree. I would like to know then: >> Which "value" is provided by the software construct "idexpression" >> in the semantic patch language? > > Do you think you will be able to find out the answer to this question? I could get some information from reading OCaml source code in principle. But I would prefer an other clarification approach. > How could we best assist you in that? Which possibilities have you got in mind at the moment? Are you really interested in improving your "assistance" in significant ways? Regards, Markus ___ Cocci mailing list Cocci@systeme.lip6.fr https://systeme.lip6.fr/mailman/listinfo/cocci
Re: [Cocci] Replacing one (specific!) type with another
>> The name is illustrated in Advanced SmPL: >> http://coccinelle.lip6.fr/papers/cocciwk4_talk2.pdf > > Call me stupid, but I just went through it again and I still don't see > it. There are lots of examples with "@ ..stuff..@" (the first on slide > 4), but as far as I can tell it never explains what this means. How do you find information from a corresponding syntax description for the semantic patch language? Metavariables for transformations http://coccinelle.lip6.fr/docs/main_grammar002.html > Actually, the frequent occurence of @script:python@ makes me think that > this actually has semantic signficance and is much more than a label > that I can choose for my own convenience...? Did you read another published document already? Metavariables for scripts http://coccinelle.lip6.fr/docs/main_grammar003.html Regards, Markus ___ Cocci mailing list Cocci@systeme.lip6.fr https://systeme.lip6.fr/mailman/listinfo/cocci
Re: [Cocci] Usage of "expressions" and "identifiers" with SmPL
> I'm not angry at all, Thanks for your feedback that there are other emotions involved. > only tired of so much incorrect Which details from my description approach are inappropriate in your view? > or unnecessary information being provided to new users. I got the impression somehow that the clarification request by Nikolaus Rath evolved into a direction where further technical background information would be useful. He can also decide on his own if and how he would eventually like to continue this software development and user support discussion. Regards, Markus ___ Cocci mailing list Cocci@systeme.lip6.fr https://systeme.lip6.fr/mailman/listinfo/cocci
Re: [Cocci] Usage of "expressions" and "identifiers" with SmPL
>>* Syntax constructs from the supported source languages are matched with >> "key words" that are often similar (or even identical) to the host >> language >> (within SmPL scripts). > > This is an enormous amount of verbiage, Interesting view … > and I don't see the relevance to the question. I am trying to explain the matching from syntax elements of the C programming language to the means of the semantic patch language from my view. > Furthermore, I would very much appreciate it if you would stop responding > to any questions from new users. I find this kind of response surprising. Would you eventually like to benefit from additional views which evolved during collaboration of some years? > You are not the developer or maintainer of Coccinelle. This information is appropriate to some degree. How do you think about the evolution around roles like "intensive SmPL user", "system tester", "bug reporter" and "supporter"? * I am trying to contribute a bit also to your software. * I am active as another ordinary free software for years. > You don't represent the tool in any way. I do not represent it "officially". - But I can imagine that my software development activities have got some useful "side effects" from which a kind of "image" or "representation" could have evolved already. > A very large portion of the information that you provide is incorrect Would you like to clarify this a bit more? > or unnecessary. I got an other impression about the necessity. > I am very concerned that you will frighten people away, I can understand your concern a bit. > especially if you ask them to do things that go beyond their > intended involvement, such as your "Do you see further chances to > reduce..." comment below. The involvement can eventually increase, can't it? > The next time you answer a question from a newcomer where your answer > either includes a question or goes beyond say 3 lines of text, I will > adjust your subscription to the mailing list such that you no longer > receive mail. Did you really get so angry that you reconsider our collaboration in such a drastic way? > I'm sure you consider that you are helping out, I hope so. Did my small script examples help a bit in the requested clarification? > but you are not, I find this feedback interesting and surprising. > and it needs to stop immediately. How can software development discussions be continued after such a response? Regards, Markus ___ Cocci mailing list Cocci@systeme.lip6.fr https://systeme.lip6.fr/mailman/listinfo/cocci
Re: [Cocci] Usage of "expressions" and "identifiers" with SmPL
> To me, an identifier is something that's written literally into the > source code and cannot be meaningfully taken apart, e.g. a function > name, variable name, or the member of a struct. An expression, on the > other hand, is something that can be meaningfully split into > sub-components. Is that also how you use these terms? Yes, in principle. The Coccinelle software was designed for the generation of semantic patches. 1. Its tool "spatch" expects some input and will usually produce corresponding output. 2. One kind of such input are source files for which data processing is also directly supported if they were mostly written in a programming language like "C". 3. Another kind of required input is the specification of source code search or transformation patterns in the semantic patch language. 4. Special data processing is also possible just because the programming languages "OCaml" and "Python" can be used in SmPL script rules already. 5. There are several languages involved. SmPL script developers need to be careful about the relevant syntax context. * So while you are looking from a view of "C source", you might tend to think about "C identifiers". The semantic patch language provides metavariables which can get the data type "identifier". * But what was an "item" in the source language can become an other in the Coccinelle technology. One example is the use of a metavariable with the type "idexpression" in your case. At which places would you start to think about a code situation by the means of "expressions"? * Syntax constructs from the supported source languages are matched with "key words" that are often similar (or even identical) to the host language (within SmPL scripts). >> @ connects patterns that match the same term. So match a term against the >> explicit name ptr and also match it against an identifier expression that >> has a particular type. This is not exactly a beginner example. > > Okay, I'll just ignore that for now. > >>> I'd hate to waste your time asking tons of such trivial questions on the >>> mailing list, but I just can't find any helpful documentation at all... >> >> If you look on the web page in the papers and slides section, at the top >> there are several tutorials and overview talks, some with video. > > I think I looked at everything that is not a video. Thanks for your feedback. > But none of it mentioned that you can put an arbitrary name between the @@ I got an other impression from the available documentation. > or explained what the different metavariable types (idexpression, > expression, etc) are. I see some improvement possibilities there, too. > Anyway, enough whining. Coccinelle seems like a really useful tool, Yes, of course. > even if I'm having an impedance mismatch with its documentation. Do you see further chances to reduce this mismatch for following software developers? Regards, Markus ___ Cocci mailing list Cocci@systeme.lip6.fr https://systeme.lip6.fr/mailman/listinfo/cocci
Re: [Cocci] scripts/coccicheck: Update for a comment?
> Do I overlook any commit for the discussed source file anyhow? > https://git.kernel.org/cgit/linux/kernel/git/next/linux-next.git/log/dev-tools/coccinelle.rst It seems that I missed this file also because an other path will be appropriate. https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/log/Documentation/dev-tools/coccinelle.rst Regards, Markus ___ Cocci mailing list Cocci@systeme.lip6.fr https://systeme.lip6.fr/mailman/listinfo/cocci
Re: [Cocci] docs: Fixing "sphinxify coccinelle.txt"?
>>> Information from a commit like "docs: sphinxify coccinelle.txt and add it >>> to dev-tools" caught also my software development attention. >>> https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/commit/Documentation/coccinelle.txt?id=4b9033a33494ec9154d63e706e9e47f7eb3fd59e >>> >>> Did an other information from a comment become outdated in the script >>> "coccicheck" >>> because of such changes for the documentation format? >>> https://git.kernel.org/cgit/linux/kernel/git/next/linux-next.git/tree/scripts/coccicheck?id=c802e87fbe2d4dd58982d01b3c39bc5a781223aa#n4 >> >> How about submitting a patch to fix the problem? > > Is the published commit (from 2016-08-08 / 2016-08-18) generally questionable > as I see it by the interface "cgit" at the moment? > > * Does this one contain only the deletion of the file > "Documentation/coccinelle.txt"? It seems that I got an inappropriate impression from this kind of data display alone. The display for the changed file name contains only the desired addition. https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/commit/Documentation/dev-tools/coccinelle.rst?id=4b9033a33494ec9154d63e706e9e47f7eb3fd59e > I find another data display also interesting and more promising. > https://patchwork.kernel.org/patch/9269973/ > > * Should this patch about the desired file format conversion become available > also by the other known interfaces? The interface "cgit v0.12" does not indicate the involved "renaming" (similarity index 56%) so far which can be better seen in the downloadable patch file. Regards, Markus ___ Cocci mailing list Cocci@systeme.lip6.fr https://systeme.lip6.fr/mailman/listinfo/cocci
Re: [Cocci] scripts/coccicheck: Update for a comment?
> Either you send the patch, or I will send the patch, when I have time, > with no credit to you. I am curious when "the time" will be appropriate. > I'm not an expert on all of the future plans for the documentation, I assumed that you know a bit more about the corresponding software evolution than me. > The only think I know is where the file is now, and that the file > is perfectly readable in its rst format, and thus it seems > quite all right to reference it. I can agree to such an information in principle. > If the situation change in the future, we can have another patch > in the future. I would appreciate another clarification: Do I overlook any commit for the discussed source file anyhow? https://git.kernel.org/cgit/linux/kernel/git/next/linux-next.git/log/dev-tools/coccinelle.rst Regards, Markus ___ Cocci mailing list Cocci@systeme.lip6.fr https://systeme.lip6.fr/mailman/listinfo/cocci
Re: [Cocci] scripts/coccicheck: Update for a comment?
https://git.kernel.org/cgit/linux/kernel/git/next/linux-next.git/tree/scripts/coccicheck?id=c802e87fbe2d4dd58982d01b3c39bc5a781223aa#n4 >>> >>> How about submitting a patch to fix the problem? >> >> I would appreciate to become a bit more sure about the current storage >> location >> which should be referenced in this file. > > In the Documentation directory find . -name "*cocci*" > > gives > > ./dev-tools/coccinelle.rst > > I think you could have done that yourself. Does the usage of the format "reStructuredText" trigger further considerations like the following? * Does your feedback suggest that the path from this source file should be put in related places? * Would it make sense to store such a file name in a special variable? * When would you refer to generated file formats (like PDF) for this use case? Regards, Markus ___ Cocci mailing list Cocci@systeme.lip6.fr https://systeme.lip6.fr/mailman/listinfo/cocci
Re: [Cocci] scripts/coccicheck: Update for a comment?
>> https://git.kernel.org/cgit/linux/kernel/git/next/linux-next.git/tree/scripts/coccicheck?id=c802e87fbe2d4dd58982d01b3c39bc5a781223aa#n4 > > How about submitting a patch to fix the problem? I would appreciate to become a bit more sure about the current storage location which should be referenced in this file. Regards, Markus ___ Cocci mailing list Cocci@systeme.lip6.fr https://systeme.lip6.fr/mailman/listinfo/cocci
[Cocci] scripts/coccicheck: Update for a comment?
Hello, Information from a commit like "docs: sphinxify coccinelle.txt and add it to dev-tools" caught also my software development attention. https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/commit/Documentation/coccinelle.txt?id=4b9033a33494ec9154d63e706e9e47f7eb3fd59e Did an other information from a comment become outdated in the script "coccicheck" because of such changes for the documentation format? https://git.kernel.org/cgit/linux/kernel/git/next/linux-next.git/tree/scripts/coccicheck?id=c802e87fbe2d4dd58982d01b3c39bc5a781223aa#n4 Regards, Markus ___ Cocci mailing list Cocci@systeme.lip6.fr https://systeme.lip6.fr/mailman/listinfo/cocci
Re: [Cocci] Replacing one variable name with another
>> @name_replacement@ >> @@ >> my_type * >> -ptr >> +pointer >> ; > > I could try it, but I'm actually more interested in understanding how > this works (I already solved my primariy problem). Thanks for your interest. > Could you explain this script a bit more? Yes, of course. It shows the general structure of a simple script for the semantic patch language. Outline: * Block for definition of SmPL metavariables * Specification of C source code fragments (together with desired changes) > What exactly does "@name_replacement@" mean? Such a SmPL rule can specify a name for it. Which identifier would you choose there according to your naming preferences? http://coccinelle.lip6.fr/docs/main_grammar002.html > Does it the interpretation of "my_type *" further down? No. - I tend to use a title there which is descriptive somewhat. >> How will your software development experiments evolve for the desired >> application of the semantic patch language? > > Aeh, sorry, I don't understand the question. Do you plan to try any more SmPL scripts out? How do you think about to share further experiences? Regards, Markus ___ Cocci mailing list Cocci@systeme.lip6.fr https://systeme.lip6.fr/mailman/listinfo/cocci
Re: [Cocci] Replacing one variable name with another
> 2. ..and how would I go about if instead of the type, I want to replace >a variable name? (my_type *ptr --> my_type *pointer). Would you like to try another small SmPL script out like the following? @name_replacement@ @@ my_type * -ptr +pointer ; How will your software development experiments evolve for the desired application of the semantic patch language? Regards, Markus ___ Cocci mailing list Cocci@systeme.lip6.fr https://systeme.lip6.fr/mailman/listinfo/cocci
Re: [Cocci] Replacing one (specific!) type with another
> Based on the LWN articles that I've read, this seems like the perfect
> use-case for coccinelle.
I would expect this also if you are interested in the simple replacement
for the data type of a function parameter.
> However, somehow I'm struggling to write a patch for this.
Are you exploring the software development possibilities together with
the semantic patch language?
> All the documentation that I could get my hands on seems to describe
> more abstract changes that require the use of variables
SmPL metavariables have got their purposes. How do you think about
to discuss a bit more if you will eventually use them also?
> - but as far as I can tell, I need something much simpler:
There are some improvement opportunities in this approach.
> @@
> "struct fasel_foo must be a type name!"
> @@
How did you get the idea to place this test string there?
> - struct fasel_foo
> + struct fasel_bar
>
>
> ...if only I knew what to put between the @@.
Are the other parts of this source code search pattern also incomplete so far?
> Can someone point me in the right direction?
I hope so.
Would the following small SmPL script fit to your expectations?
@data_type_replacement@
@@
char* fasel_foo_print(
-struct fasel_foo
+struct fasel_bar
*ptr) {
// ...
return "this struct fasel_foo string should not change";
}
Regards,
Markus
___
Cocci mailing list
Cocci@systeme.lip6.fr
https://systeme.lip6.fr/mailman/listinfo/cocci
Re: [Cocci] Match #define with SmPL
> One more, I use 1.0.2 version as I had a problem with 1.0.5 > compilation previously. Thanks that you added a bit more version information to your bug report. > I will try bump to 1.0. 5 anyway and check the behavior. Thanks for your interest to try additional software variants out. I have tried also the following script out for the semantic patch language of the software combination "Coccinelle 1.0.5-00089-g0849d71-dirty (OCaml 4.03.0)" a moment ago. @macro_deletion@ @@ -#define pr_fmt(fmt) "SVM: " fmt elfring@Sonne:~/Projekte/Linux/next-patched> spatch.opt ~/Projekte/Coccinelle/Probe/delete_macro1.cocci arch/x86/kvm/svm.c init_defs_builtins: /usr/local/lib64/coccinelle/standard.h minus: parse error: File "/home/elfring/Projekte/Coccinelle/Probe/delete_macro1.cocci", line 3, column 29, charpos = 49 around = 'fmt', whole content = -#define pr_fmt(fmt) "SVM: " fmt Do you eventually know any more software developers who would like to help in the task "Completion of support for preprocessor statements by the Coccinelle software"? I would be also looking for an interface to store the text from the macro definition into a corresponding data structure. * How do think about to look at another "story" like "Checking printk() calls with SmPL" once more? https://github.com/coccinelle/coccinelle/issues/54#issuecomment-169988424 * Which metavariable types would be appropriate for the discussed source code transformation? I would like to point another detail out for further considerations according to your initial request. 1. I guess that you should use the SmPL asterisk functionality if you are interested in only finding (or "matching") a bit of code. @macro_definition@ @@ *#define pr_fmt(fmt) "SVM: " fmt init_defs_builtins: /usr/local/lib64/coccinelle/standard.h minus: parse error: File "/home/elfring/Projekte/Coccinelle/Probe/show_macro_definition1.cocci", line 3, column 29, charpos = 51 around = 'fmt', whole content = *#define pr_fmt(fmt) "SVM: " fmt 2. Do you want to try another source code adjustment out? @statement_addition@ @@ +#undef pr_fmt #define pr_fmt(fmt) "SVM: " fmt elfring@Sonne:~/Projekte/Linux/next-patched> spatch.opt ~/Projekte/Coccinelle/Probe/add_undef1.cocci arch/x86/kvm/svm.c init_defs_builtins: /usr/local/lib64/coccinelle/standard.h minus: parse error: File "/home/elfring/Projekte/Coccinelle/Probe/add_undef1.cocci", line 4, column 29, charpos = 69 around = 'fmt', whole content = #define pr_fmt(fmt) "SVM: " fmt How often would you like to use such search patterns for your source files? Regards, Markus ___ Cocci mailing list Cocci@systeme.lip6.fr https://systeme.lip6.fr/mailman/listinfo/cocci
Re: [Cocci] Querying Git commits with Prequel
> It goes into the directory specified with the --tmp argument. Does the software process "Compilation of PQL into SmPL" generate any disjunctions (and conjunctions) into the corresponding SmPL source code? Regards, Markus ___ Cocci mailing list Cocci@systeme.lip6.fr https://systeme.lip6.fr/mailman/listinfo/cocci
Re: [Cocci] Detecting undefined behavior with Coccinelle
> Is it possible to detect such code patterns (multiplication operation > followed by division to check for overflow) which invoke undefined > behavior using Coccinelle? I imagine that it is generally possible to find some update candidates for such an use case also by means of the semantic patch language. https://www.securecoding.cert.org/confluence/display/c/CC.+Undefined+Behavior * How many software development efforts would you like to invest? * Which search patterns do you find worthwhile here? * Can it be that data flow analysis will be needed to check appropriate usage of the involved variables in a safer way? Regards, Markus ___ Cocci mailing list Cocci@systeme.lip6.fr https://systeme.lip6.fr/mailman/listinfo/cocci
Re: [Cocci] Querying Git commits with Prequel
>>* How much will the data format flexibility evolve together with the >> handling >> of a key word like "flexible" for metavariables? > > No idea what this means (data format flexibility). Is this key word the only one which was added to "PQL" (while it is processed by "SmPL" in other ways)? >> 3. I get concerns from the section "5.2 Idealized syntax". >>How would you categorise it according to the Chomsky-Schützenberger >> hierarchy? > > No idea. It seems like a straightforward BNF. Does the mentioned syntax belong to the category "context-free"? > The commit range is just X..Y. X ad Y could be commit ids or tags (v4.0, > for example). Can this range be controlled or adjusted from within Prequel scripts anyhow? >> 5. Can the combination of words like "token" and "regular expression" >>be occasionally confusing? > > Does the word regular expression occur in the paper? Yes, of course. ;-) Is the section "6.1 Commit selection" worth for a few more clarifications? >> 6. How interesting will discussions become around the used hardware >>that is mentioned in the chapter "7 Evaluation" and the corresponding >>software performance? > > Not sure what there is to discuss. Examples: * Number of processor cores * Run time distribution for the involved scripts >> 7. How is the status for function and class libraries for this application >> domain? > > No idea what this means. Would you like to share any more information around the evolving code design structure? Which class libraries will eventually combine software modules? How do you think about to use another public repository interface which would be similar to the one for Coccinelle? Regards, Markus ___ Cocci mailing list Cocci@systeme.lip6.fr https://systeme.lip6.fr/mailman/listinfo/cocci
Re: [Cocci] Querying Git commits with Prequel
> Prequel is like Coccinelle, but for searching in Git commit histories. I am curious on how this software approach will evolve further. > http://prequel-pql.gforge.inria.fr/ > > Feedback is welcome. I pick your offer up now. Thus I have read your research report 8918 (Prequel: A Patch-Like Query Language for Commit History Search) to some degree. 1. I imagine that the activities behind the description for a chapter like "5 Compilation of PQL into SmPL" triggered some development challenges. Does an interface exists to view the generated SmPL source code? 2. Did this new query language influence further improvements for the semantic patch language? * Did the error reporting become a bit safer by this command usage? * How much will the data format flexibility evolve together with the handling of a key word like "flexible" for metavariables? 3. I get concerns from the section "5.2 Idealized syntax". How would you categorise it according to the Chomsky-Schützenberger hierarchy? 4. How many means do you offer to restrict commit ranges according to the section "6.1 Commit selection"? 5. Can the combination of words like "token" and "regular expression" be occasionally confusing? 6. How interesting will discussions become around the used hardware that is mentioned in the chapter "7 Evaluation" and the corresponding software performance? 7. How is the status for function and class libraries for this application domain? Regards, Markus ___ Cocci mailing list Cocci@systeme.lip6.fr https://systeme.lip6.fr/mailman/listinfo/cocci
Re: [Cocci] “coccicheck failed” during parallel program runs
> With the 4.8-rc1 kernel, when I run Coccinelle check using the command “make > coccicheck MODE=report M=”, > I get “coccicheck failed” message. I get the same failure message even with > “MODE=patch” > in the command. I noticed this error message also. There are further software possibilities. > I didn’t see this error message with 4.7 kernel. I guess that a change like "enable parmap support" (from 2016-07-22) will be relevant for further considerations here. https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/log/scripts/coccicheck > However, with the 4.8-rc1 kernel, if I turn off parallelization with J=1 > option, > I don’t see the error message. The check seem to be running successfully, > even though it takes longer to run. I tried J=2, J=4,… and I got error > messages. > So there seem to be a problem with J not equal to 1. I imagine that the Coccinelle software can occasionally return inappropriate status codes for its program implementations. https://github.com/coccinelle/coccinelle/blob/958fcf8df0bd0e464a490b86e4f78ad4daf082d1/main.ml#L1517 > Can you please let me know how can I resolve the “coccicheck failed” message? * Would you like to try any related command variants out for a moment? * How do you think about to help with any more software development resources? Regards, Markus ___ Cocci mailing list Cocci@systeme.lip6.fr https://systeme.lip6.fr/mailman/listinfo/cocci
Re: [Cocci] [PATCH v3] Coccinelle: Script to replace allocate and memset with zalloc functions
@vz_combined depends on patch && !context && !org && !report@ type T; T* pointer; +statement S; expression express; @@ pointer = - vmalloc + vzalloc (...); if (!d) S -memset(d, 0, sizeof( ( -T | -*(express) ) -)); > > Actually, this is a mess. pointer, d, and express are all supposed to be > the same thing, as they were in the original rule. > > Furthermore, this shows a reason why the original rule was better. There is a trade-off between several unique SmPL rules and other combined variants which could eventually work with two SmPL disjunctions. Regards, Markus ___ Cocci mailing list Cocci@systeme.lip6.fr https://systeme.lip6.fr/mailman/listinfo/cocci
Re: [Cocci] [PATCH v3] Coccinelle: Script to replace allocate and memset with zalloc functions
>> How do you think about the following SmPL script example? >> >> @vz_combined >> depends on patch && !context && !org && !report@ >> type T; >> T* pointer; >> +statement S; >> expression express; >> @@ >> pointer = >> - vmalloc >> + vzalloc >>(...); >> if (!d) >> S >> -memset(d, 0, sizeof( >> ( >> -T >> | >> -*(express) >> ) >> -)); > > OK, I thought you meant to make a big disjunctions for all of the before > and after functions. I imagine that it would be nice if the function name pairs could be specified in a more succinct format for the semantic patch language. But the discussed approach can work with a recent software version already. > This is a little better because it is bounded in size. Thanks … > But I don't understand why you have introduced the variable express. I have noticed that these two SmPL rules differed only in the source code search specification for the operator "sizeof". So I would prefer to express this small difference in the script directly. > The performance issue is that disjunctions on expressions, eg (A | B), are > implemented as (A | (!A & B)), ie with a negation of all the previous > options with each option. So it is better to avoid very large > disjunctions on expressions. Is the suggested SmPL disjunction still small enough for this concern? Regards, Markus ___ Cocci mailing list Cocci@systeme.lip6.fr https://systeme.lip6.fr/mailman/listinfo/cocci
Re: [Cocci] [PATCH v3] Coccinelle: Script to replace allocate and memset with zalloc functions
>> 1. Would it make sense to merge such SmPL rules into one >>so that code duplication could be reduced a bit >>in such a script? > > I think it would suffer in readability. How do you think about the following SmPL script example? @vz_combined depends on patch && !context && !org && !report@ type T; T* pointer; +statement S; expression express; @@ pointer = - vmalloc + vzalloc (...); if (!d) S -memset(d, 0, sizeof( ( -T | -*(express) ) -)); > Perhaps in performance as well. I admit that I am unsure about the run-time characteristics for my suggestion. Regards, Markus ___ Cocci mailing list Cocci@systeme.lip6.fr https://systeme.lip6.fr/mailman/listinfo/cocci
