Re: [Cocci] Coccinelle: kzalloc-simple: Add more zero allocating functions
>>> The cast is useful when it is to a non-pointer type. >> >> Will it be needed then to use an other metavariable for the assignment >> target? >> >> How much would you like to distinguish if an item should handle a pointer >> (or not)? > > The compiler will complain about an assignment between an integer and a > pointer, This tool distinguishes data types better. > so one can assume that if the code is > > x = (T *)kmalloc(); > > then x has pointer type. We “hope” so. How do you think about restrict the type for an acceptable target expression? > If it doesn't have type T *, it doesn't matter. I have got an other opinion for the corresponding specification in SmPL scripts. I would prefer to avoid another bit of logical confusion. Regards, Markus ___ Cocci mailing list Cocci@systeme.lip6.fr https://systeme.lip6.fr/mailman/listinfo/cocci
Re: [Cocci] Coccinelle: kzalloc-simple: Add more zero allocating functions
> - x = (T)kmalloc(E1,E2); > + x = (T)kzalloc(E1,E2); > > This for useless pointer cast which is done implicitily. Actually, the above rule is for the case where the cast is useful. >> >> * Have you got any special aspects in mind here? >> >> * How do you think about to restrict it for pointer data types? > > The cast is useful when it is to a non-pointer type. Will it be needed then to use an other metavariable for the assignment target? How much would you like to distinguish if an item should handle a pointer (or not)? Regards, Markus ___ Cocci mailing list Cocci@systeme.lip6.fr https://systeme.lip6.fr/mailman/listinfo/cocci
Re: [Cocci] Coccinelle: kzalloc-simple: Add more zero allocating functions
>>> - x = (T)kmalloc(E1,E2); >>> + x = (T)kzalloc(E1,E2); >>> >>> This for useless pointer cast which is done implicitily. >> >> Actually, the above rule is for the case where the cast is useful. * Have you got any special aspects in mind here? * How do you think about to restrict it for pointer data types? >> The (T *) rule should be above this one, because that is for the case >> where the cast is not needed. I don't remember if this is done correctly. >> Please check. > > Oops! Yes, I somehow hurriedly made the typo. Does such a kind of feedback mean that the specification of pointer casts will be adjusted once more for this evolving SmPL script? Would you like to express the handling of optional items in a specific way? Regards, Markus ___ Cocci mailing list Cocci@systeme.lip6.fr https://systeme.lip6.fr/mailman/listinfo/cocci
Re: [Cocci] [PATCH] Coccinelle: kzalloc-simple: Add more zero allocating functions
> We already have zero memory allocator functions to set the memory to > 0 value instead of manually setting it using memset. Thanks for your extension of this script for the semantic patch language. Will this update suggestion get any better chances than the approach “Script to replace allocate and memset with zalloc functions”? https://systeme.lip6.fr/pipermail/cocci/2016-August/003510.html > +/// Use zeroing allocator rather than allocator followed by memset with 0 Do you find the shown function name list complete now? Did you omit a name like “kvm_kvzalloc” intentionally? How do you think about the possibility to analyse relevant source files for functions with the mentioned property? > +( > +- x = kmalloc(E1,E2); > ++ x = kzalloc(E1,E2); > +| You suggest to use another application for the SmPL disjunction. How do you think about to refactor this specification a bit like the following? +( + x = +-kmalloc ++kzalloc +(E1, E2); +| > +| > +- x = (T *)kmalloc(E1,E2); > + x = kzalloc(E1,E2); > +| Why do you find it appropriate to omit a cast at this place while it is preserved at other places? Regards, Markus ___ Cocci mailing list Cocci@systeme.lip6.fr https://systeme.lip6.fr/mailman/listinfo/cocci
Re: [Cocci] Transformation challenges with SmPL around “LIST_FOR_EACH_ENTRY”
>> @adjustment@ >> expression ex; >> statement S; >> @@ >> -LIST_FOR_EACH_ENTRY >> +hlist_for_each_entry >> (..., ..., >> -ex, >> ...) >> S > > This is not completely a good idea. I tried to show another approach. > The ... in the argument list will match a sequence of things, not a single > thing. How does this information fit to the specification of the SmPL ellipsis as a placeholder for the last macro parameter? > It could be: > > -LIST_FOR_EACH_ENTRY > +hlist_for_each_entry > (e1,e2, Are these metavariables required for such an use case? > -ex, > ...) > S Can such a transformation variant have nicer run time characteristics in comparison to the initial SmPL script example? Regards, Markus ___ Cocci mailing list Cocci@systeme.lip6.fr https://systeme.lip6.fr/mailman/listinfo/cocci
Re: [Cocci] Transformation challenges with SmPL around “LIST_FOR_EACH_ENTRY”
>> I think I've found a bug in spatch, You showed another opportunity for further development considerations. >> or maybe I'm just using it wrong. Not really. But the specification in the shown small SmPL script could be adjusted. >> - LIST_FOR_EACH_ENTRY(c,f,g,member) S >> + hlist_for_each_entry(c,f,member) S > Actually, the problem is that you have removed S and dded it back. > Then you are relying on Coccinelle to do the pretty printing, > and all comments will be dropped. Just rewrite your rule as follows, > and everything will be fine: > - LIST_FOR_EACH_ENTRY(c,f,g,member) > + hlist_for_each_entry(c,f,member) > S How do you think about further possibilities? Can it be also sufficient to express only the source code adjustment for two details? * Replacement of a macro name * Deletion (or omission) of a parameter @adjustment@ expression ex; statement S; @@ -LIST_FOR_EACH_ENTRY +hlist_for_each_entry (..., ..., -ex, ...) S Regards, Markus ___ Cocci mailing list Cocci@systeme.lip6.fr https://systeme.lip6.fr/mailman/listinfo/cocci
Re: [Cocci] [v4] Coccinelle: fix parallel build with CHECK=scripts/coccicheck
> Setting NPROC=1 is a reasonable solution; It can be sufficient for the usual purposes of the shell script “scripts/coccicheck”. > spatch does not create the subdirectory. I would like to point out that further development efforts will be needed if such a special directory handling should be avoided generally. How much do care about consequences if parameters for additional parallelisation would be passed over the command line option (or the environment variable) “SPFLAGS” as it is described in the file “Documentation/dev-tools/coccinelle.rst”? Regards, Markus ___ Cocci mailing list Cocci@systeme.lip6.fr https://systeme.lip6.fr/mailman/listinfo/cocci
Re: [Cocci] Coccinelle: fix parallel build with CHECK=scripts/coccicheck
>>> I didn't want to use a name with the pid, so that one could easily find >>> this information while Coccinelle is running. >> >> Do you mark these data as “hidden” in the file system? > > I don't know what this means. Do you fiddle with file attributes or use a dot as the first character in the file name? >> I imagine that the storage algorithm could become better configurable >> besides the parameters “--temp-files” and “--tmp-dir”. > > The problem is that running multiple coccinelles each with parallelism is > fairly pointless, The situation might be different if the execution environment provides enough computation resources for processing with special data distributions. > especially on the same semantic patch. There can be restrictions usually. > So the only people who do it will be those who don't understand > how Coccinelle works. This might be. - Or do they compare different software versions (in parallel)? > Those people won't realize that they should use such extra options either. Some time is needed to become more aware about advanced parameters. Regards, Markus ___ Cocci mailing list Cocci@systeme.lip6.fr https://systeme.lip6.fr/mailman/listinfo/cocci
[Cocci] Setting the section for a function with SmPL
Hello,
Implementations of various functions will be changed on demand. It can happen
then that you would like to know if the applied changes will result also in
differences for the generated code.
Development tools like “size” can display some information for such a
comparison.
The total size display might be occasionally not so useful. I imagine that it
can help to split a default section configuration into smaller parts.
https://gcc.gnu.org/onlinedocs/gcc-7.2.0/gcc/Common-Function-Attributes.html#index-section-function-attribute
I would like to append a corresponding attribute to a function signature by the
means of the semantic patch language.
@section_specification@
type T;
@@
T rcar_pcie_enable_msi(...)
+ __attribute__((section ("my_update")))
{
...
}
Unfortunately, the software “Coccinelle 1.0.6-00345-g2ca0befa” does not like
such an approach so far.
elfring@Sonne:~/Projekte/Linux/next-patched> spatch
~/Projekte/Coccinelle/janitor/set_section1.cocci drivers/pci/host/pcie-rcar.c
…
plus: parse error:
File "/home/elfring/Projekte/Coccinelle/janitor/set_section1.cocci", line 6,
column 1, charpos = 105
around = '{',
whole content = {
Can the software situation be improved anyhow for this use case?
Regards,
Markus
___
Cocci mailing list
Cocci@systeme.lip6.fr
https://systeme.lip6.fr/mailman/listinfo/cocci
Re: [Cocci] make install broken
>> Which information are you missing from my software surprises? > You just showed the error that looks similar to mine. I have tried another rebuild of the current software out. > You miss the part about how you configured and installed it. Would you like to take another look at affected implementation details? > And what exact commit you've used. elfring@Sonne:~/Projekte/Coccinelle/20160205> make distclean && git checkout master && git pull && ./autogen && ./configure && echo "$(./version.sh)" && grep VERSION=1 Makefile.config … 1.0.6-00335-g6e5973d4 VERSION=1.0.6-00335-g6e5973d4 elfring@Sonne:~/Projekte/Coccinelle/20160205> make world … Sonne:~ # cd /home/elfring/Projekte/Coccinelle/20160205 && make install … /usr/bin/install -c -m 755 spatch.opt /usr/local/bin/spatch /usr/bin/install -c -m 644 standard.h /usr/local/lib64/coccinelle /usr/bin/install -c -m 644 standard.iso /usr/local/lib64/coccinelle … /usr/bin/install -c -m 644 python/coccilib/*.py \ /usr/local/lib64/coccinelle/python/coccilib … elfring@Sonne:~> /usr/local/bin/spatch --version spatch version 1.0.6-00335-g6e5973d4 compiled with OCaml version 4.05.0 … elfring@Sonne:~/Projekte/Linux/next-patched> /usr/local/bin/spatch ~/Projekte/Coccinelle/janitor/use_seq_putc_or_seq_puts-20161027-a.cocci drivers/staging/irda/net/irlmp.c warning: Can't find macro file: /usr/local/bin/../lib/coccinelle/standard.h warning: Can't find default iso file: /usr/local/bin/../lib/coccinelle/standard.iso Python error: No module named coccilib.elems * How should the directory “lib64” be determined for this use case? * Do I need to add any symbolic links temporarily? * Will it help to add an installation test to the software build system? Regards, Markus ___ Cocci mailing list Cocci@systeme.lip6.fr https://systeme.lip6.fr/mailman/listinfo/cocci
Re: [Cocci] make install broken
> Do what I did: > Describe the problem in a way that makes it easy to reproduce. I find that did this already, didn't I? Which information are you missing from my software surprises? Regards, Markus ___ Cocci mailing list Cocci@systeme.lip6.fr https://systeme.lip6.fr/mailman/listinfo/cocci
Re: [Cocci] make install broken
>> I find that did this already, didn't I? > No , you didn't. I can try to repeat some more technical details. >> Which information are you missing from my software surprises? > You just showed the error that looks similar to mine. Yes. - I submitted only a kind of “ping” yesterday. > You miss the part about how you configured and installed it. The maintainers are completely informed. > And what exact commit you've used. I hope that I am using the published GitHub version mostly. How do you think about to take another look at any of my previous clarification requests? * Further improvements for safer building of the Coccinelle software https://systeme.lip6.fr/pipermail/cocci/2017-September/004522.html * Checking evolution for the build system of the Coccinelle software https://systeme.lip6.fr/pipermail/cocci/2017-September/004524.html Regards, Markus ___ Cocci mailing list Cocci@systeme.lip6.fr https://systeme.lip6.fr/mailman/listinfo/cocci
Re: [Cocci] make install broken
>> Is this software development also relevant for the following error messages? >> >> elfring@Sonne:~/Projekte/Linux/next-patched> spatch >> ~/Projekte/Coccinelle/janitor/….cocci ….c >> warning: Can't find macro file: /usr/local/bin/../lib/coccinelle/standard.h >> warning: Can't find default iso file: >> /usr/local/bin/../lib/coccinelle/standard.iso >> Python error: No module named coccilib.elems > Correct. It fixes that issue. It is nice that the current tool is working for you now as expected. I am curious on what I need to change more so that I can report the same for my test system. Would you like to contribute further ideas? Regards, Markus ___ Cocci mailing list Cocci@systeme.lip6.fr https://systeme.lip6.fr/mailman/listinfo/cocci
[Cocci] Checking for consistency of the version data by the software build system
Hello, The following command was executed on my test system again. elfring@Sonne:~/Projekte/Coccinelle/20160205> make distclean && git checkout master && git pull && ./autogen && ./configure && echo "$(./version.sh)" && grep VERSION=1 Makefile.config … 1.0.6-00335-g6e5973d4 VERSION=1.0.6-00318-g0bf53049 An essential build script contains the macro call “AC_INIT(…, m4_esyscmd([./version.sh]), …)”. https://github.com/coccinelle/coccinelle/blob/436cc65a41ce2c5885cc7f1ee09b6c14fac1981c/configure.ac#L6 So I would expect that the determined version data will be immediately reused in generated files. But I notice the shown mismatch occasionally. Do you observe also similar differences in other software build environments? Regards, Markus ___ Cocci mailing list Cocci@systeme.lip6.fr https://systeme.lip6.fr/mailman/listinfo/cocci
Re: [Cocci] Further improvements for safer building of the Coccinelle software
> What are you wondering about it? Was the make script changed in the way that the native executable variant will not be reinstalled with the suffix “opt” any more? https://github.com/coccinelle/coccinelle/blob/db8e301e8b28f20149d5edccc996cf6843047f7a/Makefile#L260 Do you find the following command relevant for further considerations? Sonne:/home/elfring/Projekte/Coccinelle/20160205 # make install … /usr/bin/install -c -m 755 spatch.opt /usr/local/bin/spatch … Regards, Markus ___ Cocci mailing list Cocci@systeme.lip6.fr https://systeme.lip6.fr/mailman/listinfo/cocci
Re: [Cocci] Checking statement order for patch generation with SmPL support
>> I wonder also about the information how an ordinary for loop could influence >> the shown source code analysis result for the function >> “snd_seq_queue_find_name” >> when the questionable marked statements are contained in a single if branch. >> http://elixir.free-electrons.com/linux/v4.13/source/sound/core/seq/seq_queue.c#L241 >> https://git.kernel.org/pub/scm/linux/kernel/git/next/linux-next.git/tree/sound/core/seq/seq_queue.c?id=c6be5a0e3cebc145127d46a58350e05d2bcf6323#n252 > > I don't understand the question. I hope that we can achieve a better common understanding also for the mentioned test examples. > I already explained the issue here. You tried it. > Execution can go from queuefree to the top of the loop, to the first if in > the loop to the second if in the loop that has the dereference. That is > how loops work. I can agree to this view. But I find the existence of a loop not so relevant for the source code search pattern which is discussed. >> @@ -1199,14 +1199,11 @@ void dpcm_be_disconnect(struct snd_soc_p >> stream ? "<-" : "->", dpcm->be->dai_link->name); >> >> /* BEs still alive need new FE */ >> -dpcm_be_reparent(fe, dpcm->be, stream); >> >> #ifdef CONFIG_DEBUG_FS >> -debugfs_remove(dpcm->debugfs_state); >> #endif >> list_del(>list_be); >> list_del(>list_fe); >> -kfree(dpcm); >> } >> } >> >> >> I find the shown matches also questionable for this test result. >> Would you like to clarify such software situations a bit more >> for the desired handling of statement sequences? > > The list_for_each_entry_safe operator also makes a loop. Yes. - But how could the Coccinelle software know more about this identifier during execution of the small script “show_use_after_free3.cocci” than that it is a macro call in the implementation of the function “dpcm_be_disconnect” (when extra include parameters were not specified)? http://elixir.free-electrons.com/linux/v4.13/source/include/linux/list.h#L542 http://elixir.free-electrons.com/linux/v4.13/source/sound/soc/soc-pcm.c#L1184 Do you find the minus characters appropriate at the beginning of these three lines? Regards, Markus ___ Cocci mailing list Cocci@systeme.lip6.fr https://systeme.lip6.fr/mailman/listinfo/cocci
Re: [Cocci] Checking statement order for patch generation with SmPL support
> Thers is a control-flow path from the bottom of a loop back up to the top. I wonder also about the information how an ordinary for loop could influence the shown source code analysis result for the function “snd_seq_queue_find_name” when the questionable marked statements are contained in a single if branch. http://elixir.free-electrons.com/linux/v4.13/source/sound/core/seq/seq_queue.c#L241 https://git.kernel.org/pub/scm/linux/kernel/git/next/linux-next.git/tree/sound/core/seq/seq_queue.c?id=c6be5a0e3cebc145127d46a58350e05d2bcf6323#n252 I have tried the following SmPL script variant out on another source file. @usage@ identifier action!~"^.+free$", member, release=~"^.+free$"; expression context, ex; @@ *release(context); ... when != context = ex when any *action(..., (context)->member, ...) elfring@Sonne:~/Projekte/Linux/next-patched> spatch.opt ~/Projekte/Coccinelle/janitor/show_use_after_free3.cocci sound/soc/soc-pcm.c … @@ -1199,14 +1199,11 @@ void dpcm_be_disconnect(struct snd_soc_p stream ? "<-" : "->", dpcm->be->dai_link->name); /* BEs still alive need new FE */ - dpcm_be_reparent(fe, dpcm->be, stream); #ifdef CONFIG_DEBUG_FS - debugfs_remove(dpcm->debugfs_state); #endif list_del(>list_be); list_del(>list_fe); - kfree(dpcm); } } I find the shown matches also questionable for this test result. Would you like to clarify such software situations a bit more for the desired handling of statement sequences? Regards, Markus ___ Cocci mailing list Cocci@systeme.lip6.fr https://systeme.lip6.fr/mailman/listinfo/cocci
Re: [Cocci] Checking statement order for patch generation with SmPL support
>> But I have got difficulties to interpret it in an useful way. > > Coccinelle follows control-flow paths. This information is generally fine. > Thers is a control-flow path from the bottom of a loop back up to the top. I can not follow with my intermediate understanding to such a view at the moment. Why was the combination of a call for the function “strncmp” before a “queuefree” really displayed as an update candidate? >>> @usage@ >>> identifier action, member, release=~"^.+free$"; >>> expression context,e; >>> @@ >>> *release(context); >>> ... when != context = e >>> when any // to get all results >>> *action(..., (context)->member, ...) >> >> Should the SmPL construct “<+.. ...+>” work also similar to your suggestion? > > <+... ...+> would also not allow context = e after the last match of the > pattern inside the nest. This information sounds promising. I am looking still for possibilities to clarify the overlap better in the shown functionality. > A when on a <+... ...+> applies to the entire matched region. Can the parameter “when” be combined with this construct anyhow? Regards, Markus ___ Cocci mailing list Cocci@systeme.lip6.fr https://systeme.lip6.fr/mailman/listinfo/cocci
Re: [Cocci] Checking statement order for patch generation with SmPL support
>> Now I wonder why the software “Coccinelle 1.0.6-00242-g3f038a5d” finds >> this place relevant when the function call sequence does not fit to the order >> I tried to express for a known use case. >> I would appreciate further advice. > > Because there is a loop, This information is appropriate. But I have got difficulties to interpret it in an useful way. > and you did nothing to prevent an update to q because the free and the > dereference. I omitted an additional constraint for a simple test. > @usage@ > identifier action, member, release=~"^.+free$"; > expression context,e; > @@ > *release(context); > ... when != context = e > when any // to get all results > *action(..., (context)->member, ...) Should the SmPL construct “<+.. ...+>” work also similar to your suggestion? Regards, Markus ___ Cocci mailing list Cocci@systeme.lip6.fr https://systeme.lip6.fr/mailman/listinfo/cocci
[Cocci] Checking statement order for patch generation with SmPL support
Hello,
I have constructed another small script for the semantic patch language.
@usage@
identifier action, member, release=~"^.+free$";
expression context;
@@
*release(context);
<+...
*action(..., (context)->member, ...)
...+>
The following source code place can be found by such a simple approach
for further software development considerations.
https://lkml.org/lkml/2017/9/6/669
elfring@Sonne:~/Projekte/Linux/next-patched> git checkout next-20170905 &&
spatch.opt ~/Projekte/Coccinelle/janitor/show_use_after_free1.cocci
sound/pci/ymfpci/ymfpci.c
…
@@ -336,8 +336,6 @@ static int snd_card_ymfpci_probe(struct
legacy_ctrl &= ~YMFPCI_LEGACY_FMEN;
pci_write_config_word(pci, PCIR_DSXG_LEGACY,
legacy_ctrl);
} else if ((err = snd_opl3_hwdep_new(opl3, 0, 1, NULL)) < 0) {
- snd_card_free(card);
- dev_err(card->dev, "cannot create opl3 hwdep\n");
return err;
}
}
I have tried the SmPL script out on another source file.
elfring@Sonne:~/Projekte/Linux/next-patched> spatch.opt
~/Projekte/Coccinelle/janitor/show_use_after_free1.cocci
sound/core/seq/seq_queue.c
…
@@ -246,9 +246,7 @@ struct snd_seq_queue *snd_seq_queue_find
for (i = 0; i < SNDRV_SEQ_MAX_QUEUES; i++) {
if ((q = queueptr(i)) != NULL) {
- if (strncmp(q->name, name, sizeof(q->name)) == 0)
return q;
- queuefree(q);
}
}
return NULL;
Now I wonder why the software “Coccinelle 1.0.6-00242-g3f038a5d” finds
this place relevant when the function call sequence does not fit to the order
I tried to express for a known use case.
I would appreciate further advice.
Regards,
Markus
___
Cocci mailing list
Cocci@systeme.lip6.fr
https://systeme.lip6.fr/mailman/listinfo/cocci
Re: [Cocci] [PATCH v4] Coccinelle: add atomic_as_refcounter script
> > Acked-by: Julia Lawall > Thank you very much Julia! I find that this acknowledgement could be presented a bit too early. > What is the correct path to merge this? It seems that you prefer to ignore some of my technical arguments (or concerns?). I hope that a safer source code search approach can still be achieved here. Regards, Markus ___ Cocci mailing list Cocci@systeme.lip6.fr https://systeme.lip6.fr/mailman/listinfo/cocci
Re: [Cocci] [PATCH v4] Coccinelle: add atomic_as_refcounter script
Dear Elena, I hoped on a more constructive feedback for remaining implementation details which should be reconsidered once more. https://systeme.lip6.fr/pipermail/cocci/2017-August/004341.html * Will any more revisions be needed for your evolving script until special development concerns will be taken into account appropriately? * How do you prefer to clarify unsafe search patterns finally? Regards, Markus ___ Cocci mailing list Cocci@systeme.lip6.fr https://systeme.lip6.fr/mailman/listinfo/cocci
Re: [Cocci] [PATCH v4] provide rule for finding refcounters
> changes in v4: > … Do you find this cover letter still relevant for the clarification of remaining open issues in a single SmPL script? Regards, Markus ___ Cocci mailing list Cocci@systeme.lip6.fr https://systeme.lip6.fr/mailman/listinfo/cocci
Re: [Cocci] Addition of support for conjunctions on types
>> I am looking for descriptions about circumstances under which the >> metavariable combination >> will be useful by the mentioned SmPL conjunctions. > > I still don't understand the question. It seems that I have got a few understanding difficulties with the added functionality. > Another example is > > ( > ( > struct i > | > union i > ) > & > t > ) > > Now you have one metavariable, t, that stores the whole type, I find this technical detail unclear. > whether it is a struct or a union. But this information sounds promising. Regards, Markus ___ Cocci mailing list Cocci@systeme.lip6.fr https://systeme.lip6.fr/mailman/listinfo/cocci
Re: [Cocci] Addition of support for conjunctions on types
> I have no idea what information is wanted. I am looking for descriptions about circumstances under which the metavariable combination will be useful by the mentioned SmPL conjunctions. Regards, Markus ___ Cocci mailing list Cocci@systeme.lip6.fr https://systeme.lip6.fr/mailman/listinfo/cocci
Re: [Cocci] Addition of support for conjunctions on types
>> I would like to know a bit more for the application of such a metavariable >> combination. I would appreciate another feedback for this aspect. >> Unfortunately, I get the information “… Fatal error occurred, no output PDF >> file produced! …” from data processing by the command “make docs”. > > Fixed. Thanks for the report. Thanks for your quick software update “latex fix”. Regards, Markus ___ Cocci mailing list Cocci@systeme.lip6.fr https://systeme.lip6.fr/mailman/listinfo/cocci
Re: [Cocci] Addition of support for conjunctions on types
> @@ > type t; > identifier i; > @@ > > ( > struct i > & > t > ) > > will now bind both i and t appropriately. I would like to know a bit more for the application of such a metavariable combination. I hoped also to read something about it in the current manual. Unfortunately, I get the information “… Fatal error occurred, no output PDF file produced! …” from data processing by the command “make docs”. Regards, Markus ___ Cocci mailing list Cocci@systeme.lip6.fr https://systeme.lip6.fr/mailman/listinfo/cocci
Re: [Cocci] Addition of support for variable attributes
> It is now (github) possible to match and transform a single such attribute > when it comes after the name of a variable in a variable declaration. This is a nice extension for your software. How does this information fit to the wording in the manual? https://github.com/coccinelle/coccinelle/blob/2776ede39f6f4197eaa0fc8344152e5ed07e0ad3/docs/manual/cocci_syntax.tex#L308 “… It is not possible to match or remove an attribute, only to add one. …” > attribute name __ro_after_init; Do you imagine any further software evolution if you start the specification of this metavariable type with two words? Regards, Markus ___ Cocci mailing list Cocci@systeme.lip6.fr https://systeme.lip6.fr/mailman/listinfo/cocci
Re: [Cocci] Matching function prototypes and casts
>> ( >> *setup_timer(>_timer@tl, \((_func)\| \) \(&\| \) _callback, \((_data)\| >> \) E); >> | >> *E->_timer@tl.function = \((_func)\| \) \(&\| \) _callback; >> ) > > No, it doesn't work, because \( \| \) matches an expression and there is > no empty expression. Will the Coccinelle software become able to generate the desired combination of casts and address-of operators anyhow automatically? Can the usage of optional items be expressed in a more convenient way here? Regards, Markus ___ Cocci mailing list Cocci@systeme.lip6.fr https://systeme.lip6.fr/mailman/listinfo/cocci
Re: [Cocci] Matching function prototypes and casts
> I had --no-includes in my .cocci. Would you like to transform also any header files by the mentioned approach? > More insane corner cases: Are you looking for further clarification there? Regards, Markus ___ Cocci mailing list Cocci@systeme.lip6.fr https://systeme.lip6.fr/mailman/listinfo/cocci
Re: [Cocci] Matching function prototypes and casts
>> You try to search for variations of a function call and an assignment to >> a data structure member. It seems that they differ then by the usage of a >> cast >> and the operator “&”. >> How do you think about to express such search criteria in a succinct way with >> the help of the semantic patch language? >> >> I imagine that nested disjunctions or the SmPL operator “?” could be used >> for your source code transformation approach. > > I don't know what you have in mind, Would the following design suggestion be usable? @check_timer_function_usage@ expression _callback, E; struct timer_list tl; identifier _timer; type _func, _data; @@ ( *setup_timer(>_timer@tl, \((_func)\| \) \(&\| \) _callback, \((_data)\| \) E); | *E->_timer@tl.function = \((_func)\| \) \(&\| \) _callback; ) I wonder that “E” should be passed twice to the shown function call. > but ? only works on top-level terms (statements, loop headers, etc), not > nested expressions. I assume that this functionality can occasionally be used more often. Regards, Markus ___ Cocci mailing list Cocci@systeme.lip6.fr https://systeme.lip6.fr/mailman/listinfo/cocci
Re: [Cocci] Matching function prototypes and casts
> ( > -setup_timer(&_E->_timer@_e, _callback, _E); > | … > -_E->_timer@_e.function = (_cast_func)&_callback; > ) You try to search for variations of a function call and an assignment to a data structure member. It seems that they differ then by the usage of a cast and the operator “&”. How do you think about to express such search criteria in a succinct way with the help of the semantic patch language? I imagine that nested disjunctions or the SmPL operator “?” could be used for your source code transformation approach. Regards, Markus ___ Cocci mailing list Cocci@systeme.lip6.fr https://systeme.lip6.fr/mailman/listinfo/cocci
Re: [Cocci] parsing of C code
> More importantly; does Julia like red jelly beans more than blue jelly beans? Would you like to discuss favourite sweets more than to clarify further improvements in parsing technology also for application together with the Coccinelle software? ;-) Regards, Markus ___ Cocci mailing list Cocci@systeme.lip6.fr https://systeme.lip6.fr/mailman/listinfo/cocci
Re: [Cocci] parsing of C code
> Parsing of foo fails due to the attribute __xxx(yyy) that Coccinelle is > not able to cope with. * Do you find information relevant from answers to a question like “Context-free grammars versus context-sensitive grammars?”? https://stackoverflow.com/questions/8236422/context-free-grammars-versus-context-sensitive-grammars#answer-8250104 * Do the “attributes” which you would like to support trigger a need to work with context-dependent grammars? * Did you ask any other developers (or software designers) for possible solutions around the mentioned aspect? * Will the software situation improve any more also for the programming language “OCaml” (besides tools like “Menhir”)? Regards, Markus ___ Cocci mailing list Cocci@systeme.lip6.fr https://systeme.lip6.fr/mailman/listinfo/cocci
Re: [Cocci] parsing of C code
>> If it's yacc based you can recover where ever you like. Knowing how to >> do it is something of a black art. > Well, ocamlyacc, to be precise. Can the software “Menhir” help any more for the needed data processing? http://gallium.inria.fr/%7Efpottier/menhir/ Regards, Markus ___ Cocci mailing list Cocci@systeme.lip6.fr https://systeme.lip6.fr/mailman/listinfo/cocci
Re: [Cocci] parsing of C code
> I have tried to improve the parsing of C code recently. This information is useful. > 1. More aggressive inclusion of header files, Why do you need to become “aggressive” there when the corresponding data processing should be just correct? > combined with caching of header files. How do you think about to configure this approach with special parameters? > flush stdout Can such a function call be omitted here? > If some functions are missing in the output when using the new version, > as compared to the output when using the old version, and if these functions > are things you might want to process in some way, thenlet me know about > the problem. I became more curious on the run time characteristics (as you already know) for the presented software evolution. I stumbled on some execution challenges during the application of special source code search patterns. How would they increase because of the desired inclusion of additional source (or header) files for analysis approaches which would need them? Regards, Markus ___ Cocci mailing list Cocci@systeme.lip6.fr https://systeme.lip6.fr/mailman/listinfo/cocci
Re: [Cocci] parsing of C code
> At least for the Linux kernel, you can't just run one make and get all the > files to be compiled. Some files are indeed very hard to compile. How do you think about to point any specific source code examples out which you find a bit too challenging so far? Regards, Markus ___ Cocci mailing list Cocci@systeme.lip6.fr https://systeme.lip6.fr/mailman/listinfo/cocci
Re: [Cocci] Comparing statement lists with SmPL
> If you like the results from the second case, what more do you want? I would like to achieve somehow that the number of presented “false positives” will become so low so that similar (and extended) SmPL scripts can be used for more automatic source code transformations with higher confidence. * How are are the chances to integrate corresponding variants into a script collection for which you care also? https://git.kernel.org/pub/scm/linux/kernel/git/next/linux-next.git/tree/scripts/coccinelle/ * How much does the attention for related issues fit to the software development capacity which is available so far? Regards, Markus ___ Cocci mailing list Cocci@systeme.lip6.fr https://systeme.lip6.fr/mailman/listinfo/cocci
Re: [Cocci] Comparing statement lists with SmPL
>> How should the source code search be improved further here? > > If you like the results from the second case, what more do you want? * There are also several questionable transformation suggestions generated besides the usable ones. * Will answers belong also to the topic “Get the non-optional nest construct completely working for SmPL”? https://github.com/coccinelle/coccinelle/issues Regards, Markus ___ Cocci mailing list Cocci@systeme.lip6.fr https://systeme.lip6.fr/mailman/listinfo/cocci
Re: [Cocci] Comparing statement lists with SmPL
> You would need when any on the ... Otherwise, it will not match anything
> (perhaps declarations). The s1 after is precludes matching any statement.
> Likewise in the next if.
The suggested variant does still not work in the way I would expect it.
@duplicated_code@
identifier work;
statement s1, s2;
type T;
@@
T work(...)
{
... when any
*if ((...) < 0)
*{
... when any
* s1
* s2
*}
<+...
*if ((...) < 0)
*{
... when any
* s1
* s2
*}
...+>
}
I got some promising test results from the following SmPL approach.
@duplicated_code@
identifier work;
statement s1, s2;
type T;
@@
T work(...)
{
... when any
*if ((...) < 0)
*{
... when any
* s1
* s2
*}
... when any
*if ((...) < 0)
*{
... when any
* s1
* s2
*}
... when any
}
elfring@Sonne:~/Projekte/Linux/next-patched> XX=$(date) && spatch.opt --timeout
12 --sp-file ~/Projekte/Coccinelle/janitor/show_same_statements3b.cocci --dir
sound >
~/Projekte/Bau/Linux/scripts/Coccinelle/tuning1/next/20170803/same_statements3b.diff
2>
~/Projekte/Bau/Linux/scripts/Coccinelle/tuning1/next/20170803/same_statements3b-errors.txt;
YY=$(date) && echo "$XX | $YY"
Mi 23. Aug 17:25:54 CEST 2017 | Mi 23. Aug 17:29:15 CEST 2017
How should the source code search be improved further here?
Regards,
Markus
___
Cocci mailing list
Cocci@systeme.lip6.fr
https://systeme.lip6.fr/mailman/listinfo/cocci
Re: [Cocci] Comparing statement lists with SmPL
> I have tried another variant out for a source code analysis.
Does the following SmPL script variant make sense?
@duplicated_code@
identifier work;
statement s1, s2;
type T;
@@
T work(...)
{
... when any
*if ((...) < 0)
*{
...
* s1
* s2
*}
<+...
*if ((...) < 0)
*{
...
* s1
* s2
*}
...+>
}
Can this approach find interesting places in any source files?
Regards,
Markus
___
Cocci mailing list
Cocci@systeme.lip6.fr
https://systeme.lip6.fr/mailman/listinfo/cocci
Re: [Cocci] Comparing statement lists with SmPL
>>> The pattern matched in two different functions? >> >> Please look once more. >> >> Yes. - This is one of the reasons why I ask here again. >> >> I would like to choose if a code analysis should happen only within a single >> function >> or on interesting parts from the complete source file. > > Feel free to make a tool of your own if you are not satisfied with the one > offered you. This is also possible. - But I wonder more about specific test results based on the currently provided software. I got matches for places where I would expect that they should not match. > Coccinelle applies a rule to one function at a time. This is also fine for a while. > That is not going to change. I imagine that the situation could evolve more if additional development resources will appear somehow. > If you want to match things in different functions, make two rules. Additional SmPL rules will search for other details. Regards, Markus ___ Cocci mailing list Cocci@systeme.lip6.fr https://systeme.lip6.fr/mailman/listinfo/cocci
Re: [Cocci] Comparing statement lists with SmPL
>> How do the presented functions “usb6fire_fw_ezusb_upload” and >> “usb6fire_fw_fpga_upload” >> fit to this information? > > The pattern matched in two different functions? Please look once more. Yes. - This is one of the reasons why I ask here again. I would like to choose if a code analysis should happen only within a single function or on interesting parts from the complete source file. Regards, Markus ___ Cocci mailing list Cocci@systeme.lip6.fr https://systeme.lip6.fr/mailman/listinfo/cocci
Re: [Cocci] Comparing statement lists with SmPL
>> * A test result was shown from two function implementations. >> It can be nice to compare several functions. >> Is it also possible that the comparison will be only performed within >> the same function bodies? > > If you make a single rule then it will only applied within individual > functions. How do the presented functions “usb6fire_fw_ezusb_upload” and “usb6fire_fw_fpga_upload” fit to this information? Regards, Markus ___ Cocci mailing list Cocci@systeme.lip6.fr https://systeme.lip6.fr/mailman/listinfo/cocci
Re: [Cocci] Comparing statement lists with SmPL
>> How would you like to treat variations in such log messages? > > They look fine as they are. I have got other development opinions there depending on the preferred design goals. * The mentioned small SmPL script can also help to find special differences. * These log statements are similar (and not identical) in the code structure. I wonder also about other marked code places how they would fit to the search pattern. * A test result was shown from two function implementations. It can be nice to compare several functions. Is it also possible that the comparison will be only performed within the same function bodies? Regards, Markus ___ Cocci mailing list Cocci@systeme.lip6.fr https://systeme.lip6.fr/mailman/listinfo/cocci
Re: [Cocci] Comparing statement lists with SmPL
>> * Are there further development challenges to consider for the safe
>> identification
>> of unique statements by metavariables?
>
> If you want to ensure that two metavariables match things
> in different places, then put a position variable on each match
> and use apython rule afterwards to discard the occurrences that
> are both in the same position.
I imagine that such an approach can only matter if there will be
several SmPL rules involved.
I have tried another variant out for a source code analysis.
@duplicated_code@
identifier work;
statement s1, s2;
type T;
@@
T work(...)
{
<+...
*if ((...) < 0)
*{
...
* s1
* s2
*}
...+>
<+...
*if ((...) < 0)
*{
...
* s1
* s2
*}
...+>
}
elfring@Sonne:~/Projekte/Linux/next-patched> XX=$(date) && spatch.opt -timeout
34 -j 4 --chunksize 1 --sp-file
~/Projekte/Coccinelle/janitor/show_same_statements3.cocci --dir sound >
~/Projekte/Bau/Linux/scripts/Coccinelle/tuning1/next/20170803/same_statements3+.diff
2>
~/Projekte/Bau/Linux/scripts/Coccinelle/tuning1/next/20170803/same_statements3+-errors.txt;
YY=$(date) && echo "$XX | $YY"
Di 22. Aug 10:24:47 CEST 2017 | Di 22. Aug 10:26:04 CEST 2017
Now I find that a test result like the following is worth for further
development considerations.
…
--- sound/usb/6fire/firmware.c
+++ /tmp/nothing/usb/6fire/firmware.c
@@ -246,14 +246,8 @@ static int usb6fire_fw_ezusb_upload(
while (usb6fire_fw_ihex_next_record(rec)) { /* write firmware */
ret = usb6fire_fw_ezusb_write(device, 0xa0, rec->address,
rec->data, rec->len);
- if (ret < 0) {
kfree(rec);
release_firmware(fw);
- dev_err(>dev,
- "unable to upload ezusb firmware %s: data
urb.\n",
- fwname);
- return ret;
- }
}
release_firmware(fw);
@@ -319,13 +313,8 @@ static int usb6fire_fw_fpga_upload(
buffer[i] = bitrev8((u8)*c);
ret = usb6fire_fw_fpga_write(device, buffer, i);
- if (ret < 0) {
release_firmware(fw);
kfree(buffer);
- dev_err(>dev,
- "unable to upload fpga firmware: fw urb.\n");
- return ret;
- }
}
release_firmware(fw);
kfree(buffer);
…
How would you like to treat variations in such log messages?
Regards,
Markus
___
Cocci mailing list
Cocci@systeme.lip6.fr
https://systeme.lip6.fr/mailman/listinfo/cocci
[Cocci] Checking evaluation of a regular expression for a SmPL constraint
Hello, Today I published the update suggestion “[PATCH 1/3] ecryptfs: Delete 21 error messages for a failed memory allocation”. https://patchwork.kernel.org/patch/9910539/ https://lkml.kernel.org/r/<1dcce3db-c162-c6bc-d1a8-bcf427248...@users.sourceforge.net> I noticed during the preparation of this change that the implementation of the function “ecryptfs_add_global_auth_tok” was not automatically found as an update candidate. A detailed regular expression is involved for a special constraint in an evolving SmPL script. I can pass the specified regular expression also to the program “pcre2test” (version 10.23-2.5) which indicates a successful match for an identifier like “kmem_cache_zalloc” then. Now I wonder about this software situation. Would anybody like to test once more if another mismatch can be observed around such regex evaluation within the Coccinelle software? Regards, Markus ___ Cocci mailing list Cocci@systeme.lip6.fr https://systeme.lip6.fr/mailman/listinfo/cocci
Re: [Cocci] Improving size determinations with SmPL
> The suggested script variant seems to work finally. I have tried it out once more with the following command. elfring@Sonne:~/Projekte/Linux/next-patched> spatch.opt --timeout 67 -j 4 --in-place ~/Projekte/Coccinelle/janitor/safer_size_determination2.cocci fs/jfs … EXN: Common.TimeoutNote: processing took74.0s: fs/jfs/jfs_txnmgr.c … EXN: Common.TimeoutNote: processing took73.5s: fs/jfs/jfs_xtree.c … EXN: Common.TimeoutNote: processing took75.2s: fs/jfs/jfs_dtree.c … How do you think about to take another look at the shown execution times? Are the run time characteristics worth for another clarification? By the way: I would find another line break nicer in these messages. Regards, Markus ___ Cocci mailing list Cocci@systeme.lip6.fr https://systeme.lip6.fr/mailman/listinfo/cocci
Re: [Cocci] Comparing statement lists with SmPL
>> * Are there further development challenges to consider for the safe >> identification >> of unique statements by metavariables? > > If you want to ensure that two metavariables match things in different > places, then put a position variable on each match and use apython rule > afterwards to discard the occurrences that are both in the same position. I hoped for a more convenient search specification to reduce duplicate source code a bit. Regards, Markus ___ Cocci mailing list Cocci@systeme.lip6.fr https://systeme.lip6.fr/mailman/listinfo/cocci
Re: [Cocci] Comparing statement lists with SmPL
>> elfring@Sonne:~/Projekte/Linux/next-patched> git checkout next-20170803 &&
>> spatch.opt ~/Projekte/Coccinelle/janitor/show_same_statements3.cocci
>> fs/ubifs/lpt.c
>> …
>> @@ -1974,10 +1974,6 @@ again:
>> int ret, lnum = lprops->lnum;
>>
>> ret = scan_cb(c, lprops, path[h].in_tree, data);
>> -if (ret < 0) {
>> -err = ret;
>> -goto out;
>> -}
>> if (ret & LPT_SCAN_ADD) {
>> /* Add all the nodes in path to the tree in memory */
>> for (h = 1; h < c->lpt_hght; h++) {
>>
>>
>> Now I wonder how this test result should fit to my source code search
>> pattern.
>
> I guess there is a loop around this code, so it is reachable from itself.
* How can we achieve progress with such a bit of information?
* Should such false positives be avoided anyhow?
* Will answers to other questions become more helpful again?
* Are there further development challenges to consider for the safe
identification
of unique statements by metavariables?
Regards,
Markus
___
Cocci mailing list
Cocci@systeme.lip6.fr
https://systeme.lip6.fr/mailman/listinfo/cocci
Re: [Cocci] Comparing statement lists with SmPL
>>> Can the search for duplicated source code be improved by the means of the
>>> semantic patch language?
>>
>> For two statements at least you could do:
>
> An other SmPL script variant can work to some degree.
How do you think about the relevance of the SmPL construct “<+... ...+>”
for this use case?
I tried my intermediate SmPL script variant a bit more.
Example:
elfring@Sonne:~/Projekte/Linux/next-patched> git checkout next-20170803 &&
spatch.opt ~/Projekte/Coccinelle/janitor/show_same_statements3.cocci
fs/ubifs/lpt.c
…
@@ -1974,10 +1974,6 @@ again:
int ret, lnum = lprops->lnum;
ret = scan_cb(c, lprops, path[h].in_tree, data);
- if (ret < 0) {
- err = ret;
- goto out;
- }
if (ret & LPT_SCAN_ADD) {
/* Add all the nodes in path to the tree in memory */
for (h = 1; h < c->lpt_hght; h++) {
Now I wonder how this test result should fit to my source code search pattern.
I find also more results questionable for this Linux software module.
It seems that there are only two functions which would be an acceptable match.
Regards,
Markus
___
Cocci mailing list
Cocci@systeme.lip6.fr
https://systeme.lip6.fr/mailman/listinfo/cocci
Re: [Cocci] Improving size determinations with SmPL
>> ... when any should be <...
>
> I have told you exactly what to do here.
You are right in the sense that I misinterpreted your change suggestion.
> The line above currently contains ... when any. It (and only it) should be
> completely removed and changed to <...
But I find it inappropriate to enclose the function call code by an optional
SmPL nest construct.
> Nowhere do I suggest to put <... ... when any ...>
The suggested script variant seems to work finally.
Thanks for your information.
@replacement@
identifier action, var, work;
type T, X;
@@
T work(...)
{
... when any
X* var;
<...
var = action(...,
sizeof(
-X
+*var
),
...
)
...>
}
Now I wonder why the SmPL construct “<+... ...+>” does not produce the same
transformation result for the source file “fs/pstore/ram.c”.
Regards,
Markus
___
Cocci mailing list
Cocci@systeme.lip6.fr
https://systeme.lip6.fr/mailman/listinfo/cocci
Re: [Cocci] Improving size determinations with SmPL
>>> ... when any should be ...>
>>
>> I am unsure if I interpret your suggestion correctly.
>
> No. Go read the original message again. It says exactly what to do and
> where it should be done.
Your hint might be nice.
> Eg, my message has two comments, but you have made three changes, which is
> not correct.
I have tried out to convert the SmPL ellipsis into the optional dot variant.
But I wonder still about the error message “minus: parse error:”
for an adjusted SmPL script variant.
@replacement@
identifier action, var, work;
type T, X;
@@
T work(...)
{
... when any
X* var;
<... ... when any ...>
var = action(...,
sizeof(
-X
+*var
),
...
)
<... ... when any ...>
}
Regards,
Markus
___
Cocci mailing list
Cocci@systeme.lip6.fr
https://systeme.lip6.fr/mailman/listinfo/cocci
Re: [Cocci] Improving size determinations with SmPL
> ... when any should be ...>
I am unsure if I interpret your suggestion correctly.
@replacement@
identifier action, var, work;
type T, X;
@@
T work(...)
{
<... ... when any ...>
X* var;
<... ... when any ...>
var = action(...,
sizeof(
-X
+*var
),
...
)
<... ... when any ...>
}
elfring@Sonne:~/Projekte/Linux/next-patched> git checkout next-20170803 &&
spatch.opt ~/Projekte/Coccinelle/janitor/safer_size_determination2.cocci
fs/pstore/ram.c
…
minus: parse error:
File
"/home/elfring/Projekte/Coccinelle/janitor/safer_size_determination2.cocci",
line 7, column 6, charpos = 80
around = '...',
whole content = <... ... when any ...>
> You request that the sizeof appear on every execution path.
I hope not. - I would like to express that this operator can appear for
a parameter of a function call.
Does the variable declaration “struct persistent_ram_zone *tmp_prz, *prz_next;”
in the function “ramoops_pstore_read” trigger challenges for the attempted
source code transformation?
Regards,
Markus
___
Cocci mailing list
Cocci@systeme.lip6.fr
https://systeme.lip6.fr/mailman/listinfo/cocci
[Cocci] Improving size determinations with SmPL
Hello,
I have constructed the following small script for another application of the
semantic patch language.
@replacement@
identifier action, var, work;
type T, X;
@@
T work(...)
{
... when any
X* var;
... when any
var = action(...,
sizeof(
-X
+*var
),
...
)
... when any
}
I know that this transformation approach is working to some degree.
I came along another source file where I do not get the desired code adjustment
with the software version “1.0.6-00186-g0acd38ee” automatically.
elfring@Sonne:~/Projekte/Linux/next-patched> git checkout next-20170803 &&
spatch.opt ~/Projekte/Coccinelle/janitor/safer_size_determination1.cocci
fs/pstore/ram.c
How should the software situation be changed further so that the function
“ramoops_pstore_read” will be also handled in the way like I published it in
the update suggestion “pstore: Improve a size determination in three functions”?
http://elixir.free-electrons.com/linux/v4.12.8/source/fs/pstore/ram.c#L291
https://patchwork.kernel.org/patch/9904523/
https://lkml.kernel.org/r/<23798ef4-5559-3454-ecf4-e6844b644...@users.sourceforge.net>
Regards,
Markus
___
Cocci mailing list
Cocci@systeme.lip6.fr
https://systeme.lip6.fr/mailman/listinfo/cocci
Re: [Cocci] Coccinelle: add atomic_as_refcounter script
Dear Elena, Why do you insist to use the variables “fname2” till “fname6” in this evolving SmPL script (instead of merging them into a single one with a special constraint)? How do you think about to omit the cover letter for the addition of such a script (when the change log can be integrated into the same message for your update suggestion)? Regards, Markus ___ Cocci mailing list Cocci@systeme.lip6.fr https://systeme.lip6.fr/mailman/listinfo/cocci
Re: [Cocci] Comparing statement lists with SmPL
>> Can the search for duplicated source code be improved by the means of the
>> semantic patch language?
>
> For two statements at least you could do:
An other SmPL script variant can work to some degree.
@duplicated_code@
identifier work;
statement s1, s2;
type T;
@@
T work(...)
{
... when any
*if ((...) < 0)
*{
* s1
* s2
...
*}
...
*if ((...) < 0)
*{
* s1
* s2
...
*}
...
}
But matched statements will not be mapped to the discussed metavariable type
in this use case.
I am curious on how such an approach can be used for corresponding source
code transformations.
Regards,
Markus
___
Cocci mailing list
Cocci@systeme.lip6.fr
https://systeme.lip6.fr/mailman/listinfo/cocci
Re: [Cocci] Comparing statement lists with SmPL
> Unfortunately, the software version “1.0.6-00186-g0acd38ee” does not like this
> SmPL specification.
I overlooked to add parentheses in my evolving SmPL script.
@duplicated_code@
identifier work;
statement s1, s2;
statement list sl;
type T;
@@
T work(...)
{
... when any
(
*if ((...) < 0) {
* s1
* s2
...
*}
&{
sl
}
)
...
(
*if ((...) < 0) {
* s1
* s2
...
*}
&{
sl
}
)
...
}
The following command does not show the expected source code analysis results
at the moment.
elfring@Sonne:~/Projekte/Coccinelle/janitor> spatch.opt
show_same_statements2.cocci ../Probe/hdsp-excerpt1.c
Is there any more clarification needed for the use of SmPL conjunctions?
Regards,
Markus
___
Cocci mailing list
Cocci@systeme.lip6.fr
https://systeme.lip6.fr/mailman/listinfo/cocci
Re: [Cocci] Comparing statement lists with SmPL
>> Can the search for duplicated source code be improved by the means of the
>> semantic patch language?
>
> For two statements at least you could do:
>
> (
> {
> s1
> s2
> ...
> }
> &
> {
> sl
> }
> )
I constructed another SmPL script based on this suggestion as follows.
@duplicated_code@
identifier work;
statement s1, s2;
statement list sl;
type T;
@@
T work(...)
{
... when any
*if ((...) < 0) {
* s1
* s2
...
*}
&{
sl
}
...
*if ((...) < 0) {
* s1
* s2
...
*}
&{
sl
}
...
}
Unfortunately, the software version “1.0.6-00186-g0acd38ee” does not like this
SmPL specification.
elfring@Sonne:~/Projekte/Coccinelle/janitor> spatch.opt --parse-cocci
show_same_statements2.cocci
…
minus: parse error:
File "show_same_statements2.cocci", line 15, column 0, charpos = 156
around = '&',
whole content = &{
How can my understanding become better also for the use of SmPL conjunctions?
Regards,
Markus
___
Cocci mailing list
Cocci@systeme.lip6.fr
https://systeme.lip6.fr/mailman/listinfo/cocci
Re: [Cocci] Comparing statement lists with SmPL
>> Now I am looking for a way to express the constraint that the statement list
>> metavariable should match only source code with two statements at least.
>> Can the search for duplicated source code be improved by the means of the
>> semantic patch language?
>
> For two statements at least you could do:
>
> (
> {
> s1
> s2
> ...
> }
> &
> {
> sl
> }
> )
Thanks for another example.
I admit that I am unused to such a variable combination with a SmPL conjunction.
Will this programming interface evolve any further?
Regards,
Markus
___
Cocci mailing list
Cocci@systeme.lip6.fr
https://systeme.lip6.fr/mailman/listinfo/cocci
Re: [Cocci] Comparing statement lists with SmPL
> The source code analysis results look promising by this command.
I have just noticed that an other source file points details out for further
development considerations around the mentioned functionality from the
Coccinelle software.
elfring@Sonne:~/Projekte/Linux/next-patched> git checkout next-20170803 &&
spatch.opt ~/Projekte/Coccinelle/janitor/show_same_statements1.cocci
sound/pci/rme9652/hdspm.c
…
(ONCE) already tagged but only removed, so safe
diff =
…
@@ -6497,9 +6497,6 @@ static int snd_hdspm_create_alsa_devices
i = 0;
while (i < hdspm->midiPorts) {
err = snd_hdspm_create_midi(card, hdspm, i);
- if (err < 0) {
- return err;
- }
i++;
}
@@ -6955,10 +6952,6 @@ static int snd_hdspm_probe(struct pci_de
hdspm->pci = pci;
err = snd_hdspm_create(card, hdspm);
- if (err < 0) {
- snd_card_free(card);
- return err;
- }
if (hdspm->io_type != MADIface) {
snprintf(card->shortname, sizeof(card->shortname), "%s_%x",
…
Now I am looking for a way to express the constraint that the statement list
metavariable should match only source code with two statements at least.
Can the search for duplicated source code be improved by the means of the
semantic patch language?
Regards,
Markus
___
Cocci mailing list
Cocci@systeme.lip6.fr
https://systeme.lip6.fr/mailman/listinfo/cocci
Re: [Cocci] Comparing statement lists with SmPL
>> * How can be determined how many code is equivalent between two statement >> list variables? > > I guess you can make special cases, for one statement, two statements, etc. I guess that we would like to avoid such an approach for some use cases. I hope that there are safer programming possibilities available. Regards, Markus ___ Cocci mailing list Cocci@systeme.lip6.fr https://systeme.lip6.fr/mailman/listinfo/cocci
Re: [Cocci] Comparing statement lists with SmPL
>> Is this programming interface usable to determine if any code could be
>> combined
>> to some degree?
>
> I don't really understand the questions.
I try again to describe the use case I became interested in a bit more.
> Statement list metavariables can be used once they are bound like any other
> metavariables.
Can it be that they will need any special computing power occasionally?
I have tried the following small SmPL script out.
@duplicated_code@
identifier work;
statement list sl;
type T;
@@
T work(...)
{
... when any
*if ((...) < 0) {
* sl
*}
...
*if ((...) < 0) {
* sl
*}
...
}
The source code analysis results look promising by this command.
elfring@Sonne:~/Projekte/Linux/next-patched> git checkout next-20170803 &&
spatch.opt ~/Projekte/Coccinelle/janitor/show_same_statements1.cocci
sound/pci/rme9652/hdsp.c
…
@@ -793,11 +793,6 @@ static int hdsp_get_iobox_version (struc
…
@@ -5382,19 +5372,11 @@ static int snd_hdsp_probe(struct pci_dev
…
How should such update candidates be transformed further?
> However they can only be bound to the complete sequence of statements in a
> block. So
>
> @@
> statement list sl;
> @@
> if (x) {
> sl
> }
>
> is ok, but
>
> @@
> statement list sl;
> @@
> if (x) {
> one();
> sl
> two();
> }
>
> is not.
Thanks for your example.
* I do not like the mentioned software restriction at the moment.
* How can be determined how many code is equivalent between two statement list
variables?
Regards,
Markus
___
Cocci mailing list
Cocci@systeme.lip6.fr
https://systeme.lip6.fr/mailman/listinfo/cocci
[Cocci] Comparing statement lists with SmPL
Hello, The Coccinelle software supports metavariables with the type “statement list”. Can statement lists be compared if they refer to the same source code structures? Is this programming interface usable to determine if any code could be combined to some degree? Regards, Markus ___ Cocci mailing list Cocci@systeme.lip6.fr https://systeme.lip6.fr/mailman/listinfo/cocci
Re: [Cocci] Coccinelle: add atomic_as_refcounter script
> Elena, please don't follow Markus's suggestion. I suggest to consider additional software development possibilities. We have got recurring different opinions around the application of advanced regular expressions. > Coccinelle doesn't interpret regular expressions when selecting code to work > on, This software behaviour can be fine. > and thus the resulting rule will be less efficient. I do not come to the same conclusion for some use cases. > Markus, if you see this somehow, Yes. - But I can read your response only by the web interface for your mailing list archive at the moment. https://systeme.lip6.fr/pipermail/cocci/2017-August/004308.html > at least yesterday all mail sent to you was bouncing. Thanks for this information. - I was not informed about such message exchange problems so far. I hope that communication difficulties will be fixed somehow. Regards, Markus ___ Cocci mailing list Cocci@systeme.lip6.fr https://systeme.lip6.fr/mailman/listinfo/cocci
Re: [Cocci] Coccinelle: add atomic_as_refcounter script
Dear Elena, Would you like to take another look at my development suggestion for your approach? https://systeme.lip6.fr/pipermail/cocci/2017-August/004300.html How do you think about to refactor another rule like the following? @r3 exists@ expression E; identifier I=~"^atomic(?:64|_long)?_add_return$"; position P; @@ E = I@P(-1, ...); Can the specification of SmPL constraints be occasionally be more succinct with the use of regular expressions? Regards, Markus ___ Cocci mailing list Cocci@systeme.lip6.fr https://systeme.lip6.fr/mailman/listinfo/cocci
Re: [Cocci] add atomic_as_refcounter script
Dear Elena, I have also taken another look at your script. The SmPL rule “r1” specifies six metavariables for identifiers. It seems that the names “fname3” till “fname6” do not care for different function parameters in the SmPL disjunction at the end. How do you think about to reduce this disjunction by using a regular expression with an alternation like “.*(?:call_rcu|del|queue_work|schedule_work).*” for a constraint? Will it help to refactor another rule a bit? @r3 exists@ identifier x; position p1; @@ x = ( atomic_add_return at p1(-1, ...); | atomic_long_add_return at p1(-1, ...); | atomic64_add_return at p1(-1, ...); ) Regards, Markus ___ Cocci mailing list Cocci@systeme.lip6.fr https://systeme.lip6.fr/mailman/listinfo/cocci
Re: [Cocci] Replacing suffix rules in make scripts?
> So I get the impression that an other build approach will be safer there. > How do you think about to add a dependency specification so that interface > descriptions will be always compiled before the corresponding OCaml code? I showed change possibilities which can improve the software situation to some degree. * Improve several dependency specifications in make rules https://github.com/coccinelle/coccinelle/pull/108 * Fix specifications for build dependencies between shared components https://github.com/coccinelle/coccinelle/issues/110 * Support references for the top-level source file directory in build scripts https://github.com/coccinelle/coccinelle/issues/104 Which ideas would you like to integrate from the available development approaches? Regards, Markus ___ Cocci mailing list Cocci@systeme.lip6.fr https://systeme.lip6.fr/mailman/listinfo/cocci
Re: [Cocci] Replacing suffix rules in make scripts?
> Why do I get such an error message for a parallel software build attempt > when the specified dependencies worked as expected for the non-parallel > build try before? How do you think about the corresponding update suggestion “Improve some dependency specifications in make rules”? https://github.com/elfring/Coccinelle-20160205/commit/9597598779f3604272621f6d42ead79eaafad2a8 Regards, Markus ___ Cocci mailing list Cocci@systeme.lip6.fr https://systeme.lip6.fr/mailman/listinfo/cocci
Re: [Cocci] Replacing suffix rules in make scripts?
> The OCaml makefile specifies an order in which the files should be > compiled in the project. If you are sidestepping that ordering, then you > are not doing things in the intended manner. I hope that some details were clarified to some degree in the meantime. > There is only an intent to support running make or make opt in the > subdirectories or at the root. Perhaps Thierry will put a new make file > that has some other properties (I don't know which ones) in the future. > But there is no intent to support anything else at the moment. I suggest to take also another look around the discussed build scripts. Example: https://github.com/coccinelle/coccinelle/blob/cd539cd12ad0674d5a8f4c33f6c21ae00aec8e57/globals/Makefile#L50 Now I wonder about the following test result for my current “developer rules”. elfring@Sonne:~/Projekte/Coccinelle/20160205/globals> make clean && LANG=C make --no-builtin-rules V=1 && make clean && LANG=C make --no-builtin-rules -j4 V=1 rm -f … /usr/bin/ocamldep -I ../commons -I /home/elfring/Projekte/Coccinelle/20160205/bundles/pcre/ *.mli *.ml > .depend … /usr/bin/ocamlc.opt -unsafe -I ../commons -I /home/elfring/Projekte/Coccinelle/20160205/bundles/pcre/ -c flag.ml … /usr/bin/ocamlc.opt -unsafe -I ../commons -I /home/elfring/Projekte/Coccinelle/20160205/bundles/pcre/ -a -o globals.cma config.cmo flag.cmo regexp_pcre.cmo regexp.cmo iteration.cmo rm -f … /usr/bin/ocamldep -I ../commons -I /home/elfring/Projekte/Coccinelle/20160205/bundles/pcre/ *.mli *.ml > .depend … /usr/bin/ocamlc.opt -unsafe -I ../commons -I /home/elfring/Projekte/Coccinelle/20160205/bundles/pcre/ -c flag.ml … /usr/bin/ocamlc.opt -unsafe -I ../commons -I /home/elfring/Projekte/Coccinelle/20160205/bundles/pcre/ -c iteration.mli make: *** No rule to make target 'flag.cmi', needed by 'iteration.cmo'. Stop. … Why do I get such an error message for a parallel software build attempt when the specified dependencies worked as expected for the non-parallel build try before? Do you get any more ideas on how to explain such a test surprise? Regards, Markus ___ Cocci mailing list Cocci@systeme.lip6.fr https://systeme.lip6.fr/mailman/listinfo/cocci
Re: [Cocci] Replacing suffix rules in make scripts?
> OK, I don't undertand what command you are doing to get the problem, Did we achieve a better common understanding in the meantime? > so I can't reproduce it, so I can't determine whether it is a real issue. * Are you really keen to achieve this also in your own test environments? * How much does my build difficulty fit into your development attention and software change capacity? > The OCaml makefile specifies an order in which the files should be > compiled in the project. If you are sidestepping that ordering, then you > are not doing things in the intended manner. I am just curious to fix the dependency checking (without unexpected side effects) for build prerequisites somehow in the involved software components. Regards, Markus ___ Cocci mailing list Cocci@systeme.lip6.fr https://systeme.lip6.fr/mailman/listinfo/cocci
Re: [Cocci] Replacing suffix rules in make scripts?
> The command “ocamldep *.mli *.ml > .depend” was performed for > relevant subdirectories. > Unfortunately, this approach seems to be insufficient for the desired > safe integration of dependency information so far. I have got another idea. The adjustment results were more promising for my build script changes in other subdirectories. Small make functions construct extra rules there. If the used make rules from the ocamldep call do not produce the desired effects for checks of prerequisites so far, I could try to convert the provided data to an other style of dependency specification. Such generated make rules would finally contain one useful command in each recipe. Regards, Markus ___ Cocci mailing list Cocci@systeme.lip6.fr https://systeme.lip6.fr/mailman/listinfo/cocci
Re: [Cocci] Replacing suffix rules in make scripts?
>> Can you get the impression from the mentioned error message examples >> that the desired software build prerequisites failed anyhow? > > Something fails in your situation, but without knowing the arguments to > the compiler, I don't know what. You could know also from the reduced information. Is it really easier for you to see the following data again? elfring@Sonne:~/Projekte/Coccinelle/20160205/parsing_c> LANG=C make V=1 /usr/bin/ocamlc.opt -unsafe -I ../commons -I ../commons/ocamlextra -I ../globals -I ../parsing_cocci -c includes.mli File "includes.mli", line 38, characters 46-60: Error: Unbound module Ast_c make: *** [/home/elfring/Projekte/Coccinelle/20160205/common_rules.make:6: includes.cmi] Error 2 Regards, Markus ___ Cocci mailing list Cocci@systeme.lip6.fr https://systeme.lip6.fr/mailman/listinfo/cocci
Re: [Cocci] Replacing suffix rules in make scripts?
> The command “ocamldep *.mli *.ml > .depend” was performed for > relevant subdirectories. > Unfortunately, this approach seems to be insufficient for the desired > safe integration of dependency information so far. How do you think about to work any more with dedicated files for each source file by pattern rules like it is described in the manual section “Generating prerequisites automatically”? https://www.gnu.org/software/make/manual/html_node/Automatic-Prerequisites.html Would it make sense to split the data from the command “ocamldep” into smaller units? Do I stumble on unexpected software behaviour by the tool “GNU Make 4.2.1-1.7” on my openSUSE Tumbleweed system? Regards, Markus ___ Cocci mailing list Cocci@systeme.lip6.fr https://systeme.lip6.fr/mailman/listinfo/cocci
Re: [Cocci] Replacing suffix rules in make scripts?
>> I showed how I could stumble on open issues. > > No you didn't. I try then once more to help you to interpret the mentioned error message examples. The command “ocamldep *.mli *.ml > .depend” was performed for relevant subdirectories. Unfortunately, this approach seems to be insufficient for the desired safe integration of dependency information so far. > I don't know what are the arguments to the ocaml compiler. If you think in terms of the build system, you would know that this tool needs input files. Unfortunately, the current build scripts did not trigger the generation of the desired prerequisites for the command examples at the end. > What file are you trying to compile? The attempted data processing steps can not succeed when required files were not built before. If these dependencies will be fixed, then we can worry if an error message like “Unbound module Ast_c” can occur because of other reasons. Regards, Markus ___ Cocci mailing list Cocci@systeme.lip6.fr https://systeme.lip6.fr/mailman/listinfo/cocci
Re: [Cocci] Replacing suffix rules in make scripts?
>> I am curious on how a dependency for a software interface like >> “Visitor_ast0_types” >> should be expressed for so many source files in a consistent way. > > What problem are you trying to solve? I would like to achieve something more for the topics which were also registered in the issue tracker. > Why do the dependency specifications need improving? Examples: elfring@Sonne:~/Projekte/Coccinelle/20160205/parsing_cocci> LANG=C make V=1 /usr/bin/ocamldep *.mli *.ml > .depend /usr/bin/ocamlc.opt … File "_none_", line 1: Error: Cannot find file flag_parsing_cocci.cmo make: *** [Makefile:67: cocci_parser.cma] Error 2 elfring@Sonne:~/Projekte/Coccinelle/20160205/parsing_c> LANG=C make V=1 /usr/bin/ocamlc.opt … File "includes.mli", line 38, characters 46-60: Error: Unbound module Ast_c make: *** [/home/elfring/Projekte/Coccinelle/20160205/common_rules.make:6: includes.cmi] Error 2 These messages indicate reasons why I am also fiddling with the build scripts for this software again. Regards, Markus ___ Cocci mailing list Cocci@systeme.lip6.fr https://systeme.lip6.fr/mailman/listinfo/cocci
Re: [Cocci] Replacing suffix rules in make scripts?
> If you have changed things, then perhaps you have caused the problem? I have found an approach for the adjustment of affected build scripts where I can improve the involved dependency specifications in a few subdirectories on my own to some degree. But I guess that a directory like “parsing_cocci” provides bigger development challenges to make the dependencies safer there. I am curious on how a dependency for a software interface like “Visitor_ast0_types” should be expressed for so many source files in a consistent way. Regards, Markus ___ Cocci mailing list Cocci@systeme.lip6.fr https://systeme.lip6.fr/mailman/listinfo/cocci
Re: [Cocci] Replacing suffix rules in make scripts?
> If you have changed things, then perhaps you have caused the problem? This is also another possibility in principle. It seems that I wonder again why some source files should be compiled without interface description files there. An other script variant showed the following result. elfring@Sonne:~/Projekte/Coccinelle/20160205/globals> LANG=C make iteration.cmx V=1 My test command: /usr/bin/ocamlopt.opt -unsafe -I ../commons -I /home/elfring/Projekte/Coccinelle/20160205/bundles/pcre/ -c flag.ml File "flag.ml", line 24, characters 17-36: Error: Unbound module Commands make: *** [Makefile:62: flag.cmx] Error 2 Can it be that the corresponding dependency specifications were incomplete anyhow in this make script for the direct use at such a place? Would you like to help any more to get these dependencies more consistent? Regards, Markus ___ Cocci mailing list Cocci@systeme.lip6.fr https://systeme.lip6.fr/mailman/listinfo/cocci
Re: [Cocci] Replacing suffix rules in make scripts?
> So I get the impression that an other build approach will be safer there. > How do you think about to add a dependency specification so that interface > descriptions will be always compiled before the corresponding OCaml code? Hello, I have achieved another bit of progress with my attempt to replace suffix rules by pattern rules in some make scripts. I stumbled on the next software development challenge as follows. elfring@Sonne:~/Projekte/Coccinelle/20160205> LANG=C make V=1 … make[6]: Entering directory '/home/elfring/Projekte/Coccinelle/20160205/globals' /usr/bin/ocamlopt.opt -unsafe -I ../commons -I /home/elfring/Projekte/Coccinelle/20160205/bundles/pcre/ -c iteration.ml File "iteration.ml", line 36, characters 4-28: Error: Unbound module Flag make[6]: *** [/home/elfring/Projekte/Coccinelle/20160205/common_rules.make:12: iteration.cmx] Error 2 … * Do I overlook an “obvious” software dependency once more in this use case? * How should the relevant details be resolved here? Regards, Markus ___ Cocci mailing list Cocci@systeme.lip6.fr https://systeme.lip6.fr/mailman/listinfo/cocci
[Cocci] Replacing suffix rules in make scripts?
Hello, I have achieved a bit of progress with one experiment for build script adjustments. I would like to try further changes out. Unfortunately, I stumbled on an error message like the following from the program “GNU Make 4.2.1-1.7” then. … /usr/bin/ocamlopt.opt … File "cocci.ml", line 1: Error: Could not find the .cmi file for interface cocci.mli. make[3]: *** [Makefile:533: cocci.cmx] Error 2 … It seems that the following code is relevant in this use case. … .ml.cmo: $(OCAMLC_CMD) -c $< .mli.cmi: $(OCAMLC_CMD) -c $< .ml.cmx: $(OCAMLOPT_CMD) -c $< … Other documentation provides this information: “… Suffix rules cannot have any prerequisites of their own. …” So I get the impression that an other build approach will be safer there. How do you think about to add a dependency specification so that interface descriptions will be always compiled before the corresponding OCaml code? Regards, Markus ___ Cocci mailing list Cocci@systeme.lip6.fr https://systeme.lip6.fr/mailman/listinfo/cocci
Re: [Cocci] Easier search for assignment targets with SmPL
>> Can it become safer and more convenient to handle assignments here? > > I have no idea what you are asking for. I wonder about such a kind of feedback. > In any case, an expression metavariable already matches a variable. This information can be appropriate in some source code situations. Do you care for any more “special use cases”? > I don't know what a designated initializer is. I find this aspect strange when it was already discussed several times. http://en.cppreference.com/w/c/language/struct_initialization How would you prefer to handle various software development challenges around specifications for variable definitions? Regards, Markus ___ Cocci mailing list Cocci@systeme.lip6.fr https://systeme.lip6.fr/mailman/listinfo/cocci
Re: [Cocci] Easier search for assignment targets with SmPL
> var = action(...); This specification looks also simple for the semantic patch language at first glance. I tried to express that a specific value is assigned to something. But this “something” could be a different item depending on the position in the source code. 1. Variable 2. Expression 3. Element for a designated initialiser I would appreciate if I do not need to repeat specifications for these cases in SmPL code. How do you think about the possibility to support the desired source code search by an additional metavariable type there? Can it become safer and more convenient to handle assignments here? Regards, Markus ___ Cocci mailing list Cocci@systeme.lip6.fr https://systeme.lip6.fr/mailman/listinfo/cocci
Re: [Cocci] Checking failure predicates with SmPL
> var = action(...); > if (failed(var)) This SmPL specification contains a few further software development challenges. 1. It could be taken into account that a function call and a corresponding check for the return value are not directly next to each other. A SmPL ellipsis could be added there. But I guess that special restrictions will be needed then for a safe “when specification”. 2. The check which should appropriately be performed for a return code depends on the called function. * Each failure predicate could be written as a separate SmPL rule. This approach would result into code duplication to some degree. * I would prefer to keep the selection and data processing for the desired predicate at a single place. But I am unsure so far how good such a variant is supported by the current Coccinelle software. Regards, Markus ___ Cocci mailing list Cocci@systeme.lip6.fr https://systeme.lip6.fr/mailman/listinfo/cocci
Re: [Cocci] Safe determination of function properties for SmPL scripts
> How can the Coccinelle software help any more in such an use case? This small SmPL script example demonstrates two special selections of functions by their name. They belong to separate categories then. * The primary actions should usually provide a return value somehow. * A call of a function which will not return from its final task is one possibility for an error reaction. Have you got any preferences for the determination and integration of name lists for functions with the desired properties into the semantic patch language in a safe way? Would you dare to query a database table (or view) for such a purpose? Regards, Markus ___ Cocci mailing list Cocci@systeme.lip6.fr https://systeme.lip6.fr/mailman/listinfo/cocci
Re: [Cocci] Checking source code (including optional parts) with SmPL
I would like to clarify the possibilities for corresponding software extensions a bit more. The Coccinelle software provides also the search operator “question mark” which denotes code on a specific line as optional. https://github.com/coccinelle/coccinelle/blob/9b5e00862900b0daadf1819a6894d31544a3ba34/docs/manual/cocci_syntax.tex#L1067 The scope of the special code there is only the marked line so that I see some software development challenges to pack desired search filters into one line. * How often do you check the handling for optional parts? - Do you find it relevant to check for the assignment of an error code to an other variable? - Are special constraints needed for additional function calls? * How are the chances to improve readability and maintainability for SmPL code? Regards, Markus ___ Cocci mailing list Cocci@systeme.lip6.fr https://systeme.lip6.fr/mailman/listinfo/cocci
[Cocci] Checking exception handling implementations with SmPL
Hello,
I would like to check the implementation of exception handling in various
source files. The following script for the semantic patch language tries to
express a few possibilities for such a source code search pattern.
@show_special_reactions@
expression error_code, var;
identifier action =~ "^(?:malloc|strdup)$",
no_return =~ "[Aa]bort",
label;
statement es;
@@
var = action(...);
if (failed(var))
{
(
* return error_code;
|
* goto label;
|
* no_return();
)
}
else
es
I find this approach too simple and therefore incomplete. Three well-known
reactions are shown. But there can be more source code before these statements
like further function calls and variable assignments.
We do eventually not know the extra code there and it can also be
generally optional.
How can the Coccinelle software help any more in such an use case?
Regards,
Markus
___
Cocci mailing list
Cocci@systeme.lip6.fr
https://systeme.lip6.fr/mailman/listinfo/cocci
Re: [Cocci] Easy search for elements besides binary operators with SmPL
>> I find it just strange that extra parentheses are needed in such an use case >> at the moment. > > ... is used in a number of contexts in SmPL, and sometimes its use has to > be restricted to avoid parsing ambiguities. I tried to use extra parentheses for other source code search patterns. I left them “accidentally” out with undesired consequences this time. > I'm a bit surprised also that it is a problem in this case. I find such a feedback interesting then if it will influence corresponding software evolution. Regards, Markus ___ Cocci mailing list Cocci@systeme.lip6.fr https://systeme.lip6.fr/mailman/listinfo/cocci
Re: [Cocci] Easy search for elements besides binary operators with SmPL
>>> Maybe put parentheses around the second argument. An isomorphism will >>> drop them and you will avoid whatever parsing issue is being encountered. >> >> I do not observe a desired improvement after such a SmPL code adjustment. > > Your change is not what I suggested either. The second argument starts to > the right of one comma and ends to the left of the next one. I interpreted the “second one” as belonging to the ellipsis in the binary operator specification (for a moment). Now I observe that the following SmPL script variants work as desired in principle. @find_last_option@ expression target; identifier action; @@ target = action(..., ( ... *| __GFP_NOFAIL ) ); @find_flag@ expression target; identifier action; @@ target = action(..., ( ... *| __GFP_NOFAIL | ... ), ...); I find it just strange that extra parentheses are needed in such an use case at the moment. Regards, Markus ___ Cocci mailing list Cocci@systeme.lip6.fr https://systeme.lip6.fr/mailman/listinfo/cocci
[Cocci] Easy search for elements besides binary operators with SmPL
Hello, I would like to check also the application of elements which are used together with binary operators like the following. @find@ expression target; identifier action; @@ target = action(..., ... *| __GFP_NOFAIL | ..., ...); elfring@Sonne:~/Projekte/Linux/next-patched> spatch.opt drivers/md/dm-region-hash.c ~/Projekte/Coccinelle/janitor/show_use_of_nofail1.cocci … minus: parse error: File "/home/elfring/Projekte/Coccinelle/janitor/show_use_of_nofail1.cocci", line 6, column 17, charpos = 87 around = '...', whole content = ... How should I improve my search pattern? A metavariable can already be used for the search of a function parameter at an undetermined position within a bigger list. Can similar functionality work also for binary operators? Regards, Markus ___ Cocci mailing list Cocci@systeme.lip6.fr https://systeme.lip6.fr/mailman/listinfo/cocci
Re: [Cocci] [PATCH v3] Coccinelle: Script to replace allocate and memset with zalloc functions
> This script finds instances of allocate and memset which can be > replaced with a direct call to zalloc equivalent of a function. What is the software development status for this SmPL script in comparison to an other directory for source code transformations? https://git.kernel.org/cgit/linux/kernel/git/next/linux-next.git/tree/scripts/coccinelle/api/alloc Regards, Markus ___ Cocci mailing list Cocci@systeme.lip6.fr https://systeme.lip6.fr/mailman/listinfo/cocci
Re: [Cocci] Replacing combination of kcalloc() and copy_from_user() by memdup_user() with SmPL?
> The comm_assoc isomorphism allows A || ... to match any disjunction where > A appears at the top level, not just in the far left position. Thanks for your explanation. Are there further software development challenges to consider around the handling of items like position variables and “mixed modifs”? > It also matches the case where A is not present at all. Will the bug report “Safer source code analysis by "memdup_user.cocci"” get another look? https://github.com/coccinelle/coccinelle/issues/78 Regards, Markus ___ Cocci mailing list Cocci@systeme.lip6.fr https://systeme.lip6.fr/mailman/listinfo/cocci
Re: [Cocci] Replacing combination of kcalloc() and copy_from_user() by memdup_user() with SmPL?
> elfring@Sonne:~/Projekte/Coccinelle/janitor> spatch.opt
> replace_kcalloc_by_memdup_user-draft1.cocci ../Probe/user_sdma-excerpt1.c
> …
> refactoring: position variables or mixed modifs interfere with comm_assoc iso
> bool (bool !to
> >>> IS_ERR(to)
> || ...)
> …
Would you like to explain this message a bit more while the following SmPL
approach seems to work
as expected?
@refactoring@
expression code, count, from, to, size;
identifier label, log;
@@
to =
- kcalloc(count, size, GFP_KERNEL)
+ memdup_user(from, size * count)
;
if (
-!to
+IS_ERR(to)
) {
- code = -ENOMEM;
- goto label;
-}
-code = copy_from_user(to, from, (size) * (count));
-if (code) {
...
+ code = PTR_ERR(to);
log(..., code, ...);
- code = -EFAULT;
goto label;
}
elfring@Sonne:~/Projekte/Coccinelle/janitor> spatch.opt
replace_kcalloc_by_memdup_user-draft2.cocci ../Probe/user_sdma-excerpt1.c
init_defs_builtins: /usr/local/lib64/coccinelle/standard.h
HANDLING: ../Probe/user_sdma-excerpt1.c
diff =
--- ../Probe/user_sdma-excerpt1.c
+++ /tmp/cocci-output-8749-f5da06-user_sdma-excerpt1.c
@@ -11,20 +11,12 @@ int hfi1_user_sdma_process_request(struc
ret = -EINVAL;
goto free_req;
}
- req->tids = kcalloc(ntids, sizeof(*req->tids), GFP_KERNEL);
- if (!req->tids) {
- ret = -ENOMEM;
- goto free_req;
- }
- /*
-* ...
-*/
- ret = copy_from_user(req->tids, iovec[idx].iov_base,
-ntids * sizeof(*req->tids));
- if (ret) {
+ req->tids = memdup_user(iovec[idx].iov_base,
+ sizeof(*req->tids) * ntids);
+ if (IS_ERR(req->tids)) {
+ ret = PTR_ERR(req->tids);
SDMA_DBG(req, "Failed to copy %d TIDs (%d)",
ntids, ret);
- ret = -EFAULT;
goto free_req;
}
req->n_tids = ntids;
Regards,
Markus
___
Cocci mailing list
Cocci@systeme.lip6.fr
https://systeme.lip6.fr/mailman/listinfo/cocci
[Cocci] Replacing combination of kcalloc() and copy_from_user() by memdup_user() with SmPL?
Hello,
I came along another function implementation where I got the impression that
the function “memdup_user” could also be used there in a way which is similar
to an existing source code transformation approach by a small script for
the semantic patch language.
SmPL script:
https://git.kernel.org/cgit/linux/kernel/git/next/linux-next.git/tree/scripts/coccinelle/api/memdup_user.cocci?id=632571b1bee00494aef749512d9f3290dfba0ead#n1
@refactoring@
expression code, count, from, to, size;
identifier label, log;
@@
to =
- kcalloc(count, size, GFP_KERNEL)
+ memdup_user(from, size * count)
;
if (
-!to
+IS_ERR(to)
|| ...) {
- code = -ENOMEM;
- goto label;
-}
-code = copy_from_user(to, from, (size) * (count));
-if (code) {
...
log(...,
- code
+ PTR_ERR(to)
, ...);
code = -ENOMEM;
goto label;
}
Source file:
https://git.kernel.org/cgit/linux/kernel/git/next/linux-next.git/tree/drivers/infiniband/hw/hfi1/user_sdma.c?id=632571b1bee00494aef749512d9f3290dfba0ead#n718
int hfi1_user_sdma_process_request(struct file *fp, struct iovec *iovec,
unsigned long dim, unsigned long *count)
{
/*
* ...
*/
if (req_opcode(req->info.ctrl) == EXPECTED) {
u16 ntids = iovec[idx].iov_len / sizeof(*req->tids);
if (!ntids || ntids > MAX_TID_PAIR_ENTRIES) {
ret = -EINVAL;
goto free_req;
}
req->tids = kcalloc(ntids, sizeof(*req->tids), GFP_KERNEL);
if (!req->tids) {
ret = -ENOMEM;
goto free_req;
}
/*
* ...
*/
ret = copy_from_user(req->tids, iovec[idx].iov_base,
ntids * sizeof(*req->tids));
if (ret) {
SDMA_DBG(req, "Failed to copy %d TIDs (%d)",
ntids, ret);
ret = -EFAULT;
goto free_req;
}
req->n_tids = ntids;
idx++;
}
/*
* ...
*/
return 0;
free_req:
user_sdma_free_request(req, true);
if (req_queued)
pq_update(pq);
set_comp_state(pq, cq, info.comp_idx, ERROR, req->status);
return ret;
}
elfring@Sonne:~/Projekte/Coccinelle/janitor> spatch.opt
replace_kcalloc_by_memdup_user-draft1.cocci ../Probe/user_sdma-excerpt1.c
…
refactoring: position variables or mixed modifs interfere with comm_assoc iso
bool (bool !to
>>> IS_ERR(to)
|| ...)
…
The software combination “Coccinelle 1.0.6-00093-ge1776d7b (OCaml 4.03.0)”
does not like this change attempt so far.
Which details should I reconsider next?
Regards,
Markus
___
Cocci mailing list
Cocci@systeme.lip6.fr
https://systeme.lip6.fr/mailman/listinfo/cocci
[Cocci] Moving setting of error codes into if branches with SmPL?
Hello, I have tried the following small script out for the semantic patch language. @show_assignments@ expression target; identifier code =~ "^E[A-Z]+$"; statement is, es; @@ *target = -(code); if (...) is else es elfring@Sonne:~/Projekte/Linux/next-patched> spatch.opt -timeout 23 -j 4 --chunksize 1 -sp-file ~/Projekte/Coccinelle/janitor/show_misplaced_error_code_assignments1.cocci -dir security/selinux 2> ~/Projekte/Bau/Linux/scripts/Coccinelle/show_misplaced_error_code_assignments1-errors.txt Some update candidates were displayed. Is such a simple source code search pattern worth to be integrated into the Coccinelle script collection? I see further development challenges for the specification of a corresponding source code transformation. How hard can it become to move the found variable assignments before condition checks to the place before the last goto statement within an if branch automatically? Regards, Markus ___ Cocci mailing list Cocci@systeme.lip6.fr https://systeme.lip6.fr/mailman/listinfo/cocci
Re: [Cocci] … adding missing NULL checks
> Is there a clever way of conditionally making this change? The answer depends on your desire how “clever” (or sophisticated) an other approach should become. You could try to work with more disjunctions and rules in the semantic patch language. This transformation might trigger a few target conflicts and further software development challenges as you might become also aware of special needs and corresponding advancements in technology. How many software developers and source code reviewers care really about unused (or ignored) return values from various function calls? Regards, Markus ___ Cocci mailing list Cocci@systeme.lip6.fr https://systeme.lip6.fr/mailman/listinfo/cocci
Re: [Cocci] … adding missing NULL checks
>> need to put another copy of that at the end of the rule with ? in front >> of it. So that it stops when it comes to that again. > > And now it's working just fine. I became also curious on how the discussed source code transformation can really work. I imagine that there are still some software development challenges left over for further considerations. * A filter criterium like “T = strdup(...);” was specified. How do you think about to use a SmPL disjunction (or an advanced SmPL constraint on a metavariable with the type “identifier” instead)? * Can the function call “pkg_emit_errno” become a bit more generic by passing a string from previously matched data? * Would you like to check any more cases so that the technology “data flow analysis” will be also needed here? Regards, Markus ___ Cocci mailing list Cocci@systeme.lip6.fr https://systeme.lip6.fr/mailman/listinfo/cocci
Re: [Cocci] Improved filtering for a big function collection with SmPL?
> Why were you looking for such functions in the first place? These functions provide various return values. I am curious on how often the returned values will actually be checked and used (or not). > If you want non-static non-void functions and you find them, > then what is the issue? NetBSD contains a lot of components from various software. I tried to get an impression from the size of the corresponding function collection. The mentioned concrete size will show further software development challenges if meaningful source code transformations should be performed with such data, won't it? Regards, Markus ___ Cocci mailing list Cocci@systeme.lip6.fr https://systeme.lip6.fr/mailman/listinfo/cocci
[Cocci] Improved filtering for a big function collection with SmPL?
Hello, I tried the Coccinelle software out for another source code analysis approach on source files from NetBSD 7.0.2. A small SmPL script worked mostly so good that 136978 non-static function implementations were finally found with a non-void return type there. * Can such an analysis result trigger further software development considerations? * I imagine that this function collection can only be used for some benefit if it will be grouped and filtered to specific needs. * Would you like to suppress any functions from this data extraction? Regards, Markus ___ Cocci mailing list Cocci@systeme.lip6.fr https://systeme.lip6.fr/mailman/listinfo/cocci
Re: [Cocci] Checking incidence for SmPL initialisation rules after parallel script execution
>> Can you generally clarify current consequences from usage of a SmPL rule >> “initialize:…” >> together with the parameter “--jobs” for the program “spatch”? > > It should happen only once. Thanks for this information. I would like to know if the script initialisation should happen for each background process and/or after the start of the program “spatch”. > Thus without your semantic patch and your .c file, I can't tell > why it doesn't behave as expected. You got the corresponding data from my source code analysis try for NetBSD. How often do you observe a simple text line output from initialisation code after parallel execution of one from your test scripts? Regards, Markus ___ Cocci mailing list Cocci@systeme.lip6.fr https://systeme.lip6.fr/mailman/listinfo/cocci
Re: [Cocci] Checking incidence for SmPL initialisation rules after parallel script execution
> Without having the semantic patch and the .c file, ideally with both > cut down to a minimal version that shows the problem, > I can't answer this question. Can you generally clarify current consequences from usage of a SmPL rule “initialize:…” together with the parameter “--jobs” for the program “spatch”? Regards, Markus ___ Cocci mailing list Cocci@systeme.lip6.fr https://systeme.lip6.fr/mailman/listinfo/cocci
