[Cocci] Querying symbol tables in SmPL scripts?
Hello, The semantic patch language supports to search for various identifiers in source files. It is often hoped then that a found identifier fits to a known application context. But it can happen with generic source code analysis approaches that a desirable confidence level can not easily be achieved. Thus I would be looking for additional means to reduce uncertainty about a software situation considerably. I imagine that a corresponding possibility would be the use of symbol tables. Such tables provide some information for known symbols. How do you think about to take such data structures into account a bit more? Regards, Markus ___ Cocci mailing list Cocci@systeme.lip6.fr https://systeme.lip6.fr/mailman/listinfo/cocci
Re: [Cocci] More support for SmPL data processing with databases?
> I think that you just want to disable the isomorphism. Put disable neg_if > in the initial @@ of your rule. >> >> Will a need evolve to switch isomorphisms completely off for safer >> (and efficient) analysis of original source code? Will the corresponding system configuration get any further development considerations? Regards, Markus ___ Cocci mailing list Cocci@systeme.lip6.fr https://systeme.lip6.fr/mailman/listinfo/cocci
Re: [Cocci] More support for SmPL data processing with databases?
>> I would find it nice to explain the different software behaviour for the >> mentioned >> SmPL code variants > > Run spatch -parse-cocci and you will easily see. The if stays in the same > place. When the isomorphism exchanges the branches, the condition gets > negated, so in one case you get the position of the full condition and > in the other case you get the position of the part of the condition under > the negation. Do any of these transformations avoid the reporting of duplicate source code positions? >>> I think that you just want to disable the isomorphism. Put disable neg_if >>> in the initial @@ of your rule. Will a need evolve to switch isomorphisms completely off for safer (and efficient) analysis of original source code? Regards, Markus ___ Cocci mailing list Cocci@systeme.lip6.fr https://systeme.lip6.fr/mailman/listinfo/cocci
Re: [Cocci] More support for SmPL data processing with databases?
>>> Maybe you consider the following results to be a problem:
>>>
>>> kzalloc mcde [("mcde_probe", 2, 0, 198, 1, "m.c", 16, 1, 16, 3)]
>>> kzalloc ! mcde [("mcde_probe", 2, 0, 198, 1, "m.c", 16, 1, 16, 3)]
>>> platform_get_irq irq [("mcde_probe", 2, 0, 198, 1, "m.c", 97, 1, 97, 3)]
>>> platform_get_irq ! irq [("mcde_probe", 2, 0, 198, 1, "m.c", 97, 1, 97, 3)]
>>>
>>> The positions are the same. The result is because of an isomorphism that
>>> exchanges the branches of a conditional.
I would find it nice to explain the different software behaviour for the
mentioned
SmPL code variants
A:
if@p (
( <+... result ...+>
& check
) )
B:
if (
( <+... result ...+>
& check@p
) )
>> Can the data which are provided because of isomorphism transformations
>> be distinguished by an additional attribute?
>
> I think that you just want to disable the isomorphism. Put disable neg_if
> in the initial @@ of your rule.
Thanks for this suggestion.
Source code analysis can be continued also with this configuration option.
Now I become also more interested in improvements around parallel data
processing.
Which data structures would support aggregate computations for a growing code
base?
Regards,
Markus
___
Cocci mailing list
Cocci@systeme.lip6.fr
https://systeme.lip6.fr/mailman/listinfo/cocci
Re: [Cocci] More support for SmPL data processing with databases?
> Maybe you consider the following results to be a problem:
>
> kzalloc mcde [("mcde_probe", 2, 0, 198, 1, "m.c", 16, 1, 16, 3)]
> kzalloc ! mcde [("mcde_probe", 2, 0, 198, 1, "m.c", 16, 1, 16, 3)]
> platform_get_irq irq [("mcde_probe", 2, 0, 198, 1, "m.c", 97, 1, 97, 3)]
> platform_get_irq ! irq [("mcde_probe", 2, 0, 198, 1, "m.c", 97, 1, 97, 3)]
>
> The positions are the same. The result is because of an isomorphism that
> exchanges the branches of a conditional.
Can the data which are provided because of isomorphism transformations
be distinguished by an additional attribute?
> I think that you could have figured this out by yourself. If you just
> printed out the data instead of wrapping it in lots of database noise,
> then you could have seen what was going on.
I got the impression that there are further software development
challenges to consider (also for safer counting of record sets).
Regards,
Markus
___
Cocci mailing list
Cocci@systeme.lip6.fr
https://systeme.lip6.fr/mailman/listinfo/cocci
Re: [Cocci] More support for SmPL data processing with databases?
> @script:ocaml@ > action << find.action; > check << find.check; > p << find.p; > @@ > Printf.printf "%s %s %s\n" action check (Dumper.dump p) > > On the code you provided whichis unfortunately much too long to include in > a message I found my code examples short enough. > and that you could have surely shortened to illustrate whatever > is the problem. Does such a feedback indicate that you are still struggling with the understanding for the presented software situation? > I get the same output from both semantic patches, except that the line > numbers are different for the position variable p, Does this information contain a contradiction? Do you observe any repeated data sets? > because you have put it in a different place. Yes. - Thus I would expect that different column numbers will be displayed while the line numbers should be (mostly?) identical. > I don't see any reason why making the script code call a database would > have any impact on this issue. The database can provide ACID properties for parallel data processing, can't it? I would like to perform aggregate computations finally. Would you become interested in counting record sets anyhow? > If you do get different results for the above semantic patches, > please make a small .c file that illustrates the problem. I presented questionable differences once more a moment ago. https://lore.kernel.org/cocci/7228efff-76bb-44c9-e2cd-ea19bbd49...@web.de/ https://systeme.lip6.fr/pipermail/cocci/2020-April/007142.html Regards, Markus ___ Cocci mailing list Cocci@systeme.lip6.fr https://systeme.lip6.fr/mailman/listinfo/cocci
Re: [Cocci] More support for SmPL data processing with databases?
>> How will the application of SmPL conjunctions evolve further?
>
> I really have no idea what you are talking about.
I suggest to take another look at affected implementation details.
> Make one single mail that contains all of the semantic patch variants
> that you want to have considered and explain what the problem is.
Would you find the following data processing approach easier to clarify?
I extracted a bit of source code from a known file for your convenience
as an attachment.
https://elixir.bootlin.com/linux/v5.6.3/source/drivers/gpu/drm/mcde/mcde_drv.c#L307
https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/drivers/gpu/drm/mcde/mcde_drv.c?id=8f3d9f354286745c751374f5f1fcafee6b3f3136#n308
> These semantic patches should involve no use of databases.
A Python dictionary can also be used as a data storage technique
for the discussed system test.
@initialize:python@
@@
import sys
records = {}
class integrity_error:
pass
def store_positions(places, check_in, action_in):
"""Add source code positions to an internal table."""
for place in places:
key = (place.file, place.line, int(place.column) + 1)
if key in records:
sys.stderr.write("\n".join(["-> duplicate data",
"file:", key[0],
"function:", place.current_element,
"line:", str(place.line)]))
sys.stderr.write("\n")
raise integrity_error
else:
records[key] = (action_in, check_in, place.current_element)
@find@
expression action, check, result;
position p;
statement is, es;
@@
result = action(...);
if@p (
( <+... result ...+>
& check
) )
is
else
es
@script:python collection@
c << find.check;
action << find.action;
places << find.p;
@@
store_positions(places, c, action)
@finalize:python@
@@
if len(records) > 0:
delimiter = "|"
sys.stdout.write(delimiter.join(['action',
'check',
'"source file"',
'line',
'column'
]))
sys.stdout.write("\r\n")
for key, value in records.items():
sys.stdout.write(delimiter.join([value[0],
value[1],
key[0],
key[1],
str(key[2])
]))
sys.stdout.write("\r\n")
else:
sys.stderr.write("No result for this analysis!\n")
elfring@Sonne:~/Projekte/Coccinelle/janitor> spatch
list_condition_checks_after_function_calls7.cocci ../Probe/mcde_drv-excerpt1.c
…
-> duplicate data
file:
../Probe/mcde_drv-excerpt1.c
function:
mcde_probe
line:
16
Traceback …:
…, line 30, in store_positions
__main__.integrity_error: <__main__.integrity_error instance at 0x7fd3bff4d730>
…
Would you like compare the software behaviour any further with
the following rule variant for the semantic patch language?
@find@
expression action, check, result;
position p;
statement is, es;
@@
result = action(...);
if (
( <+... result ...+>
& check@p
) )
is
else
es
Regards,
Markus
// SPDX-License-Identifier: GPL-2.0
static int mcde_probe(struct platform_device *pdev)
{
struct device *dev = >dev;
struct drm_device *drm;
struct mcde *mcde;
struct component_match *match = NULL;
struct resource *res;
u32 pid;
u32 val;
int irq;
int ret;
int i;
mcde = kzalloc(sizeof(*mcde), GFP_KERNEL);
if (!mcde)
return -ENOMEM;
mcde->dev = dev;
ret = drm_dev_init(>drm, _drm_driver, dev);
if (ret) {
kfree(mcde);
return ret;
}
drm = >drm;
drm->dev_private = mcde;
platform_set_drvdata(pdev, drm);
/* Enable continuous updates: this is what Linux' framebuffer expects */
mcde->oneshot_mode = false;
drm->dev_private = mcde;
/* First obtain and turn on the main power */
mcde->epod = devm_regulator_get(dev, "epod");
if (IS_ERR(mcde->epod)) {
ret = PTR_ERR(mcde->epod);
dev_err(dev, "can't get EPOD regulator\n");
goto dev_unref;
}
ret = regulator_enable(mcde->epod);
if (ret) {
dev_err(dev, "can't enable EPOD regulator\n");
goto dev_unref;
}
mcde->vana = devm_regulator_get(dev, "vana");
if (IS_ERR(mcde->vana)) {
ret = PTR_ERR(mcde->vana);
dev_err(dev, "can't get VANA regulator\n");
goto regulator_epod_off;
}
ret = regulator_enable(mcde->vana);
if (ret) {
dev_err(dev, "can't enable VANA regulator\n");
goto regulator_epod_off;
}
/*
* The vendor code uses ESRAM (onchip RAM) and need to activate
* the v-esram34 regulator, but we don't use that yet
*/
/* Clock the silicon so we can access the registers */
mcde->mcde_clk = devm_clk_get(dev, "mcde");
if (IS_ERR(mcde->mcde_clk)) {
dev_err(dev, "unable to get MCDE main clock\n");
ret = PTR_ERR(mcde->mcde_clk);
goto regulator_off;
Re: [Cocci] More support for SmPL data processing with databases?
>> Do I present another advanced use case for the application of known software >> tools? > > No. Such a view can be also interesting. > The faults in linux paper (work done 10 years ago) uses a database as well. Thanks for this background information. How will the software evolve further in affected areas? Regards, Markus ___ Cocci mailing list Cocci@systeme.lip6.fr https://systeme.lip6.fr/mailman/listinfo/cocci
Re: [Cocci] More support for SmPL data processing with databases?
> Now you have the two scripts, which is good, but you haven't included the > test data that causes the problem. Please make one mail that includes > everything. I find that I provided relevant data already. https://systeme.lip6.fr/pipermail/cocci/2020-April/007135.html https://lore.kernel.org/cocci/ab6d402e-3c12-25ab-162a-f200c382d...@web.de/ Regards, Markus ___ Cocci mailing list Cocci@systeme.lip6.fr https://systeme.lip6.fr/mailman/listinfo/cocci
Re: [Cocci] More support for SmPL data processing with databases?
>>> The database itself doesn't bring any value. >> >> Another unique constraint violation was noticed by using such technology. > > It's an accident. This might be. - I got into the development mood to publish patches for a few source files of Linux modules according to another search approach. I picked one of these files up for the discussed test. > You could have noticed the same thing by just printing the results > in the python script. This can be also possible in principle. I find the programming convenience different. >> I am curious when you are going to take related functionality better into >> account. > > Never. Interesting … > It has nothing to do with Coccinelle. Do I present another advanced use case for the application of known software tools? > When you make a bug report, you should show the minimal code that shows the > problem. I hope that variations can occasionally help also to achieve a better understanding. Regards, Markus ___ Cocci mailing list Cocci@systeme.lip6.fr https://systeme.lip6.fr/mailman/listinfo/cocci
Re: [Cocci] More support for SmPL data processing with databases?
>> I observe that the following SmPL code variant can result also in >> a significant difference. >> >> @find@ >> expression action, check, result; >> position p; >> statement is, es; >> @@ >> result = action(...); >> if ( >> ( <+... result ...+> >> & check@p >> ) ) >> is >> else >> es >> >> >> How will the application of SmPL conjunctions evolve further? > > I really have no idea what you are talking about. I suggest to take another look for the software behaviour. > that contains all of the semantic patch variants that you want to have > These semantic patches should involve no use of databases. Please compare details with the following SmPL code example. @find@ expression action, check, result; position p; statement is, es; @@ result = action(...); if@p ( ( <+... result ...+> & check ) ) is else es Under which circumstances will the Coccinelle software provide duplicate data for their processing by (Python) script rules? > The database itself doesn't bring any value. Another unique constraint violation was noticed by using such technology. I am curious when you are going to take related functionality better into account. Regards, Markus ___ Cocci mailing list Cocci@systeme.lip6.fr https://systeme.lip6.fr/mailman/listinfo/cocci
Re: [Cocci] More support for SmPL data processing with databases?
> I hope that it can become easier to clarify where unexpected duplicate keys > would occur as in my test approach. There is the possibility to apply information according to a clarification request on a topic like “Checking the handling of unique keys/indexes” from 2014-12-20. https://groups.google.com/d/msg/sqlalchemy/klmUwiirIQw/g6NLiVAGGAIJ elfring@Sonne:~/Projekte/Linux/next-patched> spatch ~/Projekte/Coccinelle/janitor/list_condition_checks_after_function_calls2c.cocci drivers/gpu/drm/mcde/mcde_drv.c … Using SQLAlchemy version: 1.3.15 … %% function: mcde_probe line: 322 %% function: mcde_probe line: 403 action|check|"source file"|line|column … How should the error reporting be clarified around such source code places? https://elixir.bootlin.com/linux/v5.6.3/source/drivers/gpu/drm/mcde/mcde_drv.c#L307 https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/drivers/gpu/drm/mcde/mcde_drv.c?id=8f3d9f354286745c751374f5f1fcafee6b3f3136#n308 I observe that the following SmPL code variant can result also in a significant difference. @find@ expression action, check, result; position p; statement is, es; @@ result = action(...); if ( ( <+... result ...+> & check@p ) ) is else es How will the application of SmPL conjunctions evolve further? Regards, Markus ___ Cocci mailing list Cocci@systeme.lip6.fr https://systeme.lip6.fr/mailman/listinfo/cocci
Re: [Cocci] More support for SmPL data processing with databases?
>> I hope that it can become easier to clarify where unexpected duplicate keys >> would occur as in my test approach. > > Use --debug and just print out the infomation rather than putting is in > your database. We have got different views around this logging approach. > With the database you add complexity for nothing. I disagree to this view. Database software usually provides some helpful functionality, doesn't it? The mentioned programming interface quickly pointed an unique constraint violation out. Now I am looking again for possible solutions. Regards, Markus ___ Cocci mailing list Cocci@systeme.lip6.fr https://systeme.lip6.fr/mailman/listinfo/cocci
Re: [Cocci] More support for SmPL data processing with databases?
> I think that your issue about something matching or not has nothing to do > with the database code, and you could easily remove it for the purposes of > reporting a concern with Coccinelle. Software evolution can be continued also together with your constructive feedback. I adapted another SmPL script according to a recent information. https://lore.kernel.org/cocci/17a4592b-92bd-e4c9-8481-7d46616cb...@web.de/T/#m4f59721d2011ac4ded602f2e63e0c63f22a15fa9 https://systeme.lip6.fr/pipermail/cocci/2020-April/007133.html It can become more interesting to increase also the application of SmPL conjunctions like the following. @find@ expression action, check, result; position p; statement is, es; @@ result = action(...); if@p ( ( <+... result ...+> & check ) ) is else es Now I stumble on corresponding software development challenges. elfring@Sonne:~/Projekte/Linux/next-patched> spatch ~/Projekte/Coccinelle/janitor/list_condition_checks_after_function_calls2.cocci drivers/gpu/drm/mcde/mcde_drv.c … Using SQLAlchemy version: 1.3.15 … sqlalchemy.exc.IntegrityError: (sqlite3.IntegrityError) UNIQUE constraint failed: pairs.function, pairs.source_file, pairs.line, pairs.column [SQL: INSERT INTO pairs (function, source_file, line, "column", action, "check") VALUES (?, ?, ?, ?, ?, ?)] … I hope that it can become easier to clarify where unexpected duplicate keys would occur as in my test approach. Regards, Markus ___ Cocci mailing list Cocci@systeme.lip6.fr https://systeme.lip6.fr/mailman/listinfo/cocci
Re: [Cocci] Reporting variations of condition checks according to function calls
>> Can the matched source code from the construct “<+... e ...+>” be assigned >> to a metavariable like “check”? > > ( > <+... e ...+> > & > check > ) Did I just stumble on incomplete knowledge for the safe application of the conjunction functionality with the semantic patch language? It seems that some source code search approaches can be improved in such a way. Regards, Markus ___ Cocci mailing list Cocci@systeme.lip6.fr https://systeme.lip6.fr/mailman/listinfo/cocci
Re: [Cocci] Reporting variations of condition checks according to function calls
@display2@ expression check; position display1.p; statement display1.is, display1.es; >>> >>> The problem is that you inherit es. Since you inherit it, Coccinelle >>> considers that its presence is important, and so the isomorphism will not >>> eliminate it. >> >> Thanks for your explanation of such consequences by the metavariable >> inheritance. It seems that the specification of the inheritance for the metavariable “es” can be (temporarily) omitted as a workaround. >>> It is sufficient to make another pattern for the case with no else. >> >> I might stumble on another software design conflict (or the need >> for corresponding software extensions). >> >> I tried the specification of the SmPL rule “display2” out because I wanted >> to determine the source code which was matched by the SmPL nest construct >> in the SmPL rule “display1”. >> The specification of a SmPL disjunction in the first rule would trigger >> other known development difficulties (or open issues), wouldn't it? > > No idea. I would prefer to find the shown statement combination by a single SmPL rule. Can the matched source code from the construct “<+... e ...+>” be assigned to a metavariable like “check”? Regards, Markus ___ Cocci mailing list Cocci@systeme.lip6.fr https://systeme.lip6.fr/mailman/listinfo/cocci
Re: [Cocci] Reporting variations of condition checks according to function calls
>> @display1@ >> expression action, e; >> position p; >> statement is, es; >> @@ >> *e = action(...); >> if@p (<+... e ...+>) >> is >> else >> es >> >> @display2@ >> expression check; >> position display1.p; >> statement display1.is, display1.es; > > The problem is that you inherit es. Since you inherit it, Coccinelle > considers that its presence is important, and so the isomorphism will not > eliminate it. Thanks for your explanation of such consequences by the metavariable inheritance. > It is sufficient to make another pattern for the case with no else. I might stumble on another software design conflict (or the need for corresponding software extensions). I tried the specification of the SmPL rule “display2” out because I wanted to determine the source code which was matched by the SmPL nest construct in the SmPL rule “display1”. The specification of a SmPL disjunction in the first rule would trigger other known development difficulties (or open issues), wouldn't it? Regards, Markus ___ Cocci mailing list Cocci@systeme.lip6.fr https://systeme.lip6.fr/mailman/listinfo/cocci
Re: [Cocci] Reporting variations of condition checks according to function calls
>> I hope that another clarification can be achieved also for the software >> behaviour of the following source code analysis approach. > > I don't run code that involves databases. If you believe that there is a > problem with the semantic patch, you have to make a small version that > illustrates the problem. Would you find the test result easier to clarify for the following SmPL script example? @display1@ expression action, e; position p; statement is, es; @@ *e = action(...); if@p (<+... e ...+>) is else es @display2@ expression check; position display1.p; statement display1.is, display1.es; @@ *if@p (check) is else es elfring@Sonne:~/Projekte/Coccinelle/janitor> spatch --parse-cocci show_condition_checks_after_function_calls2.cocci … warning: iso drop_else does not match the code below on line 17 if@p (check) iselse es … Will any software improvements become interesting for such an use case? Regards, Markus ___ Cocci mailing list Cocci@systeme.lip6.fr https://systeme.lip6.fr/mailman/listinfo/cocci
Re: [Cocci] Checking the application of the SmPL isomorphism “drop_else”
> I noticed that there was a report about an isomorphism not applying. I hoped for another clarification also for the message “warning: iso drop_else does not match the code below on line 55” (and the corresponding debug display). > That issurely the problem. So you have to figure out why it does not apply. > There is an isomorphism that drops else's in ifs, but that only applies if > the else is a metavariable that is not used elsewhere. I am looking for further improvements in this software area. Regards, Markus ___ Cocci mailing list Cocci@systeme.lip6.fr https://systeme.lip6.fr/mailman/listinfo/cocci
Re: [Cocci] More support for SmPL data processing with databases?
> I think that your issue about something matching or not has nothing to do > with the database code, Such a view can be partly appropriate. > and you could easily remove it I hope that the understanding of the presented SmPL code example could also be sufficient in the way that the data processing for the SmPL rules “find1” and “check1” will need another bit of clarification. > for the purposes of reporting a concern with Coccinelle. Another code variant can eventually be clarified a bit later. Regards, Markus ___ Cocci mailing list Cocci@systeme.lip6.fr https://systeme.lip6.fr/mailman/listinfo/cocci
Re: [Cocci] More support for SmPL data processing with databases?
>> I hope that another clarification can be achieved also for the software >> behaviour of the following source code analysis approach. > > I don't run code that involves databases. Can the situation evolve in a way so that this programming interface will become better supported together with programming languages and development tools you are more familiar with (so far)? Regards, Markus ___ Cocci mailing list Cocci@systeme.lip6.fr https://systeme.lip6.fr/mailman/listinfo/cocci
Re: [Cocci] Reporting variations of condition checks according to function calls
>> I hope that another clarification can be achieved also for the software >> behaviour of the following source code analysis approach. > > I don't run code that involves databases. I find the handling of higher level data storage relevant and useful also for such a software test. > If you believe that there is a problem with the semantic patch, > you have to make a small version that illustrates the problem. My SmPL script example seems to work as expected (in principle) if I omit the else branch from the SmPL rules “find1” and “check1”. Regards, Markus ___ Cocci mailing list Cocci@systeme.lip6.fr https://systeme.lip6.fr/mailman/listinfo/cocci
Re: [Cocci] Reporting variations of condition checks according to function calls
> If you want something done, please do it yourself.
I hope that another clarification can be achieved also for the software
behaviour of the following source code analysis approach.
@initialize:python@
@@
import sqlalchemy, sys
sys.stderr.write("\n".join(["Using SQLAlchemy version:",
sqlalchemy.__version__]))
sys.stderr.write("\n")
from sqlalchemy import create_engine, Column, Integer, String
engine = create_engine("sqlite:///:memory:", echo = False)
from sqlalchemy.ext.declarative import declarative_base
base = declarative_base()
class pair(base):
__tablename__ = "pairs"
function = Column(String, primary_key = True)
source_file = Column(String, primary_key = True)
line = Column(Integer, primary_key = True)
column = Column(Integer, primary_key = True)
action = Column(String)
check = Column(String)
from sqlalchemy.orm import sessionmaker
configured_session = sessionmaker(bind = engine)
session = configured_session()
base.metadata.create_all(engine)
def store_positions(places, check_in, action_in):
"""Add source code positions to an internal table."""
for place in places:
entry = pair(function = place.current_element,
source_file = place.file,
line = place.line,
column = int(place.column) + 1,
action = action_in,
check = check_in
)
session.add(entry)
@find1@
expression action, e;
position p;
statement is, es;
@@
e = action(...);
if@p (<+... e ...+>)
is
else
es
@check1@
expression check;
position find1.p;
statement find1.is, find1.es;
@@
if@p (check)
is
else
es
@script:python collection1@
c << check1.check;
action << find1.action;
places << find1.p;
@@
store_positions(places, c, action)
@finalize:python@
@@
session.commit()
from sqlalchemy import func
entries = session.query(func.count("*")).select_from(pair).scalar()
if entries > 0:
delimiter = "|"
sys.stdout.write(delimiter.join(['action',
'check',
'"source file"',
"line",
"column"
]))
sys.stdout.write("\r\n")
for action, \
check, \
file, \
line, \
column in session.query(pair.action,
pair.check,
pair.source_file,
pair.line,
pair.column
).order_by(pair.action,
pair.check,
pair.source_file,
pair.line,
pair.column
):
sys.stdout.write(delimiter.join([action,
check,
file,
str(line),
str(column)
]))
sys.stdout.write("\r\n")
else:
sys.stderr.write("No result for this analysis!\n")
elfring@Sonne:~/Projekte/Coccinelle/janitor> spatch --parse-cocci
list_condition_checks_after_function_calls2.cocci
…
warning: iso drop_else does not match the code below on line 55
if@p (check) iselse es
…
elfring@Sonne:~/Projekte/Linux/next-patched> spatch
~/Projekte/Coccinelle/janitor/list_condition_checks_after_function_calls2.cocci
drivers/gpu/drm/mcde/mcde_drv.c
…
Using SQLAlchemy version:
1.3.10
HANDLING: drivers/gpu/drm/mcde/mcde_drv.c
No result for this analysis!
Will any software development possibilities become interesting here?
Regards,
Markus
___
Cocci mailing list
Cocci@systeme.lip6.fr
https://systeme.lip6.fr/mailman/listinfo/cocci
Re: [Cocci] Reporting variations of condition checks according to function calls
>> Would you like to display statistics around the distribution of condition >> checks after function calls (with help of the semantic patch language)? > > I'm working on other things. Thanks for your feedback. > If you want something done, please do it yourself. I assumed that you (or other contributors) could help to reduce my concerns also around limited data processing resources for challenging source code analysis approaches. Regards, Markus ___ Cocci mailing list Cocci@systeme.lip6.fr https://systeme.lip6.fr/mailman/listinfo/cocci
Re: [Cocci] Reporting variations of condition checks according to function calls
> Maybe you can just take inspiration from this paper: > > https://pages.lip6.fr/Julia.Lawall/dsn09.pdf I got another idea which can be related to information from the section “Case Study: Inconsistent Error Checks” in your document “WYSIWIB: A Declarative Approach to Finding API Protocols and Bugs in Linux Code”. A key aspect is the knowledge about failure predicates for functions from programming interfaces. How do you think about to improve the knowledge by the means of advanced source code analysis? I got the impression that such analysis attempts can become especially challenging if data processing resources might occasionally be too limited for a growing code base like Linux software. Would you like to display statistics around the distribution of condition checks after function calls (with help of the semantic patch language)? Regards, Markus ___ Cocci mailing list Cocci@systeme.lip6.fr https://systeme.lip6.fr/mailman/listinfo/cocci
Re: [Cocci] Coccinelle rule for CVE-2019-18683
> Analysing each matching case would take a lot of time. How many efforts would you like to invest in adjusting the situation? Will any more development possibilities picked up to reduce the presentation of false positives by the mentioned source code analysis approach considerably? Regards, Markus ___ Cocci mailing list Cocci@systeme.lip6.fr https://systeme.lip6.fr/mailman/listinfo/cocci
Re: [Cocci] Searching for functions with negative return values as error indication
> Maybe you can just take inspiration from this paper: > > https://pages.lip6.fr/Julia.Lawall/dsn09.pdf I find the information from this document “WYSIWIB: A Declarative Approach to Finding API Protocols and Bugs in Linux Code” also interesting. Did the mentioned tools “Search”, “Instantiate”, “MakeBugReport” and “MultiSearch” evolve any further (besides the Coccinelle software in the meantime)? It seems to be more popular to look for inconsistent error checks together with pointer data types. Can the software situation be improved also around integral data types currently? Will the known limitations of Coccinelle for protocol and bug finding get any more software development attention? Regards, Markus ___ Cocci mailing list Cocci@systeme.lip6.fr https://systeme.lip6.fr/mailman/listinfo/cocci
[Cocci] Searching for functions with negative return values as error indication
Hello, Several functions for a programming language like C are designed in the way that values are returned by an integral data type. Specific value ranges can indicate then a failed function call. A well-known variant of this design pattern is that negative return values represent failures (while the other values can be used for succesful data processing. Can the semantic patch language help any more to determine the list of functions which use this style of error reporting in a selected code base? Regards, Markus ___ Cocci mailing list Cocci@systeme.lip6.fr https://systeme.lip6.fr/mailman/listinfo/cocci
Re: [Cocci] Coccinelle rule for CVE-2019-18683
>> * The source code search pattern can be too generic. >> How do you think about to consider additional constraints >> for safer data control flow analysis? > > Could you please elaborate on that? Julia Lawall chose to mention the design possibility “put when != mutex_lock(E) after the ...”. https://systeme.lip6.fr/pipermail/cocci/2020-April/007107.html https://lore.kernel.org/cocci/alpine.DEB.2.21.2004091248190.2403@hadrien/ > I used 'exists' keyword to find at least one branch that has > mutex_unlock+kthread_stop+mutex_lock chain. Are you informed about development challenges for data flow analysis (or even escape analysis according to computer science)? How many experiences can be reused from other known approaches? Regards, Markus ___ Cocci mailing list Cocci@systeme.lip6.fr https://systeme.lip6.fr/mailman/listinfo/cocci
Re: [Cocci] Coccinelle: zalloc-simple: Fix patch mode for dma_alloc_coherent()
> What should I be looking for in the archives? https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/scripts/coccinelle/api/alloc/zalloc-simple.cocci?id=dfd32cad146e3624970eee9329e99d2c6ef751b3 Several software development discussions might be interesting also before the change “dma-mapping: remove dma_zalloc_coherent()”. > I've referenced what I think is the relevant commit here. How do you think about the extend the software development background another bit? > I have replied to these messages now. Do you refer to your response from 2020-04-07? https://systeme.lip6.fr/pipermail/cocci/2020-April/007102.html https://lore.kernel.org/cocci/20200407155005.rdyiu7xqss57rzhq@lenovo-laptop/ * You proposed a change for this SmPL script which contains programming mistakes. * Would you like to achieve the restore of a SmPL code variant from a published patch hunk instead? * Will the software development attention grow also around the recurring topic “Code duplications in SmPL disjunctions”? Regards, Markus ___ Cocci mailing list Cocci@systeme.lip6.fr https://systeme.lip6.fr/mailman/listinfo/cocci
Re: [Cocci] Coccinelle rule for CVE-2019-18683
> Do you have any idea how to improve it? I see further software development possibilities of varying relevance also for this script of the semantic patch language. * The SmPL variables “lock_p”, “unlock_p” and “stop_p” could be declared in a more succinct way just by listing them in the same statement. * The source code search pattern can be too generic. How do you think about to consider additional constraints for safer data control flow analysis? * Other operation modes might become helpful. Regards, Markus ___ Cocci mailing list Cocci@systeme.lip6.fr https://systeme.lip6.fr/mailman/listinfo/cocci
Re: [Cocci] Coccinelle: zalloc-simple: Fix patch mode for dma_alloc_coherent()
>> Will the software development attention grow in a way so that further >> implementation details can be adjusted also for the mentioned SmPL script? > > I'm not sure I understand what you mean. Would you mind clarifying? You would like to a change a file for which some software development discussions happened a while ago. Do you get any further ideas from available information in message archives? Julia Lawall (and me) pointed special development concerns out. Example: Code duplications in SmPL disjunctions https://lore.kernel.org/cocci/alpine.DEB.2.21.2004062115000.10239@hadrien/ https://systeme.lip6.fr/pipermail/cocci/2020-April/007099.html Regards, Markus ___ Cocci mailing list Cocci@systeme.lip6.fr https://systeme.lip6.fr/mailman/listinfo/cocci
Re: [Cocci] Function-like macro with the same name as a typedef
> * spatch seems to have trouble parsing > > ARMSSE *s = ARMSSE(opaque); > > where ARMSSE is a typedef, and ARMSSE() is a function-like macro. > Yes, I know that's in rather poor taste. I am curious for the safe handling of related software development challenges. > * Workaround #1: rename the typedef to ARMSSE_ Will adjustments become more interesting also around such name space issues? Regards, Markus ___ Cocci mailing list Cocci@systeme.lip6.fr https://systeme.lip6.fr/mailman/listinfo/cocci
Re: [Cocci] [PATCH] Coccinelle: zalloc-simple: Fix patch mode for dma_alloc_coherent()
> Commit dfd32cad146e ("dma-mapping: remove dma_zalloc_coherent()"), in
> removing dma_zalloc_coherent() treewide, inadvertently removed the patch
> rule for dma_alloc_coherent(), leaving Coccinelle unable to auto-generate
> patches for this case. Fix this.
I suggest to reconsider also the distribution of recipients for your patch
according to the fields “Cc” and “To”.
Will the software development attention grow in a way so that further
implementation details can be adjusted also for the mentioned SmPL script?
Regards,
Markus
___
Cocci mailing list
Cocci@systeme.lip6.fr
https://systeme.lip6.fr/mailman/listinfo/cocci
Re: [Cocci] linux: special logic in repetitive tests
> Wonder harder. > > If something is (& 3) it is also either (& 2 || & 1) > so why have both below the first test? Would you occasionally prefer to use enumeration values together with switch statements at such source code places? Regards, Markus ___ Cocci mailing list Cocci@systeme.lip6.fr https://systeme.lip6.fr/mailman/listinfo/cocci
Re: [Cocci] linux: special logic in repetitive tests
> Wonder harder. > > If something is (& 3) it is also either (& 2 || & 1) > so why have both below the first test? I can interpret the source code example in the way that the number of set bits are checked. I can eventually become concerned around the preferred coding style because of the usage of these constant values. Would you like to check the use of bit fields? >> * Where do you stumble on difficulties to express a corresponding source code >> search pattern by the means of the semantic patch language? > > #defines, shifts, masks You come along the usual development challenges for the safe handling of possible code variations. With which approach would you like to start and perform further source code analysis? Regards, Markus ___ Cocci mailing list Cocci@systeme.lip6.fr https://systeme.lip6.fr/mailman/listinfo/cocci
Re: [Cocci] linux: special logic in repetitive tests
> There is a block of if tests against the same variable > in include/linux/mtd/pfow.h that likely is defective I wonder about this interpretation. > Likely the first test should be something like > > if ((prog_status & 0x03) == 0x03) Why would you prefer such a code variant? > Is there a way for cocci to find this style of bitwise logic defect? * Probably, yes. * How often will this implementation detail really be a “defect”? * Where do you stumble on difficulties to express a corresponding source code search pattern by the means of the semantic patch language? Regards, Markus ___ Cocci mailing list Cocci@systeme.lip6.fr https://systeme.lip6.fr/mailman/listinfo/cocci
Re: [Cocci] [v3 05/10] mmap locking API: Improving the Coccinelle software
>> How will the support evolve for data processing around the application >> of similar macros? > > Not at all. Will such feedback trigger any collateral evolution? Regards, Markus ___ Cocci mailing list Cocci@systeme.lip6.fr https://systeme.lip6.fr/mailman/listinfo/cocci
Re: [Cocci] [v3 05/10] mmap locking API: Improving the Coccinelle software
>> … >> (ONCE) CPP-MACRO: found known macro = FNAME >> … >> parse error … >> How would you like to improve the affected software areas? > > This can be addressed by adding a macro definition to standard.h. A corresponding specification is used already. https://github.com/coccinelle/coccinelle/blob/fdf0b68ddd0a518cc6ca64f063bc74ed54e29a7b/standard.h#L508 > But once the change is done, I don't see any reason to bother with this. How will the support evolve for data processing around the application of similar macros? Regards, Markus ___ Cocci mailing list Cocci@systeme.lip6.fr https://systeme.lip6.fr/mailman/listinfo/cocci
Re: [Cocci] [v3 05/10] mmap locking API: Improving the Coccinelle software
>> How will corresponding software development resources evolve?
>
> I don't think I understand the question, or, actually, are you asking
> me or the coccinelle developers ?
I hope that more development challenges will be picked up.
The code from a mentioned source file can be reduced to the following
test file.
https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/arch/x86/kvm/mmu/paging_tmpl.h?id=7111951b8d4973bda27ff663f2cf18b663d15b48#n122
// deleted part
static inline int FNAME(is_present_gpte)(unsigned long pte)
{
#if PTTYPE != PTTYPE_EPT
return pte & PT_PRESENT_MASK;
#else
return pte & 7;
#endif
}
// deleted part
Application of the software “Coccinelle 1.0.8-00029-ga549b9f0” (OCaml 4.10.0)
elfring@Sonne:~/Projekte/Coccinelle/Probe> spatch --parse-c
paging_tmpl-excerpt1.h
…
(ONCE) CPP-MACRO: found known macro = FNAME
…
parse error
= File "paging_tmpl-excerpt1.h", line 2, column 41, charpos = 57
around = 'unsigned',
…
BAD:! static inline int FNAME(is_present_gpte)(unsigned long pte)
…
NB total files = 1; perfect = 0; pbs = 1; timeout = 0; => 0%
nb good = 1, nb passed = 1 => 10.00% passed
nb good = 1, nb bad = 8 => 20.00% good or passed
How would you like to improve the affected software areas?
Regards,
Markus
___
Cocci mailing list
Cocci@systeme.lip6.fr
https://systeme.lip6.fr/mailman/listinfo/cocci
Re: [Cocci] Using SmPL script to switch to formatted log/print function
> This discussion is about parsing the source code. It seems that an acceptable solution was found according to your advice yesterday. I could offer other ideas for another bit of software fine-tuning. Regards, Markus ___ Cocci mailing list Cocci@systeme.lip6.fr https://systeme.lip6.fr/mailman/listinfo/cocci
Re: [Cocci] Using SmPL script to switch to formatted log/print function
> I have this all working with the following script with the caveat that > running it on moderately complicated source files makes it never > finish (after an hour or so the spatch process crashes with a stack > overflow error). Do you find any information interesting from a previous bug report? https://github.com/coccinelle/coccinelle/issues/30 > Any suggestions on changes to my script that would make this work on > lengthy source files would be greatly appreciated! Additional adjustments can be considered for transformation approaches. Regards, Markus ___ Cocci mailing list Cocci@systeme.lip6.fr https://systeme.lip6.fr/mailman/listinfo/cocci
Re: [Cocci] [v3 05/10] mmap locking API: Checking the Coccinelle software
> The problem can be seen with the --debug option: > > FLOW: can't jump to VMALLOC_FAULT_TARGET: because we can't find this label > > It's not apparent with the --parse-c option because it's not a parsing > problem. Thanks for such information. Can the example be transformed even if extra source code was intentionally deleted for the easier clarification of the shown software test? Regards, Markus ___ Cocci mailing list Cocci@systeme.lip6.fr https://systeme.lip6.fr/mailman/listinfo/cocci
Re: [Cocci] [v3 05/10] mmap locking API: Checking the Coccinelle software
>> How would you like to fix this situation? > > Who exactly do you think "you" is? Every contributor with helpful software development resources for this issue. > I will look at it, Thanks for another promising feedback. > but it is not very polite to ask a user of Coccinelle such a question. We come along different views for the handling of such a bug report. Regards, Markus ___ Cocci mailing list Cocci@systeme.lip6.fr https://systeme.lip6.fr/mailman/listinfo/cocci
Re: [Cocci] [v3 05/10] mmap locking API: Checking the Coccinelle software
>> How will corresponding software development resources evolve?
>
> I don't think I understand the question, or, actually, are you asking
> me or the coccinelle developers ?
I hope that another communication approach can eventually increase
the chances for a better common understanding of development challenges.
The code from a mentioned source file can be reduced to the following
test file.
https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/arch/mips/mm/fault.c?id=69c5eea3128e775fd3c70ecf0098105d96dee330#n34
// deleted part
static void __kprobes __do_page_fault(struct pt_regs *regs, unsigned long write,
unsigned long address)
{
struct vm_area_struct * vma = NULL;
struct task_struct *tsk = current;
struct mm_struct *mm = tsk->mm;
// deleted part
retry:
down_read(>mmap_sem);
vma = find_vma(mm, address);
if (!vma)
goto bad_area;
// deleted part
}
// deleted part
Application of the software “Coccinelle 1.0.8-00029-ga549b9f0” (OCaml 4.10.0)
elfring@Sonne:~/Projekte/Coccinelle/Probe> spatch --parse-c
do_page_fault-excerpt3.c
…
NB total files = 1; perfect = 1; pbs = 0; timeout = 0; => 100%
nb good = 15, nb passed = 1 => 6.25% passed
nb good = 15, nb bad = 0 => 100.00% good or passed
The discussed transformation approach can also be reduced for a test
to the following script for the semantic patch language.
@replacement@
expression x;
@@
-down_read
+mmap_read_lock
(
- &
x
- ->mmap_sem
)
elfring@Sonne:~/Projekte/Coccinelle/Probe> spatch use_mmap_locking_API_3.cocci
do_page_fault-excerpt3.c
The desired diff is not generated so far.
How would you like to fix this situation?
Regards,
Markus
___
Cocci mailing list
Cocci@systeme.lip6.fr
https://systeme.lip6.fr/mailman/listinfo/cocci
Re: [Cocci] [v3 05/10] mmap locking API: convert mmap_sem call sites missed by coccinelle
> I would be interested to find out why coccinelle wasn't able to do the > last 1%, but only as part of a long-term learning process on getting > better with coccinelle - … How will corresponding software development resources evolve? Regards, Markus ___ Cocci mailing list Cocci@systeme.lip6.fr https://systeme.lip6.fr/mailman/listinfo/cocci
Re: [Cocci] [PATCH v3 05/10] mmap locking API: convert mmap_sem call sites missed by coccinelle
> Convert the last few remaining mmap_sem rwsem calls to use the new > mmap locking API. These were missed by coccinelle for some reason Will the clarification of this software situation become more interesting? > (I think coccinelle does not support some of the preprocessor > constructs in these files ?) I suggest to omit this information from the final change description. Would you like to help any more to find nicer solutions for remaining open issues? Regards, Markus ___ Cocci mailing list Cocci@systeme.lip6.fr https://systeme.lip6.fr/mailman/listinfo/cocci
Re: [Cocci] [PATCH v2 05/10] mmap locking API: convert mmap_sem call sites missed by coccinelle
> Convert the last few remaining mmap_sem rwsem calls to use the new > mmap locking API. These were missed by coccinelle for some reason I find such a software situation still unfortunate. * Should the data processing results be clarified any further for the shown transformation approach? * Will additional improvements be considered? Regards, Markus ___ Cocci mailing list Cocci@systeme.lip6.fr https://systeme.lip6.fr/mailman/listinfo/cocci
Re: [Cocci] [v2 04/10] mmap locking API: use coccinelle to convert mmap_sem rwsem call sites
>>> The change is generated using coccinelle with the following rules: >> >> Would you like to apply only a single SmPL rule here? > > I think this version of the patch is already a single rule, > similar to what you suggested ? Yes. - But you repeated the wording “rules” in the change description. Are there any other software extensions still in the waiting queue? > I'm not sure it matters that much, as long as it produces the correct > end result. The run takes about 25 seconds before any optimizations, > which I find very acceptable. I am used to look also at the run time characteristics of SmPL script execution. The discussed SmPL code can be good enough for your current needs. Regards, Markus ___ Cocci mailing list Cocci@systeme.lip6.fr https://systeme.lip6.fr/mailman/listinfo/cocci
Re: [Cocci] [PATCH v2 04/10] mmap locking API: use coccinelle to convert mmap_sem rwsem call sites
> This change converts the existing mmap_sem rwsem calls to use the new > mmap locking API instead. > > The change is generated using coccinelle with the following rules: Would you like to apply only a single SmPL rule here? > // spatch --sp-file mmap_lock_api.cocci --in-place --include-headers --dir . Command parameters like “--jobs 8 --chunksize 1” can be also helpful for a parallel execution of the desired software transformation. I suggest to consider another possibility for a bit of fine-tuning in the shown SmPL script if you would eventually care for nicer run time characteristics for the application of such a SmPL disjunction. How do you think about to order the elements according to a probable function call frequency? Regards, Markus ___ Cocci mailing list Cocci@systeme.lip6.fr https://systeme.lip6.fr/mailman/listinfo/cocci
Re: [Cocci] [PATCH 4/8] mmap locking API: convert mmap_sem call sites missed by coccinelle
> Convert the last few remaining mmap_sem rwsem calls to use the new > mmap locking API. These were missed by coccinelle for some reason I find such a software situation unfortunate. Should the transformation approach be clarified any further? Regards, Markus ___ Cocci mailing list Cocci@systeme.lip6.fr https://systeme.lip6.fr/mailman/listinfo/cocci
Re: [Cocci] Determination of failure predicates for selected function calls with SmPL?
>> I have noticed the patch “tty: serial: fsl_lpuart: fix return value >> checking”. >> https://lore.kernel.org/linux-serial/20200325090658.25967-2-mich...@walle.cc/ … > I'm not sure what there is to clarify. One can surely write rules that > find this kind of problem, with a varying rate of false positives. Will further changes become more interesting also according to other development approaches you care for? * Prequel http://prequel-pql.gforge.inria.fr/ * Machine learning for software corrections https://hal.inria.fr/hal-02373994 https://doi.org/10.1109/TSE.2019.2952614 Regards, Markus ___ Cocci mailing list Cocci@systeme.lip6.fr https://systeme.lip6.fr/mailman/listinfo/cocci
Re: [Cocci] Determination of failure predicates for selected function calls with SmPL?
>> I have noticed the patch “tty: serial: fsl_lpuart: fix return value >> checking”. >> https://lore.kernel.org/linux-serial/20200325090658.25967-2-mich...@walle.cc/ … > I'm not sure what there is to clarify. Such an use case can point some software development challenges out. > One can surely write rules that find this kind of problem, I know this in principle. > with a varying rate of false positives. I am trying again to reduce this special case considerably. > A simple approach would be to check all occurrences of if (!ret) for an > integer-typed variable, because typically that is the success case. I would like to increase the probability for generation of corresponding fixes. The success predicate can be also the opposite if you would like to take a Linux macro like access_ok() or valid pointers (after memory allocations for example) into account. > But sometimes people do put the success case under an if. I am curious how often such source code variations should be considered. Regards, Markus ___ Cocci mailing list Cocci@systeme.lip6.fr https://systeme.lip6.fr/mailman/listinfo/cocci
[Cocci] Determination of failure predicates for selected function calls with SmPL?
Hello, I have noticed the patch “tty: serial: fsl_lpuart: fix return value checking”. https://lore.kernel.org/linux-serial/20200325090658.25967-2-mich...@walle.cc/ https://lore.kernel.org/patchwork/patch/1215542/ https://lkml.org/lkml/2020/3/25/202 I am curious if such a software transformation can be extended for similar source code searches also by the means of the semantic patch language. The proper knowledge of failure predicates for function calls is a key aspect for this use case. Would you like to clarify any corresponding development possibilities? Regards, Markus ___ Cocci mailing list Cocci@systeme.lip6.fr https://systeme.lip6.fr/mailman/listinfo/cocci
Re: [Cocci] [PATCH v3 05/26] parsing_cocci: visitor_ast: Add cases for ParenType/FunctionType
> The order of the terms in ParenType require implementing a special > case for ParenType. This case handles only the following: * Would it be nice to mention a key word like “parentheses”? * How do you distinguish “visitors” for the affected software? * I would prefer less repeated change descriptions for this patch series. Can an other patch granularity become helpful? Regards, Markus ___ Cocci mailing list Cocci@systeme.lip6.fr https://systeme.lip6.fr/mailman/listinfo/cocci
Re: [Cocci] [PATCH v3 02/26] parsing_cocci: AST: Add ParenType and FunctionType to SmPL ASTs
> ParenType and FunctionType are types present in the C AST that > are not present in the SmPL AST. Can the understanding of mentioned details be improved by reconsidering the usage of abbreviations? > In the pursuit of aligning both the C and SmPL ASTs, I find this approach also helpful. > add these types to the SmPL ASTs. * Does such an extension mean a copy of functionality? * Can any components be shared between affected software areas? Regards, Markus ___ Cocci mailing list Cocci@systeme.lip6.fr https://systeme.lip6.fr/mailman/listinfo/cocci
Re: [Cocci] [Linux-kernel-mentees] [v2 05/26] parsing_cocci: visitor_ast: Add cases for ParenType/FunctionType
> One of the hardest things in dealing with a community is to learn who to > ignore when it comes to patch reviews and comments. I suggest to reconsider also this information once more. > Just a friendly reminder that this person is on many upstream kernel > developer's > email-block lists due to the ability to cause huge time-sinks Some clarification attempts will trigger the usual efforts. I hope that our communication difficulties can become more constructive often enough. > for no valid reason. Please take another look at the published software development statistics. Regards, Markus ___ Cocci mailing list Cocci@systeme.lip6.fr https://systeme.lip6.fr/mailman/listinfo/cocci
Re: [Cocci] [v2 05/26] parsing_cocci: visitor_ast: Add cases for ParenType/FunctionType
> Changes in v2: > … > - Change the "visitor_ast0" in patch subject to "visitor_ast" Thanks for such a software update. I suggest to reconsider the patch granularity once more. Would you like to reduce the possibilities for confusion around the presented software transformation? Regards, Markus ___ Cocci mailing list Cocci@systeme.lip6.fr https://systeme.lip6.fr/mailman/listinfo/cocci
Re: [Cocci] [06/26] parsing_cocci: arity: Add cases for ParenType/FunctionType
>> I got special views also around patch squashing. >> Would you like to reduce the number of update steps >> according to the really desired software transformation? > > I asked him to cut it up into small chunks. We come along different preferences for the size of these pieces. Regards, Markus ___ Cocci mailing list Cocci@systeme.lip6.fr https://systeme.lip6.fr/mailman/listinfo/cocci
Re: [Cocci] [06/26] parsing_cocci: arity: Add cases for ParenType/FunctionType
>> How do you think about to combine such changes in a bigger update step? > > These are separated this way so that it's easier for Julia to review. I suggest to reconsider such an approach. > She will probably just squash all 26 of these together and apply them later. I got special views also around patch squashing. Would you like to reduce the number of update steps according to the really desired software transformation? Regards, Markus ___ Cocci mailing list Cocci@systeme.lip6.fr https://systeme.lip6.fr/mailman/listinfo/cocci
Re: [Cocci] [PATCH 06/26] parsing_cocci: arity: Add cases for ParenType/FunctionType
> ParenType and FunctionType are added to the SmPL ASTs. > Add cases for these types in ….ml. A change description template was used several times. I wonder if there is really a need to modify only a single OCaml source file in these update steps. How do you think about to combine such changes in a bigger update step? Regards, Markus ___ Cocci mailing list Cocci@systeme.lip6.fr https://systeme.lip6.fr/mailman/listinfo/cocci
Re: [Cocci] [PATCH 05/26] parsing_cocci: visitor_ast0: Add cases for ParenType/FunctionType
> The order of the terms in ParenType require implementing a special > case for ParenType. This case handles only the following: … It seems that this change description was repeated in other update steps. Does this detail indicate a need to reconsider the patch granularity? Regards, Markus ___ Cocci mailing list Cocci@systeme.lip6.fr https://systeme.lip6.fr/mailman/listinfo/cocci
Re: [Cocci] [PATCH 02/26] parsing_cocci: AST: Add ParenType and FunctionType to SmPL ASTs
> ParenType and FunctionType are types present in the C AST that > are not present in the SmPL AST. Will a bit more background information become helpful for these data types? Regards, Markus ___ Cocci mailing list Cocci@systeme.lip6.fr https://systeme.lip6.fr/mailman/listinfo/cocci
Re: [Cocci] [PATCH 01/26] parsing_cocci: Add Function Prototype token
> Thus, a slightly hacky approach is taken to determine a function > prototype with not the best certainty but what works for most cases > in SmPL. If the identifier is preceded by any token that does not > seem to be part of a type, then it is not identified as a function > prototype. Else, it is. Would you like to extend this explanation? Regards, Markus ___ Cocci mailing list Cocci@systeme.lip6.fr https://systeme.lip6.fr/mailman/listinfo/cocci
Re: [Cocci] [PATCH 00/26] cocci: Add ParenType/FunctionType to SmPL ASTs
> The aim of this patch series is to add the types ParenType Can it be that this kind of data processing will need a bit more explanation? > The FunctionPointer type is removed from the SmPL AST, since > all the productions that produce FunctionPointers in the SmPL > parser are replaced by those that produce ParenType. I imagine that dedicated data types can occasionally be helpful. Regards, Markus ___ Cocci mailing list Cocci@systeme.lip6.fr https://systeme.lip6.fr/mailman/listinfo/cocci
[Cocci] Advanced processing for data streams?
Hello, Can any circumstances evolve under which you would like to perform data processing by the programming means of a software framework which would be similar to GStreamer? https://gstreamer.freedesktop.org/features/ https://opam.ocaml.org/packages/gstreamer/ Regards, Markus ___ Cocci mailing list Cocci@systeme.lip6.fr https://systeme.lip6.fr/mailman/listinfo/cocci
Re: [Cocci] Clarification for application of null pointer checks with SmPL
>>> pointer == NULL would have been sufficient (not related to the noted >>> problems). >> >> Should this detail be handled as an isomorphism? >> https://github.com/coccinelle/coccinelle/blob/19ee1697bf152d37a78a20cefe148775bf4b0e0d/standard.iso#L134 > > Yes. Unfortunately, I might stumble still on difficulties for the understanding of desired and actually observed software behaviour (because of remaining open issues). Regards, Markus ___ Cocci mailing list Cocci@systeme.lip6.fr https://systeme.lip6.fr/mailman/listinfo/cocci
Re: [Cocci] Clarification for application of null pointer checks with SmPL
>> ... when any >> when != pointer = \( action(...) \| x \) >> *if ( >> (!pointer >> |pointer == NULL >> ) > > pointer == NULL would have been sufficient (not related to the noted > problems). Should this detail be handled as an isomorphism? https://github.com/coccinelle/coccinelle/blob/19ee1697bf152d37a78a20cefe148775bf4b0e0d/standard.iso#L134 >> elfring@Sonne:~/Projekte/Linux/next-patched> spatch >> ~/Projekte/Coccinelle/janitor/show_pointer_access_before_null_check3.cocci >> drivers/firmware/efi/runtime-wrappers.c >> … >> Fatal error: exception File "engine/ctlcocci_integration.ml", line 361, >> characters 4-10: Assertion failed >> elfring@Sonne:~/Projekte/Linux/next-patched> spatch >> ~/Projekte/Coccinelle/janitor/show_pointer_access_before_null_check3.cocci >> net/core/dev.c >> … >> no position information > > Thanks for the reports. These are related to dead code and will be > addressed sooner (second case) or later (first case). I am curious on further collateral evolution. Would you like to discuss any extensions for the affected software areas (besides the mentioned error messages)? Regards, Markus ___ Cocci mailing list Cocci@systeme.lip6.fr https://systeme.lip6.fr/mailman/listinfo/cocci
Re: [Cocci] [PATCH 12/13] tests: Add test case for assigned enumerator
> minused assigned enumerators, plussed assigned enumerators Does this description contain typos? Regards, Markus ___ Cocci mailing list Cocci@systeme.lip6.fr https://systeme.lip6.fr/mailman/listinfo/cocci
Re: [Cocci] [PATCH 09/13] cocci: Reflect changes in SmPL visitor_ast in codebase
> as well as adding the additional arguments needed in the visitor How do you think about to reconsider such a wording? Regards, Markus ___ Cocci mailing list Cocci@systeme.lip6.fr https://systeme.lip6.fr/mailman/listinfo/cocci
Re: [Cocci] [PATCH 07/13] parsing_cocci: Reflect visitor_ast0 changes in parsing_cocci
> The SmPL AST0 visitor has functions for handling enumerators > separately. Handle these collateral evolutions in parsing_cocci I got the impression that an other change description would be easier to understand. Regards, Markus ___ Cocci mailing list Cocci@systeme.lip6.fr https://systeme.lip6.fr/mailman/listinfo/cocci
Re: [Cocci] [PATCH 04/13] parsing_cocci: Add EnumDeclTag and EnumDeclDotsTag to SmPL ASTs
> constructs w/r/t changes in the SmPL AST. Would an other change description be more helpful? Regards, Markus ___ Cocci mailing list Cocci@systeme.lip6.fr https://systeme.lip6.fr/mailman/listinfo/cocci
Re: [Cocci] [PATCH 01/13] parsing_cocci: Align C AST and SmPL AST for enum
> Make the enumerator type of the SmPL AST closer to that of > the C AST. Are any deviations between these data structures still relevant (or undesirable)? Regards, Markus ___ Cocci mailing list Cocci@systeme.lip6.fr https://systeme.lip6.fr/mailman/listinfo/cocci
Re: [Cocci] [PATCH 00/13] cocci: Align the C AST and SmPL AST for enum
> The C AST and SmPL AST differs with respect to the enum type. I suggest to consider wording variants. “The … and … differ ….” > collateral evolutions caused by changed in the SmPL visitors are “… by changes in …” Regards, Markus ___ Cocci mailing list Cocci@systeme.lip6.fr https://systeme.lip6.fr/mailman/listinfo/cocci
[Cocci] Checking two error messages
Hello, I have tried the following small script variant out again for the semantic patch language. @display@ expression action; expression* pointer, target, x; identifier member, var; statement is; type t; @@ ( *t* var = pointer->member; | *pointer->member; | *target = pointer->member; | *t* var = pointer->member(...); | *pointer->member(...); | *target = pointer->member(...); ) ... when any when != pointer = \( action(...) \| x \) *if ( (!pointer |pointer == NULL ) ) is The software “Coccinelle 1.0.8-00029-ga549b9f0” (OCaml 4.10.0) reports the following error messages for an analysis approach on source files from “Linux next-20200306”. elfring@Sonne:~/Projekte/Linux/next-patched> spatch ~/Projekte/Coccinelle/janitor/show_pointer_access_before_null_check3.cocci drivers/firmware/efi/runtime-wrappers.c … Fatal error: exception File "engine/ctlcocci_integration.ml", line 361, characters 4-10: Assertion failed elfring@Sonne:~/Projekte/Linux/next-patched> spatch ~/Projekte/Coccinelle/janitor/show_pointer_access_before_null_check3.cocci net/core/dev.c … no position information Is such information worth for further clarification? Regards, Markus ___ Cocci mailing list Cocci@systeme.lip6.fr https://systeme.lip6.fr/mailman/listinfo/cocci
Re: [Cocci] Supporting Coccinelle software
> Why do you want to know? I am looking for related incentives to achieve further progress around affected areas. Regards, Markus ___ Cocci mailing list Cocci@systeme.lip6.fr https://systeme.lip6.fr/mailman/listinfo/cocci
Re: [Cocci] Supporting Coccinelle software
> I cloned coccinelle's GIT repository, I suggested a patch, > and the patch has been accepted. Will the chances increase for further contributions? Regards, Markus ___ Cocci mailing list Cocci@systeme.lip6.fr https://systeme.lip6.fr/mailman/listinfo/cocci
Re: [Cocci] Supporting Coccinelle software
> And because Coccinelle is free software, I could inspect > Coccinelle and even adapt a few minimal things to my needs! Would you like to share a bit more information about such adjustments? > And, dear Coccinelle devels, what we users can do to ensure > good health to the Coccinelle project ? I suggest to contribute additional resources. Regards, Markus ___ Cocci mailing list Cocci@systeme.lip6.fr https://systeme.lip6.fr/mailman/listinfo/cocci
Re: [Cocci] [v4 3/3] tests: Add test case to match const pointer variants
> > Do you mean that the C file/res file should conform to the Linux style? > > I think it would be nicer if it did. Is the interest for the clarification of a topic like “Make change influence configurable for coding style rules” growing? https://github.com/coccinelle/coccinelle/issues/37 Regards, Markus ___ Cocci mailing list Cocci@systeme.lip6.fr https://systeme.lip6.fr/mailman/listinfo/cocci
Re: [Cocci] [v2 1/2] parsing_c: Align C AST and Coccinelle AST for pointers
>> https://en.wikipedia.org/wiki/AST > > Not necessary. Anyone who is looking at the Coccinelle commits should > know what an AST is. I would appreciate to reduce the potential for confusion around the application of parsing technologies. Regards, Markus ___ Cocci mailing list Cocci@systeme.lip6.fr https://systeme.lip6.fr/mailman/listinfo/cocci
Re: [Cocci] [PATCH v2 1/2] parsing_c: Align C AST and Coccinelle AST for pointers
> For a pointer, the C parser constructed an AST dissimilar from that I find it helpful to mention the meaning of the term “AST” without abbreviation for a nicer patch description. https://en.wikipedia.org/wiki/AST Regards, Markus ___ Cocci mailing list Cocci@systeme.lip6.fr https://systeme.lip6.fr/mailman/listinfo/cocci
Re: [Cocci] [v2 1/2] parsing_c: Align C AST and Coccinelle AST for pointers
> + const char *const *y; > } > > There should be a space between * and const on the + line. Will such a detail become more interesting for the clarification around coding styles? https://github.com/coccinelle/coccinelle/issues/37 Regards, Markus ___ Cocci mailing list Cocci@systeme.lip6.fr https://systeme.lip6.fr/mailman/listinfo/cocci
Re: [Cocci] [v2 0/2] Align the C AST and Coccinelle AST for pointers
> Like what? Do find my corresponding previous suggestion reasonable? https://lore.kernel.org/cocci/702596c2-84bf-13ca-b014-8784bbb98...@web.de/ https://systeme.lip6.fr/pipermail/cocci/2020-February/006842.html Regards, Markus ___ Cocci mailing list Cocci@systeme.lip6.fr https://systeme.lip6.fr/mailman/listinfo/cocci
Re: [Cocci] [PATCH v2 0/2] Align the C AST and Coccinelle AST for pointers
> This series is to address … Under which circumstances would you choose an adjusted wording for the message in the cover letter? Regards, Markus ___ Cocci mailing list Cocci@systeme.lip6.fr https://systeme.lip6.fr/mailman/listinfo/cocci
Re: [Cocci] Replacing the type casting
>> REQUIRE(( >> - (T*) >> source) != NULL); > > Thanks, that’s helpful. Thanks for such a view on the suggested transformation approach. >>> REQUIRE(( >>> - (T *) >>> + (dns_rdatacommon_t *) >> >> ( >> -T >> +dns_rdatacommon_t >> *) > > This cannot be done, I propose to take another look at how many adjustments such a type cast will need actually. Will the replacement of the data type be sufficient at this place of SmPL code? > as I need to convert only .common members, … This concern can be added to the referenced change pattern. Regards, Markus ___ Cocci mailing list Cocci@systeme.lip6.fr https://systeme.lip6.fr/mailman/listinfo/cocci
Re: [Cocci] Replacing the type casting
Can the following transformation variants be also helpful? > @@ > type T; > expression source; > @@ > > - REQUIRE(((T *)source) != NULL); > + REQUIRE(source != NULL); REQUIRE(( - (T*) source) != NULL); > @@ > type T; > type R; type T; > identifier common, rdtype; > T *target; > expression source; typedef dns_rdatacommon_t; > @@ > > REQUIRE(source != NULL); > ... > REQUIRE(( > - (T *) > + (dns_rdatacommon_t *) ( -T +dns_rdatacommon_t *) Will any constraints become relevant for the deleted data types? Regards, Markus ___ Cocci mailing list Cocci@systeme.lip6.fr https://systeme.lip6.fr/mailman/listinfo/cocci
Re: [Cocci] Replacing the type casting
> > I must be missing something very obvious as just removing the type works, > > but adding the `+ (dns_rdatacommon_t *)` line makes the rule fail. > > Coccinelle is not able to infer typedefs in casts. Under which circumstances can this software situation be reconsidered? How long will an extra metavariable specification like “typedef dns_rdatacommon_t;” be needed for the mentioned transformation approach? Regards, Markus ___ Cocci mailing list Cocci@systeme.lip6.fr https://systeme.lip6.fr/mailman/listinfo/cocci
Re: [Cocci] Linux dev-tools: Handling complex types and attributes in Coccinelle
> A little collection of scripts/tools I've written[1], Thanks for your link on another bit of background information. https://github.com/jajajasalu2/cocci-type-test-suite Would you like to point any related information sources out? > and the invaluable OCaml debugger :) This development tool can help occasionally. I am curious if your contributions will influence also other known open issues for this software in positive ways. Regards, Markus ___ Cocci mailing list Cocci@systeme.lip6.fr https://systeme.lip6.fr/mailman/listinfo/cocci
Re: [Cocci] Linux dev-tools: Handling complex types and attributes in Coccinelle
> A little collection of scripts/tools I've written[1], Thanks for your link on another bit of background information. https://github.com/jajajasalu2/cocci-type-test-suite Would you like to point any related information sources out? > and the invaluable OCaml debugger :) This development tool can help occasionally. I am curious if your contributions will influence also other known open issues for this software in positive ways. Regards, Markus ___ Cocci mailing list Cocci@systeme.lip6.fr https://systeme.lip6.fr/mailman/listinfo/cocci
Re: [Cocci] Source code transformations around g_autoptr()
> > minus: parse error: > > File "test.cocci", line 3, column 15, charpos = 38 > > around = 'p', > > whole content = - g_autoptr(d) p; > > > > > > is it correct? > > No, Coccinelle knows nothing about these things. How are the chances to improve the software situation for the safe handling of macros besides the extension of a source file like “standard.h”? https://developer.gnome.org/glib/stable/glib-Miscellaneous-Macros.html#g-autoptr Will an additional configuration file be occasionally helpful? https://github.com/coccinelle/coccinelle/blob/7ec31ed1fadf738bc487ccefdc63bfe0598f44cc/docs/manual/spatch_options.tex#L384 Regards, Markus ___ Cocci mailing list Cocci@systeme.lip6.fr https://systeme.lip6.fr/mailman/listinfo/cocci
Re: [Cocci] [PATCH 1/2] parsing_c: Align C AST and Cocci AST for const pointer variants
> For a pointer, the C parser constructed an AST dissimilar from that > of the Cocci AST. This caused failures in matching with certain > pointer types. For example, for the following case: > > char *1 const *2 id; > > The C AST constructed would be: > const Pointer1 -> Pointer2 -> char > > The Cocci AST constructed would be: > Pointer2 -> const Pointer1 -> char Which software development tools support to check such information? Regards, Markus ___ Cocci mailing list Cocci@systeme.lip6.fr https://systeme.lip6.fr/mailman/listinfo/cocci
Re: [Cocci] [2/2] tests: Add test case to match const pointer variants
>> * Should the potential for differences in these “AST” be checked any more? > > An effort is already being made to do this by me and Julia :) I am curious how this collaboration will evolve further. >> * How can such items be compared safely? > > Not sure what you mean. If you mean how the two ASTs can be compared, > it's just a process of data collection and debugging. Which programming interfaces do you use for this purpose at the moment? Regards, Markus ___ Cocci mailing list Cocci@systeme.lip6.fr https://systeme.lip6.fr/mailman/listinfo/cocci
Re: [Cocci] [2/2] tests: Add test case to match const pointer variants
> Would the following be a better subject? The alternative is nicer. Will other implementation details become also more interesting? The information “Align C AST and Cocci AST” was provided. https://lore.kernel.org/cocci/20200205130327.6812-2-jaskaransingh7654...@gmail.com/ https://systeme.lip6.fr/pipermail/cocci/2020-February/006840.html * Should the potential for differences in these “AST” be checked any more? * How can such items be compared safely? Is there a need to extend the test format? https://github.com/coccinelle/coccinelle/issues/134 Regards, Markus ___ Cocci mailing list Cocci@systeme.lip6.fr https://systeme.lip6.fr/mailman/listinfo/cocci
Re: [Cocci] [2/2] tests: Add test case to match const pointer variants
> Would the following be a better subject? I find the alternative succinct. Regards, Markus ___ Cocci mailing list Cocci@systeme.lip6.fr https://systeme.lip6.fr/mailman/listinfo/cocci
Re: [Cocci] [PATCH 2/2] tests: Add test case for matching pointers to const pointers (and variants)
Would you like to avoid any abbreviations in the patch subject? > These types would not match previously. I imagine that this information can be improved for a nicer patch description, can't it? Regards, Markus ___ Cocci mailing list Cocci@systeme.lip6.fr https://systeme.lip6.fr/mailman/listinfo/cocci
Re: [Cocci] [PATCH 1/2] parsing_c: Align C AST and Cocci AST for pointers
… > +++ b/parsing_c/pretty_print_c.ml > @@ -804,11 +804,12 @@ and pp_string_format (e,ii) = > (FunctionType (return=void, params=int i) *) >(*WRONG I THINK, use left & right function *) >(* bug: pp_type_with_ident_rest None t; print_ident ident *) > + pp_type_left t; >pr_elem i; >iiqu +> List.iter pr_elem; (* le const est forcement apres le '*' > *) >if iiqu <> [] || get_comments_after i <> [] >then pr_space(); > - pp_type_with_ident_rest ident t attrs Ast_c.noattr; > + print_ident ident > >(* ugly special case ... todo? maybe sufficient in practice *) >| (ParenType ttop, [i1;i2]) -> Are the source code comments still appropriate for this patch hunk? Regards, Markus ___ Cocci mailing list Cocci@systeme.lip6.fr https://systeme.lip6.fr/mailman/listinfo/cocci
Re: [Cocci] [PATCH 0/2] Align the C AST and Cocci AST for pointers
> This series is to address the type matching problem in Coccinelle. How do you think about the wording “This series addresses a type matching problem in the Coccinelle software.”? Regards, Markus ___ Cocci mailing list Cocci@systeme.lip6.fr https://systeme.lip6.fr/mailman/listinfo/cocci
Re: [Cocci] Usage concerns around the SmPL construct “<+... … ...+>”
> Coccinelle could provide an appropriate running time by just ignoring the > <+... ...+>. But that seems like a pointless optimization, when the user > could just not put the <+... ...+> in the first place. I would like to express a specific SmPL functionality. > Coccinelle already applies a rule everywhere that it occurs, This aspect is clear to some degree. > regardless of whether the two occurrences are in the same function. Our views can be different also for the discussed use case. * How much will the requirement for changing selected items multiple times matter? * Are we eventually looking for another clarification also in the software documentation? Regards, Markus ___ Cocci mailing list Cocci@systeme.lip6.fr https://systeme.lip6.fr/mailman/listinfo/cocci
Re: [Cocci] Replacing a std::string check by strcmp() with SmPL
> Perhaps, but it will unnecessarily raise the running time of your semantic > patch. We come along different expectations again for the possible application of the construct “<+... … ...+>” for the semantic patch language. Should the concrete run time characteristics be clarified any more for current update candidates in known source files? Regards, Markus ___ Cocci mailing list Cocci@systeme.lip6.fr https://systeme.lip6.fr/mailman/listinfo/cocci
Re: [Cocci] Replacing a std::string check by strcmp() with SmPL
>> @replacement@ >> constant c; >> identifier text; >> statement is, es; >> @@ >> <+... > > What is thhe point of the outer <+... ...+>? I would like to stress the possibility that source code adjustments can be performed multiple times. https://github.com/coccinelle/coccinelle/blob/a549b9f0a20e14fe9c36f45990b40dc5708ef8f2/docs/manual/cocci_syntax.tex#L696 Regards, Markus ___ Cocci mailing list Cocci@systeme.lip6.fr https://systeme.lip6.fr/mailman/listinfo/cocci
