Re: [Cocci] [PATCH 4/4] coccicheck: add indexing enhancement options
On Mon, 13 Jun 2016, Luis R. Rodriguez wrote: > On Mon, Jun 13, 2016 at 09:48:30PM +0200, Julia Lawall wrote: > > > > > > On Mon, 13 Jun 2016, Wolfram Sang wrote: > > > > > > > > > Is there another scripts/coccinelle/ file I can use to test against to > > > > demo > > > > against glimpse/idutils/gitgrep best? > > > > > > I'd think this one may be a candidate: > > > > > > scripts/coccinelle/misc/irqf_oneshot.cocci > > > > > > Not too many, but quite some matches over the tree. > > > > Seems like a reasonable choice. > > With this one on a 32-core system, I get: > > glimpse: > real0m6.549s > user0m49.136s > sys 0m3.076s > > idutils: > real0m6.749s > user0m51.936s > sys 0m3.876s > > gitgrep: > real0m6.805s > user0m51.572s > sys 0m4.432s > > coccigrep: > real0m16.369s > user0m58.712s > sys 0m5.064s > > I redirected stderr to stdout, and verified glimpse output has: > > glimpse request = request_threaded_irq > > Does this match expectations? Yes. I'm not sure that gitgrep would work as well when there are multiple keywords. It may descend to coccigrep. julia
Re: [Cocci] [PATCH 4/4] coccicheck: add indexing enhancement options
On Mon, 13 Jun 2016, Luis R. Rodriguez wrote: > On Mon, Jun 13, 2016 at 09:48:30PM +0200, Julia Lawall wrote: > > > > > > On Mon, 13 Jun 2016, Wolfram Sang wrote: > > > > > > > > > Is there another scripts/coccinelle/ file I can use to test against to > > > > demo > > > > against glimpse/idutils/gitgrep best? > > > > > > I'd think this one may be a candidate: > > > > > > scripts/coccinelle/misc/irqf_oneshot.cocci > > > > > > Not too many, but quite some matches over the tree. > > > > Seems like a reasonable choice. > > With this one on a 32-core system, I get: > > glimpse: > real0m6.549s > user0m49.136s > sys 0m3.076s > > idutils: > real0m6.749s > user0m51.936s > sys 0m3.876s > > gitgrep: > real0m6.805s > user0m51.572s > sys 0m4.432s > > coccigrep: > real0m16.369s > user0m58.712s > sys 0m5.064s > > I redirected stderr to stdout, and verified glimpse output has: > > glimpse request = request_threaded_irq > > Does this match expectations? Yes. I'm not sure that gitgrep would work as well when there are multiple keywords. It may descend to coccigrep. julia
Re: [Cocci] [PATCH 4/4] coccicheck: add indexing enhancement options
On Mon, Jun 13, 2016 at 09:48:30PM +0200, Julia Lawall wrote: > > > On Mon, 13 Jun 2016, Wolfram Sang wrote: > > > > > > Is there another scripts/coccinelle/ file I can use to test against to > > > demo > > > against glimpse/idutils/gitgrep best? > > > > I'd think this one may be a candidate: > > > > scripts/coccinelle/misc/irqf_oneshot.cocci > > > > Not too many, but quite some matches over the tree. > > Seems like a reasonable choice. With this one on a 32-core system, I get: glimpse: real0m6.549s user0m49.136s sys 0m3.076s idutils: real0m6.749s user0m51.936s sys 0m3.876s gitgrep: real0m6.805s user0m51.572s sys 0m4.432s coccigrep: real0m16.369s user0m58.712s sys 0m5.064s I redirected stderr to stdout, and verified glimpse output has: glimpse request = request_threaded_irq Does this match expectations? Luis
Re: [Cocci] [PATCH 4/4] coccicheck: add indexing enhancement options
On Mon, Jun 13, 2016 at 09:48:30PM +0200, Julia Lawall wrote: > > > On Mon, 13 Jun 2016, Wolfram Sang wrote: > > > > > > Is there another scripts/coccinelle/ file I can use to test against to > > > demo > > > against glimpse/idutils/gitgrep best? > > > > I'd think this one may be a candidate: > > > > scripts/coccinelle/misc/irqf_oneshot.cocci > > > > Not too many, but quite some matches over the tree. > > Seems like a reasonable choice. With this one on a 32-core system, I get: glimpse: real0m6.549s user0m49.136s sys 0m3.076s idutils: real0m6.749s user0m51.936s sys 0m3.876s gitgrep: real0m6.805s user0m51.572s sys 0m4.432s coccigrep: real0m16.369s user0m58.712s sys 0m5.064s I redirected stderr to stdout, and verified glimpse output has: glimpse request = request_threaded_irq Does this match expectations? Luis
Re: [Cocci] [PATCH 4/4] coccicheck: add indexing enhancement options
On Mon, 13 Jun 2016, Wolfram Sang wrote: > > > Is there another scripts/coccinelle/ file I can use to test against to demo > > against glimpse/idutils/gitgrep best? > > I'd think this one may be a candidate: > > scripts/coccinelle/misc/irqf_oneshot.cocci > > Not too many, but quite some matches over the tree. Seems like a reasonable choice. julia
Re: [Cocci] [PATCH 4/4] coccicheck: add indexing enhancement options
On Mon, 13 Jun 2016, Wolfram Sang wrote: > > > Is there another scripts/coccinelle/ file I can use to test against to demo > > against glimpse/idutils/gitgrep best? > > I'd think this one may be a candidate: > > scripts/coccinelle/misc/irqf_oneshot.cocci > > Not too many, but quite some matches over the tree. Seems like a reasonable choice. julia
Re: [Cocci] [PATCH 4/4] coccicheck: add indexing enhancement options
> Is there another scripts/coccinelle/ file I can use to test against to demo > against glimpse/idutils/gitgrep best? I'd think this one may be a candidate: scripts/coccinelle/misc/irqf_oneshot.cocci Not too many, but quite some matches over the tree. signature.asc Description: PGP signature
Re: [Cocci] [PATCH 4/4] coccicheck: add indexing enhancement options
> Is there another scripts/coccinelle/ file I can use to test against to demo > against glimpse/idutils/gitgrep best? I'd think this one may be a candidate: scripts/coccinelle/misc/irqf_oneshot.cocci Not too many, but quite some matches over the tree. signature.asc Description: PGP signature
Re: [Cocci] [PATCH 4/4] coccicheck: add indexing enhancement options
On Sat, Jun 11, 2016 at 07:24:51AM +0200, Julia Lawall wrote: > > > On Fri, 10 Jun 2016, Wolfram Sang wrote: > > > > AFAICT coccinelle does not have integration support for id-utils though. > > > > I used it just today ;) -- "--use-idutils ./ID" > > > > ID was generated with simple 'mkid -s'. > > Coccinelle includes a script scripts/idutils_index.sh > > This does mkid -i C --output .id-utils.index * I'll add support for detecting both. An issue of course is if any of these indexes grows stale. So I'll advise against these and recommend gitgrep unless the user has a hook to update index regularly. Luis
Re: [Cocci] [PATCH 4/4] coccicheck: add indexing enhancement options
On Sat, Jun 11, 2016 at 07:24:51AM +0200, Julia Lawall wrote: > > > On Fri, 10 Jun 2016, Wolfram Sang wrote: > > > > AFAICT coccinelle does not have integration support for id-utils though. > > > > I used it just today ;) -- "--use-idutils ./ID" > > > > ID was generated with simple 'mkid -s'. > > Coccinelle includes a script scripts/idutils_index.sh > > This does mkid -i C --output .id-utils.index * I'll add support for detecting both. An issue of course is if any of these indexes grows stale. So I'll advise against these and recommend gitgrep unless the user has a hook to update index regularly. Luis
Re: [Cocci] [PATCH 4/4] coccicheck: add indexing enhancement options
On Sat, Jun 11, 2016 at 07:54:39AM +0200, Julia Lawall wrote: > > > On Sat, 11 Jun 2016, Wolfram Sang wrote: > > > > > > real16m11.692s > > > user127m50.388s > > > sys 0m2.168s > > > > That's better but not a magnitude, I wonder. > > I think that it is because the filtering that Coccinelle does already > works pretty well, and there are quite a lot of files (7514) that contain > kfree. Is there another scripts/coccinelle/ file I can use to test against to demo against glimpse/idutils/gitgrep best? Luis
Re: [Cocci] [PATCH 4/4] coccicheck: add indexing enhancement options
On Sat, Jun 11, 2016 at 07:54:39AM +0200, Julia Lawall wrote: > > > On Sat, 11 Jun 2016, Wolfram Sang wrote: > > > > > > real16m11.692s > > > user127m50.388s > > > sys 0m2.168s > > > > That's better but not a magnitude, I wonder. > > I think that it is because the filtering that Coccinelle does already > works pretty well, and there are quite a lot of files (7514) that contain > kfree. Is there another scripts/coccinelle/ file I can use to test against to demo against glimpse/idutils/gitgrep best? Luis
Re: [Cocci] [PATCH 4/4] coccicheck: add indexing enhancement options
> works pretty well, and there are quite a lot of files (7514) that contain > kfree. Ah, kfree. That explains, I missed that info. signature.asc Description: PGP signature
Re: [Cocci] [PATCH 4/4] coccicheck: add indexing enhancement options
> works pretty well, and there are quite a lot of files (7514) that contain > kfree. Ah, kfree. That explains, I missed that info. signature.asc Description: PGP signature
Re: [Cocci] [PATCH 4/4] coccicheck: add indexing enhancement options
On Sat, 11 Jun 2016, Wolfram Sang wrote: > > > It's not as efficient as glimpse because the query language is simpler. > > Interesting, what is missing compared to glimpse? Glimpse allows queries that are arbitrary formulas, up to a limited level of complexity, involving both && and ||. For idutils, Coccinelle runs lid on each of the tokens in the formula, and then does unions and intersections on the result. julia > > So more filtering has to be done at the ocaml level. But it's probably > > fine in most cases. > > For me, it has two advantages over glimpse: > > a) it is in the debian package repository > b) the same database can be used with the code browser 'seascope' >which can do nice things by feeding ctags on the fly with data >from id-utils. > > Mileages vary, of course, just wanted to mention it to give pointers. > >
Re: [Cocci] [PATCH 4/4] coccicheck: add indexing enhancement options
On Sat, 11 Jun 2016, Wolfram Sang wrote: > > > It's not as efficient as glimpse because the query language is simpler. > > Interesting, what is missing compared to glimpse? Glimpse allows queries that are arbitrary formulas, up to a limited level of complexity, involving both && and ||. For idutils, Coccinelle runs lid on each of the tokens in the formula, and then does unions and intersections on the result. julia > > So more filtering has to be done at the ocaml level. But it's probably > > fine in most cases. > > For me, it has two advantages over glimpse: > > a) it is in the debian package repository > b) the same database can be used with the code browser 'seascope' >which can do nice things by feeding ctags on the fly with data >from id-utils. > > Mileages vary, of course, just wanted to mention it to give pointers. > >
Re: [Cocci] [PATCH 4/4] coccicheck: add indexing enhancement options
> It's not as efficient as glimpse because the query language is simpler. Interesting, what is missing compared to glimpse? > So more filtering has to be done at the ocaml level. But it's probably > fine in most cases. For me, it has two advantages over glimpse: a) it is in the debian package repository b) the same database can be used with the code browser 'seascope' which can do nice things by feeding ctags on the fly with data from id-utils. Mileages vary, of course, just wanted to mention it to give pointers. signature.asc Description: PGP signature
Re: [Cocci] [PATCH 4/4] coccicheck: add indexing enhancement options
> It's not as efficient as glimpse because the query language is simpler. Interesting, what is missing compared to glimpse? > So more filtering has to be done at the ocaml level. But it's probably > fine in most cases. For me, it has two advantages over glimpse: a) it is in the debian package repository b) the same database can be used with the code browser 'seascope' which can do nice things by feeding ctags on the fly with data from id-utils. Mileages vary, of course, just wanted to mention it to give pointers. signature.asc Description: PGP signature
Re: [Cocci] [PATCH 4/4] coccicheck: add indexing enhancement options
> in practice though this seems to not perform better than > regular grep however its expected to help with some use cases > so we use that if you have no other indexing options in place > available. Would you like to fix a typo in this paragraph? Regards, Markus
Re: [Cocci] [PATCH 4/4] coccicheck: add indexing enhancement options
> in practice though this seems to not perform better than > regular grep however its expected to help with some use cases > so we use that if you have no other indexing options in place > available. Would you like to fix a typo in this paragraph? Regards, Markus
Re: [Cocci] [PATCH 4/4] coccicheck: add indexing enhancement options
On Sat, 11 Jun 2016, Wolfram Sang wrote: > > > real16m11.692s > > user127m50.388s > > sys 0m2.168s > > That's better but not a magnitude, I wonder. I think that it is because the filtering that Coccinelle does already works pretty well, and there are quite a lot of files (7514) that contain kfree. julia
Re: [Cocci] [PATCH 4/4] coccicheck: add indexing enhancement options
On Sat, 11 Jun 2016, Wolfram Sang wrote: > > > real16m11.692s > > user127m50.388s > > sys 0m2.168s > > That's better but not a magnitude, I wonder. I think that it is because the filtering that Coccinelle does already works pretty well, and there are quite a lot of files (7514) that contain kfree. julia
Re: [Cocci] [PATCH 4/4] coccicheck: add indexing enhancement options
> real16m11.692s > user127m50.388s > sys 0m2.168s That's better but not a magnitude, I wonder. signature.asc Description: PGP signature
Re: [Cocci] [PATCH 4/4] coccicheck: add indexing enhancement options
> real16m11.692s > user127m50.388s > sys 0m2.168s That's better but not a magnitude, I wonder. signature.asc Description: PGP signature
Re: [Cocci] [PATCH 4/4] coccicheck: add indexing enhancement options
On Fri, 10 Jun 2016, Wolfram Sang wrote: > > AFAICT coccinelle does not have integration support for id-utils though. > > I used it just today ;) -- "--use-idutils ./ID" > > ID was generated with simple 'mkid -s'. Coccinelle includes a script scripts/idutils_index.sh This does mkid -i C --output .id-utils.index * julia
Re: [Cocci] [PATCH 4/4] coccicheck: add indexing enhancement options
On Fri, 10 Jun 2016, Wolfram Sang wrote: > > AFAICT coccinelle does not have integration support for id-utils though. > > I used it just today ;) -- "--use-idutils ./ID" > > ID was generated with simple 'mkid -s'. Coccinelle includes a script scripts/idutils_index.sh This does mkid -i C --output .id-utils.index * julia
Re: [Cocci] [PATCH 4/4] coccicheck: add indexing enhancement options
On Sat, 11 Jun 2016, Luis R. Rodriguez wrote: > On Fri, Jun 10, 2016 at 11:51:26PM +0200, Wolfram Sang wrote: > > > AFAICT coccinelle does not have integration support for id-utils though. > > > > I used it just today ;) -- "--use-idutils ./ID" > > > > ID was generated with simple 'mkid -s'. > > > > Sweet, testing that now. It's not as efficient as glimpse because the query language is simpler. So more filtering has to be done at the ocaml level. But it's probably fine in most cases. julia
Re: [Cocci] [PATCH 4/4] coccicheck: add indexing enhancement options
On Sat, 11 Jun 2016, Luis R. Rodriguez wrote: > On Fri, Jun 10, 2016 at 11:51:26PM +0200, Wolfram Sang wrote: > > > AFAICT coccinelle does not have integration support for id-utils though. > > > > I used it just today ;) -- "--use-idutils ./ID" > > > > ID was generated with simple 'mkid -s'. > > > > Sweet, testing that now. It's not as efficient as glimpse because the query language is simpler. So more filtering has to be done at the ocaml level. But it's probably fine in most cases. julia
Re: [Cocci] [PATCH 4/4] coccicheck: add indexing enhancement options
On Fri, 10 Jun 2016, Luis R. Rodriguez wrote: > On Fri, Jun 10, 2016 at 11:43:57PM +0200, Wolfram Sang wrote: > > > > Well, slightly better. > > > > > > No, it should be much better. You would have to look at the standard > > > > I use id-utils regularly and it is indeed at least a magnitude better. > > The indexing often pays off already with the first coccinelle run for > > me. Highly recommended. > > AFAICT coccinelle does not have integration support for id-utils though. Yes it does, for years. julia
Re: [Cocci] [PATCH 4/4] coccicheck: add indexing enhancement options
On Fri, 10 Jun 2016, Luis R. Rodriguez wrote: > On Fri, Jun 10, 2016 at 11:43:57PM +0200, Wolfram Sang wrote: > > > > Well, slightly better. > > > > > > No, it should be much better. You would have to look at the standard > > > > I use id-utils regularly and it is indeed at least a magnitude better. > > The indexing often pays off already with the first coccinelle run for > > me. Highly recommended. > > AFAICT coccinelle does not have integration support for id-utils though. Yes it does, for years. julia
Re: [Cocci] [PATCH 4/4] coccicheck: add indexing enhancement options
On Sat, Jun 11, 2016 at 12:08:32AM +0200, Luis R. Rodriguez wrote: > On Fri, Jun 10, 2016 at 11:51:26PM +0200, Wolfram Sang wrote: > > > AFAICT coccinelle does not have integration support for id-utils though. > > > > I used it just today ;) -- "--use-idutils ./ID" > > > > ID was generated with simple 'mkid -s'. > > > > Sweet, testing that now. Boooyah :D real16m11.692s user127m50.388s sys 0m2.168s Luis
Re: [Cocci] [PATCH 4/4] coccicheck: add indexing enhancement options
On Sat, Jun 11, 2016 at 12:08:32AM +0200, Luis R. Rodriguez wrote: > On Fri, Jun 10, 2016 at 11:51:26PM +0200, Wolfram Sang wrote: > > > AFAICT coccinelle does not have integration support for id-utils though. > > > > I used it just today ;) -- "--use-idutils ./ID" > > > > ID was generated with simple 'mkid -s'. > > > > Sweet, testing that now. Boooyah :D real16m11.692s user127m50.388s sys 0m2.168s Luis
Re: [Cocci] [PATCH 4/4] coccicheck: add indexing enhancement options
On Fri, Jun 10, 2016 at 11:51:26PM +0200, Wolfram Sang wrote: > > AFAICT coccinelle does not have integration support for id-utils though. > > I used it just today ;) -- "--use-idutils ./ID" > > ID was generated with simple 'mkid -s'. > Sweet, testing that now. Luis
Re: [Cocci] [PATCH 4/4] coccicheck: add indexing enhancement options
On Fri, Jun 10, 2016 at 11:51:26PM +0200, Wolfram Sang wrote: > > AFAICT coccinelle does not have integration support for id-utils though. > > I used it just today ;) -- "--use-idutils ./ID" > > ID was generated with simple 'mkid -s'. > Sweet, testing that now. Luis
Re: [Cocci] [PATCH 4/4] coccicheck: add indexing enhancement options
> AFAICT coccinelle does not have integration support for id-utils though. I used it just today ;) -- "--use-idutils ./ID" ID was generated with simple 'mkid -s'. signature.asc Description: PGP signature
Re: [Cocci] [PATCH 4/4] coccicheck: add indexing enhancement options
> AFAICT coccinelle does not have integration support for id-utils though. I used it just today ;) -- "--use-idutils ./ID" ID was generated with simple 'mkid -s'. signature.asc Description: PGP signature
Re: [Cocci] [PATCH 4/4] coccicheck: add indexing enhancement options
On Fri, Jun 10, 2016 at 11:43:57PM +0200, Wolfram Sang wrote: > > > Well, slightly better. > > > > No, it should be much better. You would have to look at the standard > > I use id-utils regularly and it is indeed at least a magnitude better. > The indexing often pays off already with the first coccinelle run for > me. Highly recommended. AFAICT coccinelle does not have integration support for id-utils though. Luis
Re: [Cocci] [PATCH 4/4] coccicheck: add indexing enhancement options
On Fri, Jun 10, 2016 at 11:43:57PM +0200, Wolfram Sang wrote: > > > Well, slightly better. > > > > No, it should be much better. You would have to look at the standard > > I use id-utils regularly and it is indeed at least a magnitude better. > The indexing often pays off already with the first coccinelle run for > me. Highly recommended. AFAICT coccinelle does not have integration support for id-utils though. Luis
Re: [Cocci] [PATCH 4/4] coccicheck: add indexing enhancement options
> > Well, slightly better. > > No, it should be much better. You would have to look at the standard I use id-utils regularly and it is indeed at least a magnitude better. The indexing often pays off already with the first coccinelle run for me. Highly recommended. signature.asc Description: PGP signature
Re: [Cocci] [PATCH 4/4] coccicheck: add indexing enhancement options
> > Well, slightly better. > > No, it should be much better. You would have to look at the standard I use id-utils regularly and it is indeed at least a magnitude better. The indexing often pays off already with the first coccinelle run for me. Highly recommended. signature.asc Description: PGP signature
