Hey Marc,

Wow, that was fast! this improves performance by 5-6x. I think this is now 
feasible for the first version and I can start working on the port. I'll 
keep you posted once I have the first version ready.

Also, can you push this api to a release version?

Much appreciated!
Yevgeny

On Friday, November 8, 2019 at 5:02:42 PM UTC+2, Marc R. Hoffmann wrote:
>
> Hi Yevgeny,
>
> that sounds like an interesting project! If you publish it let us know, we 
> can add your project to our integration matrix.
>
> I added an experimental API for you that just counts the number of hits 
> (no overhead for serializing the data):
>
> https://github.com/jacoco/jacoco/tree/experimental-hit-count-api
>
> Can you please tell us, whether this improves your performance?
>
> Cheers,
> -marc
>
>
> On 8. Nov 2019, at 09:14, Yevgeny P <yev...@gmail.com <javascript:>> 
> wrote:
>
> Hey Evgeny,
>
> Thanks you for the detailed response, and sorry if I offended JaCoCo 
> performance:) I didn't mean that.
>
> It is totally possible that I'm getting something wrong (I'm far from 
> being a Java expert). I'll share both the use-case and the benchmark code.
>
> I'm porting my jsfuzz <https://github.com/fuzzitdev/jsfuzz> library to a 
> new javafuzz (coverage guided fuzzer) library where I want to use JaCoCo as 
> the coverage library.
>
> Essentially for the basic first version of javafuzz I only need the total 
> hit count and I don't need to reset the counters so that should help save 
> some time.
>
> Currently I'm getting 930 execs/s (I had a mistake in my early benchmarks) 
> for both calling getExecutionData and getting the total hitCount.
>
> Maybe it is possible to speed it up both with the reflection tricks + I 
> can add a Pull-Request for JaCoCo to save the total number of hitCounts. 
> Just as reference the "same logic" in jsfuzz and pythonfuzz can get to 
> about 20,000 execs/s (inclduing the overhead of the fuzzer logic itself 
> which is currently not in the benchmark).  
>
> here is a link <https://github.com/yevgenypats/jacoco-test> to the 
> updated repo with the benchmarks.
>
> Much appreciated,
> Also Yevgeny:)
>
>
> On Friday, November 8, 2019 at 1:01:37 AM UTC+2, Evgeny Mandrikov wrote:
>>
>>
>> On Thursday, November 7, 2019 at 8:06:55 AM UTC+1, Yevgeny P wrote: 
>>>
>>> the call to getExecutionData(0.5s)
>>>
>>
>> If you use reflective call that was shown by me as is, without usual 
>> speedup tricks for reflective calls, then I'm afraid that you're measuring 
>> reflection overhead and not at all performance of this method in JaCoCo.
>> This method boils down mostly to just simple traversal of an entries in a 
>> Map.
>>
>>  
>>
>>> and parsing the data(0.5s).
>>>
>>
>> I'm not sure what you mean by "parsing the data", because there is 
>> nothing to parse in this byte array - for further analysis it is used 
>> mostly as it is, undergoing only simple bytes into booleans decompression - 
>> https://github.com/jacoco/jacoco/blob/v0.8.5/org.jacoco.core/src/org/jacoco/core/data/ExecutionDataReader.java
>>
>> If you mean actual analysis, then AFAIR on example of JDK rt.jar classes 
>> JaCoCo is capable of analysing at speed ~20 Mb of class files per second on 
>> even non-warmed JVM, which is thousands of classes and which is IMO pretty 
>> good given HDD and OS data access overheads, which scales linearly with 
>> amount of classes, and which is much faster than execution of all tests for 
>> these classes.
>>
>>
>> But the performance is pretty slow
>>>
>>
>> Of course if you do both operations in a very tight loop on a big amount 
>> of classes with just a tiny bit of something else, then of course compared 
>> to this something else, they will look slow. But IMO this is incredibly 
>> suspicious use-case and even benchmark to look correct. 
>>
>>
>> Is there anything to do with that or I would need to reimplement the 
>>> instrumentation myself if I would like this to be faster?
>>
>>
>> Instrumentation is not involved in a call of first method and not 
>> performed at a time of analysis of its data. 
>>
>> All in all IMO when speaking about performance, one need to be extremely 
>> careful/pedantic and to avoid bold statements about poor performance must 
>> make sure that benchmark/measurement/use-case is relevant and makes sense, 
>> and must describe not only the use-case, but also the way of measurement 
>> and setup/environment.
>>
>> To add few examples of why this matters: How far your measurement from 
>> the time to read same class files from disk? to parse them using ASM 
>> library? If not far, then you're actually not measuring performance of 
>> JaCoCo code ;) and etc.
>>
>>
>> Regards,
>> Evgeny
>>
>>
>>
>>
>>> On Wednesday, November 6, 2019 at 7:33:11 PM UTC+2, Yevgeny P wrote:
>>>>
>>>> Thank you, this works!!
>>>>
>>>> On Wednesday, November 6, 2019 at 7:15:27 PM UTC+2, Evgeny Mandrikov 
>>>> wrote:
>>>>>
>>>>> Hi,
>>>>>
>>>>> On Wednesday, November 6, 2019 at 8:26:43 AM UTC+1, Yevgeny P wrote:
>>>>>>
>>>>>> Hey Marc, Attached is a github repo with complete reproduction steps 
>>>>>> in the README https://github.com/yevgenypats/jacoco-test
>>>>>>
>>>>>
>>>>> AFAIK Maven plugins use child-first classloading strategy ( 
>>>>> https://maven.apache.org/guides/mini/guide-maven-classloading.html ),
>>>>> so the most reasonable explanation - is that you have JaCoCo classes 
>>>>> in classpath of plugin and therefore they are used instead of actual 
>>>>> classes of agent.
>>>>> As a proof - https://github.com/yevgenypats/jacoco-test/pull/1
>>>>>  
>>>>>
>>>>>>
>>>>>> Thanks again.
>>>>>>
>>>>>> On Wednesday, November 6, 2019 at 6:40:24 AM UTC+2, Marc R. Hoffmann 
>>>>>> wrote:
>>>>>>>
>>>>>>> Interesting. Can you please provide the full stack trace?
>>>>>>>
>>>>>>>
>>>>>>> On 5. Nov 2019, at 11:41, Yevgeny P <yev...@gmail.com> wrote:
>>>>>>>
>>>>>>> I think I got it, this goal is running before the test goal so it's 
>>>>>>> changes the JVM command-line. I have different problem now - I'm 
>>>>>>> running 
>>>>>>> with MVN_OPTS="-javaagent:<path>" this indeed generates the exec 
>>>>>>> coverage 
>>>>>>> file when the program exits but it I'm not able to use the API as I get:
>>>>>>>
>>>>>>> Caused by: java.lang.IllegalStateException: JaCoCo agent not 
>>>>>>> started. even though it is started as I get the coverage after the 
>>>>>>> program 
>>>>>>> exits.
>>>>>>>
>>>>>>> On Tuesday, November 5, 2019 at 8:37:07 AM UTC+2, yev...@fuzzit.dev 
>>>>>>> wrote:
>>>>>>>>
>>>>>>>> Thanks! I'll try that out.  Also Im trying to start the coverage in 
>>>>>>>> a maven plugin, kind like you did here: 
>>>>>>>> https://github.com/jacoco/jacoco/blob/master/jacoco-maven-plugin/src/org/jacoco/maven/AbstractAgentMojo.java
>>>>>>>>  .
>>>>>>>> But this doesn't work for also I don't understand how it works in 
>>>>>>>> jacoco as the JVM is already started or does it spawn another JVM per 
>>>>>>>> test 
>>>>>>>> and then it's start the agent like specified in the properties.
>>>>>>>>
>>>>>>>> On Tuesday, November 5, 2019 at 12:54:45 AM UTC+2, Marc R. Hoffmann 
>>>>>>>> wrote:
>>>>>>>>>
>>>>>>>>> Hi Yevgeny,
>>>>>>>>>
>>>>>>>>> it is possible to talk to the agent through its runtime API: 
>>>>>>>>> https://www.jacoco.org/jacoco/trunk/doc/api/org/jacoco/agent/rt/package-summary.html
>>>>>>>>>
>>>>>>>>> Regards,
>>>>>>>>> -marc
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> On 4. Nov 2019, at 21:22, Yevgeny P <yev...@gmail.com> wrote:
>>>>>>>>>
>>>>>>>>> Hey Team,
>>>>>>>>>
>>>>>>>>> Let's say I've started a Java program with Jacoco agent. Is it 
>>>>>>>>> possible to retrieve the coverage from within the Java program itself 
>>>>>>>>> via 
>>>>>>>>> somekind of API or it should be an external program?
>>>>>>>>> I only saw an example doing something similar for a single class 
>>>>>>>>> without Jacoco agent.
>>>>>>>>>
>>>>>>>>> Thanks,
>>>>>>>>> Yevgeny
>>>>>>>>>
>>>>>>>>> -- 
>>>>>>>>> You received this message because you are subscribed to the Google 
>>>>>>>>> Groups "JaCoCo and EclEmma Users" group.
>>>>>>>>> To unsubscribe from this group and stop receiving emails from it, 
>>>>>>>>> send an email to jac...@googlegroups.com.
>>>>>>>>> To view this discussion on the web visit 
>>>>>>>>> https://groups.google.com/d/msgid/jacoco/ab4e80bc-1c1e-4189-87dc-640d2ce5858a%40googlegroups.com
>>>>>>>>>  
>>>>>>>>> <https://groups.google.com/d/msgid/jacoco/ab4e80bc-1c1e-4189-87dc-640d2ce5858a%40googlegroups.com?utm_medium=email&utm_source=footer>
>>>>>>>>> .
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>> -- 
>>>>>>> You received this message because you are subscribed to the Google 
>>>>>>> Groups "JaCoCo and EclEmma Users" group.
>>>>>>> To unsubscribe from this group and stop receiving emails from it, 
>>>>>>> send an email to jac...@googlegroups.com.
>>>>>>> To view this discussion on the web visit 
>>>>>>> https://groups.google.com/d/msgid/jacoco/290d2d71-da0c-4085-ad5d-81842f2d33fe%40googlegroups.com
>>>>>>>  
>>>>>>> <https://groups.google.com/d/msgid/jacoco/290d2d71-da0c-4085-ad5d-81842f2d33fe%40googlegroups.com?utm_medium=email&utm_source=footer>
>>>>>>> .
>>>>>>>
>>>>>>>
>>>>>>>
> -- 
> You received this message because you are subscribed to the Google Groups 
> "JaCoCo and EclEmma Users" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to jac...@googlegroups.com <javascript:>.
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/jacoco/517ba116-ce05-4b3e-8a73-6d683d1b7be5%40googlegroups.com
>  
> <https://groups.google.com/d/msgid/jacoco/517ba116-ce05-4b3e-8a73-6d683d1b7be5%40googlegroups.com?utm_medium=email&utm_source=footer>
> .
>
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"JaCoCo and EclEmma Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to jacoco+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/jacoco/e23e17aa-fdd2-4446-b2b2-3e05f76c0313%40googlegroups.com.

Reply via email to