Hello Philippe !
I would rather lean towards re-using the expert settings for the basic
> benchmarks, and to provide some "reset" button, so that if one messes
> things up he could still retrieve the original basic results.
Yea I was planning on adding a "reset" button. But, I don't see the point
in having a basic mode that does the same thing as the expert mode. All the
info in basic mode will still be visible in expert mode. So why not have
independent modes as an additional functionality?
It's actually important to have finer grained data for small vectors, and
> more spaced points as the data grows bigger : this is why it is better to
> choose the sizes according to a a^x law than an a*x one. You can experiment
> other values than 2 for a, if you want. If I were you, I'd probably go with
> something like :
[int(1.5**x) for x in range(30,45)]
> That is, an increment 1.5 factor from ~190,000 to ~55,000,000
Alright, I'll play around with the numbers and see what goes best.
Blas3 multiplies two matrices : A(size1, size2) * B(size2, size3), hence
> the three sizes required :-p Not sure about what kind of 3D matrices you
> are referring to! ;)
Bah, my bad. I thought Karl was talking about a third dimension. Thanks for
the clarification.
Regards, Namik
On Tue, Aug 12, 2014 at 4:05 AM, Philippe Tillet <phil.til...@gmail.com>
wrote:
> Hello !
>
> This all looks pretty good. Good job!
>
>
> 2014-08-12 3:40 GMT+02:00 Namik Karovic <namik.karo...@gmail.com>:
>
> Hi Karl,
>>
>>
>> I'm fine with splitting things into something like "Basic Benchmark" and
>>> "Expert Benchmark" ('view' sounds inappropriate), but as long as both
>>> benchmark do the same thing, I don't see the problem why the expert version
>>> cannot be a refinement of the basic version. Could you please elaborate?
>>
>>
>> The entire issue comes down to this: should basic mode be able to run the
>> benchmark with expert mode's settings? Or should it always run using the
>> default settings, no matter what.
>>
>> My motivation for bringing this up is that one could first do a basic
>> benchmark, then continue on to playing with the expert mode. The basic mode
>> can then be used for quick reference, as it will be not be altered by
>> expert mode runs.
>>
>> So both modes will have their own results, and their own settings. I will
>> prevent users from running both modes at the same time, of course.
>>
>> I hope it's clearer now.
>>
>>
> I would rather lean towards re-using the expert settings for the basic
> benchmarks, and to provide some "reset" button, so that if one messes
> things up he could still retrieve the original basic results.
>
>
>>
>>> Looks great, this is a really useful graph (something is fishy with the
>>> values on the y-axis, though...) :-) Can you please draw the x- and the
>>> y-axis in logarithmic scale and make the vector increment a multiplicative
>>> factor (2 by default)?
>>
>>
>> The axis labels are fishy because they aren't properly set up yet :)
>> Sure, I can make em logarithmic. What about the default number of
>> increments? I got it currently set to increment by 1 million from 1M to
>> 15M, so 14 increments. Should there be more increment steps? I need to know
>> so I can calculate the optimum min and max vector size for x2 factor
>> increment.
>>
>
> It's actually important to have finer grained data for small vectors, and
> more spaced points as the data grows bigger : this is why it is better to
> choose the sizes according to a a^x law than an a*x one. You can experiment
> other values than 2 for a, if you want. If I were you, I'd probably go with
> something like :
> [int(1.5**x) for x in range(30,45)]
>
> That is, an increment 1.5 factor from ~190,000 to ~55,000,000
>
>
>>
>> This looks quite okay, actually.
>>
>>
>> Alright, if you say so. But note that in fullscreen it will be a lot more
>> stretched, and thus a lot less visually appealing. I'll do some more
>> thinking to try and make it a bit more organized.
>>
>>
>> There should be a third size for Blas3 part. This will then also make all
>>> four boxes (Blas3, Sparse, Copy, Vector) equally high, which should improve
>>> the visual appearance.
>>
>>
>> So x,y,z dimensions for Blas3? Blas3 currently uses 2D matrices, so I'll
>> have to modify the benchmark to use 3D matrices?
>>
>>
> Blas3 multiplies two matrices : A(size1, size2) * B(size2, size3), hence
> the three sizes required :-p Not sure about what kind of 3D matrices you
> are referring to! ;)
>
> In any case, great job!
> Philippe
>
>
>> I don't see a problem with making the string conversion routines
>>> public, so I just pushed a commit for doing so. :-)
>>
>>
>> Thanks. Appreciate it.
>>
>>
>> Regards, Namik
>>
>>
>> On Mon, Aug 11, 2014 at 8:18 PM, Karl Rupp <r...@iue.tuwien.ac.at> wrote:
>>
>>> Hi Namik,
>>>
>>>
>>> > I'm starting work on the expert view and would appreciate some feedback
>>>
>>>> before I get into it more seriously.
>>>>
>>>
>>> thanks for the latest bunch of features :-)
>>>
>>>
>>>
>>> I've got quite a lot of questions, so bear with me please. Here we go:
>>>>
>>>> -Should basic and expert views be changed to independent benchmarking
>>>> modes, or remain different views of the same banchmark backend? I
>>>> initially imagined basic & expert views as differently detailed
>>>> "presentations" of the same benchmark instance (one could run the basic
>>>> benchmark, and switch to expert view after it's done to examine the
>>>> results in more detail).
>>>>
>>>> However, now I'm thinking it would be better not to mix them. Let basic
>>>> mode be a simple benchmark with default settings, and let expert be
>>>> fully customizable & independent. That way the basic mode would be
>>>> unaffected by expert mode's settings. This would allow basic mode to act
>>>> as a "safe reference" mode. It would also allow easier usage of
>>>> benchmark profiles (saving user's expert mode config info for later
>>>> usage), but that's a story for another time.
>>>>
>>>> It's worth mentioning that it's easier to implement two independent
>>>> modes than to have them share a single benchmark mode.
>>>>
>>>> So, which version am I to develop?
>>>>
>>>
>>> I'm fine with splitting things into something like "Basic Benchmark" and
>>> "Expert Benchmark" ('view' sounds inappropriate), but as long as both
>>> benchmark do the same thing, I don't see the problem why the expert version
>>> cannot be a refinement of the basic version. Could you please elaborate?
>>>
>>>
>>>
>>> -I've implemented line plotting of copy & vector benchmarks. There's
>>>> still some minor tweaks to be done, but the main functionality is ready.
>>>> Here's a screenshot for quick reference:
>>>> http://pokit.org/get/?f4594ac1c0e771184e36ada937772706.jpg
>>>>
>>>> The axis labels need to be properly named, legend needs to be placed in
>>>> a less obtrusive place, graphs selectable through the legend and a few
>>>> more tweaks.
>>>>
>>>> Comments, suggestions & critiques are welcome.
>>>>
>>>
>>> Looks great, this is a really useful graph (something is fishy with the
>>> values on the y-axis, though...) :-) Can you please draw the x- and the
>>> y-axis in logarithmic scale and make the vector increment a multiplicative
>>> factor (2 by default)?
>>>
>>>
>>>
>>> -The expert tab was getting kinda cluttered. After reducing minimum
>>>> supported resolution to 1000x700 there was very little room left for all
>>>> the input widgets. So I decided to remove the plot where final results
>>>> were drawn. I didn't think this plot was very useful in expert mode,
>>>> since it shows only one number per benchmark. However, removing it
>>>> created too much space and now the expert tab looks too stretched x)
>>>>
>>>> I need some constructive suggestions on this matter. Here's a screenshot
>>>> so you know what I'm talking about:
>>>> http://pokit.org/get/?3c293d1ff6a2bdce38af4c5a226ee3fb.jpg
>>>>
>>>
>>> This looks quite okay, actually.
>>>
>>>
>>>
>>> Also, did I miss any settings in the benchmark config part? I know there
>>>> should be a "number of benchmark runs" option, but don't know if it
>>>> should be global or benchmark-specific.
>>>>
>>>
>>> There should be a third size for Blas3 part. This will then also make
>>> all four boxes (Blas3, Sparse, Copy, Vector) equally high, which should
>>> improve the visual appearance.
>>>
>>>
>>>
>>> Are there any additional benchmark details that need to be shown here?
>>>>
>>>
>>> Not yet, I'd say.
>>>
>>>
>>>
>>> -Displayed system info in the "system info" screen is incomplete due to
>>>> several functions in viennacl::ocl::device being private. I wanted to
>>>> avoid parsing that huge string returned by full_info() function, so I
>>>> decided to re-implement a customized version of it. Unfortunately, I
>>>> couldn't re-implement it fully, since functions like:
>>>> local_mem_type(); platform(); device_type_to_string(iter->type()); are
>>>> private.
>>>>
>>>
>>> I don't see a problem with making the string conversion routines public,
>>> so I just pushed a commit for doing so. :-)
>>>
>>> Thanks and best regards,
>>> Karli
>>>
>>>
>>
>>
>> ------------------------------------------------------------------------------
>>
>> _______________________________________________
>> ViennaCL-devel mailing list
>> ViennaCL-devel@lists.sourceforge.net
>> https://lists.sourceforge.net/lists/listinfo/viennacl-devel
>>
>>
>
------------------------------------------------------------------------------
_______________________________________________
ViennaCL-devel mailing list
ViennaCL-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/viennacl-devel
