Hello,
with my second intended larger patch, I changed the filter logic so that
there are always all three ones intantiated. This is much easier to use.
Internaly a filter then does not allocate memory, if no games are
filtered out. So it also saves memory in the default case. I will create
a dedicated patch for this, as the whole patch does no longer make sense
if the gamelist widget is changed.
Sorting needs some megabaytes resources for a large base. So I think its
a good idea to reuse efforts. I remember that even the algorithm I used
does not enforce deterministic output, so why should I.
Today there are already two customers for sorting (best games and game
list). I guess there will be others. I see no alternative to handles for
controlling memory consumption. I try to make the sortcache class as
resistent as possible to prevent crashes, if clients call it with
invalid handles.
The new sorting is fast, but if you look for the current gamelist
window, e.g. scrolling, search game by name, it is still too slow for
5-Mega bases. But what can be done is to start with incremental sorting
when a user selects the columns to sort. Then the decision to sort the
whole base can be provided in the context menu and no sort widget is
necessary anymore. I like this idea, but users have to read help to
understand the benefits of full sorting.
sc_sort does allow to use incremental sorting. So I do not know why to
duplicate this functionality in sc_base.
So what needs to be done to move the gamelist window to your widget.
- Replace the old control by the new one.
- Implement the additional functionality (mark games, goto current, goto
game with number...) through an extended context menu of your widget
- Allow user to switch to a completely sorted base, to save and restore
sorting through the context menu.
- The widget should allow to put the scrollbar at the bottom
- Store properties in config
- Update help
Then switching the game list between bases can be done also through the
context menu and my second patch becomes obsolete.
I can implement bullet 2 and 3 and the switching of bases, if there are
volunteers for the other ones.
Gerd
Am 29.06.2011 4:03, schrieb Fulvio:
Gerd Lorscheid wrote:
Hello,
I had a look. sc_base_gameslist assumes that dbFilter and treeFilter are
always initialized. This is wrong with the actual code in git. If no tree is
open treeFilter may be NULL, if no header filter is defined then dbFilter
may be NULL. This leads to the crash described below reproducible on all
platforms.
Thanks, when you have time can you please confirm that the current git
code solve the issue?
The sort cache is used to reuse sorting where possible. So if one client
sorts X+Y-, a second client X+Z- and then a third X+ he may get back results
from either of the two first clients.
Thanks again, I didn't read carefully MatchCriteria and i wasn't aware
of that.
But is this correct?
It confuse me a little that sorting can give different results.
A simple way I used in GetRange to perform a stable sort (necessary
for scrolling bases not fully sorted) is to use the game number.
What's your opinion about that?
Do you think we can change the code to make HeapSort stable?
(And this means that X+ will not give the same results of X+Z-)
So it is insufficient to remember the
search criteria only to free memory later. The tcl code in gamelist
remembers that a sort cache is allocated. It has to remember the handle
also.
Considering that sc_base_gameslist don't use handles in order to use
the best sortcache available, the minimal tcl/c++ interface is:
1) Get a list of games ( sc_base gameslist <db> <start> <count>
<all|dbfilter|tree> [<sort>] )
2) Create/Release a sortcache to speed up sorting ( sc_base sortcache
<db> <create|release> <sort> )
3) And a future function to find the position of a game in the sorted
base by it's number (sc_base gamelocation <db> <gnumber>
<all|dbfilter|tree> [<sort>] )
This is very simple so if i can i will prefer to avoid passing handles
to tcl code just to release the sortcache.
Initially i wrote the sc_base_sortcache function only because your
sc_sort_sort always create a fully sorted SortCache, thinking of
removing it later and to add an option for sc_sort_sort to produce
just an hash SortCache. The idea is that sc_base_gameslist
automatically use a suitable sortcache if it exists, so it's
trasparent for the widget if it had been created by another window, by
a new options into "Sort Database" window or loaded from a file.
Now, however, i'm very surprised by how fast the sorting is (compared
for example to the ChessBase sorting) and i'm not sure what's the best
option is.
Different ideas:
1) Keep things extremely simple: just use the 3 new sc_base functions.
- sc_base sortcache create just an hash cache (and it is very fast)
- a new thread in background create the fully sorted map (about 20
seconds for big bases)
- in the meanwhile sc_base_gameslist will return results using the
hash sortcache (very fast, the actual code for "best games" window).
- when the fully sorted map is created sc_base gameslist will
automatically start to use it (super fast results, in the actual code
this can be simulated creating a fully sorted sortcache with the
"games list" window using the same criteria of the "best games" window)
Big plus: an incredible user friendly interface.
Even novice scid users just right click over an header and enjoy
immediate results and super fast scrolling after few seconds.
And we can avoid much code (less code less problems)
2) Stay with the original plan and leave the user the choice to decide
when to perform a full sort
Plus:
- Sortcache can be saved and loaded
- Users have more options (for example one can create three fully
sorted cache with X+ Y- and Z+ and don't destroy them so changing
sorting criteria will always be superfast)
Minus:
- Less user friendly (user must know that exists an optional sortcache
and understand what it do)
- Don't take advantage of modern multi-core cpu (you're waiting 20
seconds and maybe 3 cores are idling :-(
3) An hybrid solution
- Change the Maintenance -> "Sort Database" window to
show/create/delete/save/load fully sorted sortcaches
This allows power users to decide which sortcaches they want to create
and when to delete it (and eventually save them to disk)
- If a suitable fully sorted sortcache don't exist sc_base_sortcache
will create a new one with a background thread
- sc_base_gameslist (and all the new glist widgets) automatically
start to use the fastest caches when they are ready.
You mention in the code that you would like to perform complete sorting of a
base in the background, after a user has changed the criteria. This will be
a challenge, as this can take around 20 seconds and the user continues
working at the same time. As long as this is not done I would do complete
sorting in all the time for bases with less than let say 5000 games, because
it is very fast. For larger bases I would over a menu entry in the context
to do a complete sort. Then the user has to wait for some seconds but
afterwards he will have a different experience scrolling through the games.
Currently I do not like scrolling at all in a 5.000.000 base. It is much
slower than in the old game list using presorting.
What i want to do is to write some quick code to see how solution 1)
performs.
If it's very fast we can consider avoiding the saving/restoring of
sortcaches.
If it's not, i'm oriented versus solution 3)
If writing multi-threading prove to be too complicated (i don't know
what's the actual support for c++0x std::thread on OS X, Windows and
old gcc; using boost.thread adds a big dependency imho) we will be
happy with solution 2)
What's your opinion?
I think you are right on the format of the new gamelist window.
Just a combobox for database choice (that don't change the current
game), two checkboxes "filter" and "current position" to restrict
results (this is difficult) and a glist widget.
Step by step.
Bye,
Fulvio
------------------------------------------------------------------------------
All of the data generated in your IT infrastructure is seriously valuable.
Why? It contains a definitive record of application performance, security
threats, fraudulent activity, and more. Splunk takes this data and makes
sense of it. IT sense. And common sense.
http://p.sf.net/sfu/splunk-d2d-c2
_______________________________________________
Scid-users mailing list
Scid-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/scid-users
