Hi Chris, Hi Matthias,
> I see. So to me, if you can compute a numerical value which flags when > the AI made a mistake, then if it is any good then you can naturally > just turn around and base an AI on it, since the AI could compute the > However, I think I see what you mean. If you just want to have a way to > diagnose when one player in the game began to lose, why don't you just > plot gold value over time? In most top tier ladder matches, most games Thats what I meant. I don't think that this Value will be good enough to base an AI on it. It should just be an indicator. So I guess we mean the same when thing > It might be a really nice feature if the replay viewer had a graph which > would plot both gold and villages vs time, and then even better, if you > could click on the graph to jump to that turn. This would be very > helpful for human players and AI devs I think -- I imagine that it would > be good for your workflow, as you should be able to at a glance figure > out where the key turning points were. This is describing exactly what I was thinking. Thank you for finding better words to describe it. > First, I think a lot of what you are saying is very similar to how we do > our AI testing (and from how interpret it, you and Chris really agree on > more things than not). We figure out some weakness in the current AI, > add something to a new experimental AI, watch them against each other > for a few games to identify if things got better and worse, then pitch > them against each other for a large number of automated games, while > varying some parameters, etc. I think the details may be different, but > the general idea is very similar to what you suggest. I agree. That sounds like what I suggested. Only one point I want to mention that is missing. I don't know if it is intentional or just me misunderstanding it. But I feel that there is a focus on "the one AI". My Idea is more like lets have 20 of them. Or even more. I think it would be nice to be able to choose the AI. So I can play against an aggressive AI or against a more defensive AI. Perhaps someone comes up with a AI that tries some tricks on it's opponents. Just lets open this up, make the AI a plugin or something so that we it is easy for people to try themselves by writing an AI. In the Official game only the best would be integrated to not confuse the player. With a ai_test Arena and an API plugin mechanism(an API?) people like me could come up with new AIs that they have already tested against the existing ones. > Now, I have not used the tool pointed out by Alarantalara (Simon) > because I have my own CL tools, but they are really very simple, > few-line scripts and I think most of what you suggest can already be > done. That might be true. And it seems to be needed (You both use it). But am I the only one not knowing about it? >> If a smarter AI is a desired feature then I would suggest to add more >> features to this ai_test thing. >> -recording of the matches > > When we are testing our AIs dubbed 'Ron' and 'Fred' against the default > AI, we generally have very high win rates (85-98%) and are mostly > interested in what goes wrong when they lose. I once implemented > something that, in case of a loss, turns automatic control over to the > human player who can then save a replay manually (although there was a > bug that actually prevented it from working back then, but in principle > I got it to work). That was only needed for a few games and thus could > be done manually. Having it done automatically for every game would be > nice and might be useful for some things, but I don't think that would > be very hard to do. It sounds more like an Easy Coding task during the > GSoC application period to me. So then we now have a Project for GSoC ! Yeah ! ;-) >> Doing several thousand test games is a way to test if the fractions are >> good balanced > > Actually, no, it will not tell you that. If you pit the same AI against > itself with a different faction, Northerners will always win on average > because they are designed to be played more easily by the AI You are probably right on this. I only wanted to point out that if the AI would be able to play all sides equally well after a change you could detect an unbalance by unbalanced results of the AI matches. > Well, that's all I have time for right now, but if you want to come onto > IRC, I'm always happy to chat endlessly about this. :-) I fear I don't have time for endless chat ;-) > And ... I just saw Chris' last email come in and don't have time to > reply to it, but in brief, I agree. And that's what I meant up there > with coming up with an indicator is hard. I think we can all agree on this. The Indicator will be bad. But better than nothing, and a Human might be able to make sense of it. Cheers, Lars _______________________________________________ Wesnoth-dev mailing list [email protected] https://mail.gna.org/listinfo/wesnoth-dev
