|
|
Post by Steve Draper on Apr 4, 2014 11:34:29 GMT -8
Can I assume that the choice of games is made randomly from some curated set (e.g. - the Tiltyard set say, though only by way of example - hopefully a real championship draws from a larger set with at least some new-to-the-players games included), or is it a humanly chosen set? If the latter how are biases (and potentially pre-knowledge) issues avoided?
I'd kind of hope it worked by having a largish curated pool from which a random draw is performed immediately prior to the competition (or pools with certain characteristics if it is desired to include things by category). Ideally the draw would occur immediately before each round.
How has it been done previous years? How will it be do this year?
|
|
|
|
Post by Sam Schreiber on Apr 4, 2014 13:38:04 GMT -8
It's definitely not random. The people organizing the competition choose games, and frequently write their own games for the competition. They often make choices and write games to emphasize some specific aspect of GGP that they're interested in. For example, the last few world competitions have emphasized game factorability in a way that's pretty unrepresentative of actual games (e.g. including games that were factorable in contrived ways, just to have factorable games). This was because the organizers wanted to encourage research into game factoring.
The people organizing the competition don't participate, and they don't tell the participants about the games in advance (except for vague things like "expect lots of factorable games" designed to motivate research). So it's not really a conflict of interest to have humans choosing the games manually, at least in terms of pre-knowledge.
|
|
|
|
Post by Steve Draper on Apr 4, 2014 14:52:48 GMT -8
It's definitely not random. The people organizing the competition choose games, and frequently write their own games for the competition. They often make choices and write games to emphasize some specific aspect of GGP that they're interested in. For example, the last few world competitions have emphasized game factorability in a way that's pretty unrepresentative of actual games (e.g. including games that were factorable in contrived ways, just to have factorable games). This was because the organizers wanted to encourage research into game factoring. The people organizing the competition don't participate, and they don't tell the participants about the games in advance (except for vague things like "expect lots of factorable games" designed to motivate research). So it's not really a conflict of interest to have humans choosing the games manually, at least in terms of pre-knowledge. Are there any 'expect...' hints known for this year? Coincidentally I have been working on factorization and have it working so far in C4 simul and Chinook (it has defeated the unfactorizing Sancho 10:0 in both just this afternoon, which was the first time I had it working fully) |
|
|
|
Post by Sam Schreiber on Apr 4, 2014 15:14:19 GMT -8
The only thing I'm aware of is a continued focus on factoring, but that's been the case since 2010 or so.
|
|
|
|
Post by maciej on Apr 5, 2014 1:34:17 GMT -8
The choice will and cannot really be perfect. There will always be bias towards something. Some of the games will suit some of the players better but this is unavoidable and somewhat fascinating, unpredictable.
However, I found many games really unfair in terms of roles (starting positions). I hope this will not be an issue, if I recall it was well handled last year, where players played each two-player game twice for both configurations. They faced different opponents each time, but this is still a well-defined tournament.
For example in any variants of Connect-N, Pentago my player defeats its clone by a significant margin when it is first to act. There are some games complex enough where humans maybe could exploit the favourable role but GGP agents cannot. What I am suggesting to use either fair games or have each player play each role in the course of the tournament.
|
|
|
|
Post by Steve Draper on Apr 5, 2014 6:30:42 GMT -8
The only thing I'm aware of is a continued focus on factoring, but that's been the case since 2010 or so. Sam (or anyone who has suggestions) - other than the obvious Tiltyard examples (Chinook, C4 simul., Breaktrhough w/walls) are there any other good examples of factorized games (other than a few toys like the Lights On variants) in the repositories that would be good to look at, which you are aware of? |
|
wat
New Member
Posts: 32 
|
Post by wat on Jun 17, 2014 20:14:12 GMT -8
Most games (outside AI field) are not factorable.
Advances in imperfect information games and hybrid cooperation-competition games (non zero-sum games) would have a lot more practical applications. And most players currently suck at both areas.
|
|
|
|
Post by Steve on Jun 18, 2014 6:17:02 GMT -8
Most games (outside AI field) are not factorable. Advances in imperfect information games and hybrid cooperation-competition games (non zero-sum games) would have a lot more practical applications. And most players currently suck at both areas. Actually I'm not sure I totally agree with most games being non factor able. While it is strictly true, many games have approximations which are factor able (e.g. Civilization, where approximate factorization into mostly independent tactical theaters is in principal possible). Even large c4 variants are approximately factor able by striping into overlapping sub- boards, which are then considered independently. This is not strict factorization (it will produce imperfect results), but it makes good heuristic approximations. This means that techniques developed for perfectly factor able games, likely have interesting application as a generator for heuristic approximations of more complex games, which approximations then become tractable for search. |
|
|
|
Post by bertrand on Jun 27, 2014 13:41:58 GMT -8
It's definitely not random. The people organizing the competition choose games, and frequently write their own games for the competition. They often make choices and write games to emphasize some specific aspect of GGP that they're interested in. For example, the last few world competitions have emphasized game factorability in a way that's pretty unrepresentative of actual games (e.g. including games that were factorable in contrived ways, just to have factorable games). This was because the organizers wanted to encourage research into game factoring. The people organizing the competition don't participate, and they don't tell the participants about the games in advance (except for vague things like "expect lots of factorable games" designed to motivate research). So it's not really a conflict of interest to have humans choosing the games manually, at least in terms of pre-knowledge. I would like to say that Sam mostly got it right. Stanford staff/students are not allowed to participate. To be more precise, they are allowed to participate, but they can't win.
I would also like to emphasize how come factoring games has been such a concern since 2010. We agree that not all games are factorable in a really relevant manner (but, for example, you can factor out whose turn it is to play on a lot of games). However, I strongly feel that factoring is at the heart of many problems in A.I. I will give you the reasons why I left the field of machine learning for the one of logic. I was a strong proponent of machine learning, it is a tool that actually works, and can get extremely good results. Google is just machine learning, and no one can deny that it has been successful. And there are many many other success-stories (face detection, targeted ads, etc...) My major problem with statistical methods is that they are excellent at describing things, but not at explaining things. If I ask you to turn on the TV, you will simply grab the remote control, and press the relevant button. But if I ask a robot to do the same, there is a "sensory overload". The robot has a lot of inputs (audio, video, etc...) but most of them are not useful for the current task at hand. The fact that the TV is red is irrelevant. The fact that it's 2:43 p.m is irrelevant. A lot of facts are irrelevant. And this is factoring. Another example is how, in the machine learning articles I read and conferences I attended, one major part of the work consists of cleaning up the input. For example, cancer predisposition detection started with considering 100.000 genes, and discarding all of them but 40. They actually "factored" the 40 relevant genes. This is a trend is not unique to the examples I gave. Currently, when you tackle a huge problem, a lot of work is spent getting rid of the less relevant information. A very basic automatic one (and very popular as soon as you use linear algebra) is PCA. It has its uses, but also a lot of flaws (and thus, a lot of improved versions of PCA) We think that there is a huge step up that can be done in this area, where we can discover what is relevant with logic, rather than statistical analysis. |
|
wat
New Member
Posts: 32
|
Post by wat on Jul 3, 2014 11:39:29 GMT -8
Got it. Factoring as a kind of classification.
And classification is a part of machine learning too. Learn what is relevant and what is not.
And it can be done with statistics too, although the focus seems to be targeted at analytical factorization. Probably because it is harder and only come as the fruit of research, while statistical challenges can usually be overcome with raw CPU power.
|
|
wat
New Member
Posts: 32
|
Post by wat on Jul 3, 2014 17:24:58 GMT -8
Scored zero in every game. MCTS didn't even find a path with non-zero goals during mid-game. Game analysis only GGP qualification.  |
|
|
|
Post by Steve Draper on Jul 11, 2014 4:58:20 GMT -8
A thought just occurred to me (which is probably obvious, but I figured I should mention it): in any elimination round that is strictly between two players you should probably avoid any non-fixed-sum games, since in those games there is an incentive to maximize one's score even if that means (knowingly) scoring less than the other role. The meta-structure imposed by the context the game is being run in (a direct elimination round), means that the goal structure of non-fixed sum games do not reflect the actual (competition) goals. Of course such games are fine (and appropriate) in pool rounds where everyone plays the same games against each other (in both role orders) and the scores are totally across all games in the round to determine the qualifiers.
|
|
wat
New Member
Posts: 32
|
Post by wat on Jul 11, 2014 12:58:20 GMT -8
A thought just occurred to me (which is probably obvious, but I figured I should mention it): in any elimination round that is strictly between two players you should probably avoid any non-fixed-sum games, since in those games there is an incentive to maximize one's score even if that means (knowingly) scoring less than the other role. The meta-structure imposed by the context the game is being run in (a direct elimination round), means that the goal structure of non-fixed sum games do not reflect the actual (competition) goals. Of course such games are fine (and appropriate) in pool rounds where everyone plays the same games against each other (in both role orders) and the scores are totally across all games in the round to determine the qualifiers. Nice observation. Players (robots) don´t know in which kind of competition they are playing. And non-constant sum games in head-to-head elimination would eliminate the player which knows how to play non-constant sum and reward the hard-coded constant sum only player. |
|
|
|
Post by bertrand on Jul 11, 2014 18:13:46 GMT -8
A thought just occurred to me (which is probably obvious, but I figured I should mention it): in any elimination round that is strictly between two players you should probably avoid any non-fixed-sum games, since in those games there is an incentive to maximize one's score even if that means (knowingly) scoring less than the other role. The meta-structure imposed by the context the game is being run in (a direct elimination round), means that the goal structure of non-fixed sum games do not reflect the actual (competition) goals. Of course such games are fine (and appropriate) in pool rounds where everyone plays the same games against each other (in both role orders) and the scores are totally across all games in the round to determine the qualifiers. Yes, last year, a competitor pointed that to us, and we embarrassingly had to abort the match, play another match that was scheduled, update the GDL on the fly in the meantime and play the updated version. Needless to say, we would rather not do the same mistake this time (as you may have noticed with the updated Alquerque version played during the qualifiers) |
|
