Task registration and preliminary run submission extended to Aug 23!

--------------------------------------------------------------------------------------------
2nd Call for Shared Task Participation
AI Werewolf and Dialog System (AIWolfDial2019)
As part of the 1st International Workshop of AI Werewolf and Dialog System
Collocated with INLG 2019 conference, October 29, 2019, Tokyo, Japan
https://aiwolfdial.kanolab.net/shared-task
--------------------------------------------------------------------------------------------

< Aims >

"Are You a Werewolf?", or "Mafia" (hereafter "werewolf game"), is a
communication game conducted solely through discussion. Players must
exert their cognitive faculties fully in order to win. In the game,
players must hide information, in contrast to perfect information
games such as chess or Reversi. Each player acquires secret
information from other players’ conversations and behavior and acts by
hiding information to accomplish their objectives. Players are
required persuasion for earning confidence, and speculation for
detecting fabrications.
We employ this werewolf game as a novel way of evaluations for dialog
systems. While studies of dialog systems are very hot topics recently,
they are still insufficient to make natural conversations with
consistent context, or with complex sentences. One of the fundamental
issues is a lack of an appropriate evaluation.
Because the werewolf game forces players to deceive, persuade, and
detect lies, neither inconsistent nor vague response are evaluated as
“unnatural”, losing in the game. Our werewolf game competition and
evaluation could be a new interesting evaluation criteria for dialog
systems, but also for imperfect information game theories.
In addition, the werewolf game allows any conversation, so the game
includes both task-oriented and non-task-oriented conversations. This
aspect would provide a handy intermediate goal rather than to create a
general dialog system from scratch.

< Important dates >

Shared tasks:

[EXTENDED] Shared task preliminary run and registration due: August 23
(AoE), 2019
Shared task formal run: end of August
Shared task evaluation result feedback: beginning of September

Papers:

Submissions due: Sep 10, 2019
Notification of acceptance: October 1, 2019
Camera-ready papers due: October 15, 2019
Workshop: October 29, 2019

< Shared task >

* Background *

We have been annually holding the AI werewolf contests under the AI
Werewolf project. The AI werewolf contest has two divisions, the
protocol division and the natural language division. The protocol
division asks participants to implement an AI werewolf player agent
that communicates in a middle language called the AI werewolf
protocol. The natural language division asks participants to implement
an AI werewolf agent that communicates in natural language. We follow
the previous configurations of the natural language division in our
AIWolfDial2019 shared task.

* The Werewolf Game in this Shared Task *

Agents of the AIWolfDial2019 shared task will play the werewolf game
of five players, including roles of a seer, a werewolf, a possessed,
and two villagers. Players do not know other players' roles. All
players, other than the werewolf, are humans.
A game consists of a couple of days, continuing until the human team
or the werewolf team survives. A werewolf can specify and attack
another player in the end of the day; the attached player will be
eliminated from the game. All of players are required to vote to
another player, and a player voted most will be eliminated from the
game. When humans survive, the villager team wins. When a werewolf
survives, the werewolf team wins. A possessed is a human but belongs
to the werewolf team. A seer can specify another player in the end of
the day, then either human or werewolf is notified.

* Agent Technical Specifications *

Language Requirement

A shared task participant of AIWolfDial2019 is required to implement
an AI werewolf agent that communicate either in English or Japanese.
Agents of the Japanese language are required to make an English
version, at least by using machine translation internally.

AIWolf Agent APIs

A shared task participant of AIWolfDial2019 is required to implement
an AI werewolf agent that connects to our AIWolf server via network.
Therefore, there is no limitation on the client side hardware/software
other than the network I/O specifications.

We recommend to use our sample agent code with runtime environment
(java), distributed from our Github repository prepared for this
shared task:
 https://github.com/aiwolfdial/RandomTalkAgent
Please clone the repository to your Eclipse environment and run the
org.aiwolf.ui.bin.AutoStarter class with specifying AutoStarter.ini as
its program argument. This configuration runs the same five random
talk agents. We provide our test server to connect and test your agent
as same as the preliminary/formal run.

Agent Specification

- Day 0 has greetings only.
- The end of Day 0 has an inspection by a seer, and the game starts from Day 1.
- After Day 1, the end of the days have votes by all players and an
attack by a werewolf. Vote, attack, inspection are made via specific
APIs (network communications).
- A day consists of a couple turns, where all of agents can make a
talk for each turn, receiving talks of previous turns.
- An agent should make a talk within a specified periods after a talk
request is sent.
- During days, Agents can communicate anything in natural language.  A
talk should consists of normal letters and punctuations only. An agent
returns "Skip" when nothing to talk, returns "Over" if nothing to talk
anymore in that day.
- Use Agent[0x] (e.g. Agent[05], x is 1-5)  to mention other agents.
- An anchor e.g. ">>Agent[0x]" could be inserted at the beginning of a
talk to refer to another agent, to whom your agent with to talk with.
That agent is assumed to respond something to your agent by using an
anchor.

* From Registration to Formal Run *

Registration

A team should send a mail to aiwolf at kanolab.net (replace at by @)
to register the shared task, describing your team name, a contact
e-mail address, names and affiliations of your members (please mark a
contact person when a team consists of multiple members),
communication language (English and/or Japanese) of your agent. There
is no fee required to register/participate the shared task.

Testing Your Agent System Beforehand

A shared task participant of AIWolfDial2019 is required to implement
an AI werewolf agent that connects to our AIWolf server at the
specified timing. We will provide an AIWolf server running, where
participants can try connecting with dummy agents to check their
system behavior. Participants are required to check their systems
certainly work before the shared task run.

Preliminary Run

Participants should run your five agents connected to our server,
running 50 games. Then submit your game logs to the organisers via
e-mail. If there are too many participants to run the formal run,
organisers might select formal run teams depending on these logs. In
either case, these logs will be used in the final evaluations.

Formal Run

A participant team is required to connect an agent to our server at a
specified timing to play games with other participants. If your team
cannot remotely attend the formal run, you should prepare your system
where the organisers can somehow trigger your remote system to connect
our server and run, e.g. by via a web browser.

System Evaluation

Participants should submit a system design description document to the
organizers. This document and logs of the games might be used for
research purpose and included and published in our overview paper
without any further permission. Participants are encouraged to submit
a paper to the workshop.
Games will be between the same agents, different agents, and/or human
players. Reviewers will perform subjective evaluations on the game
logs, using following criteria:
A Natural utterance expressions
B Contextually natural conversation
C Coherent (not contradictory) conversation
D Coherent game actions (vote, attack, divine) with conversation contents
E Diverse utterance expressions, including coherent characterization

Please not that vague utterances, that can be used regardless of
contexts, are not always natural in the werewolf game.

< Committee >

Contact

E-mail to aiwolf at kanolab.net (replace at by @)

Organizers

Yoshinobu Kano, Shizuoka University, Japan
Claus Aranha, Tsukuba University, Japan
Michimasa Inaba, The University of Electro-Communications, Japan
Fujio Toriumi, The University of Tokyo, Japan
Hirotaka Osawa, University of Tsukuba, Japan
Daisuke Katagami, Tokyo Polytechnic University, Japan
Takashi Otsuki, Yamagata University, Japan

Program committee members
Hitoshi Matsubara, Future University Hakodate, Japan
TBA

< Workshop >

Please see our workshop website for details of the AIWolfDial
workshop, including paper submissions:
https://aiwolfdial.kanolab.net/


-- 
Yoshinobu Kano, Ph.D.
Associate Professor, Faculty of Informatics, Shizuoka University
personal webpage: http://kanolab.net/kano/  e-mail: k...@inf.shizuoka.ac.jp

_______________________________________________
UNSUBSCRIBE from this page: http://mailman.uib.no/options/corpora
Corpora mailing list
Corpora@uib.no
https://mailman.uib.no/listinfo/corpora

Reply via email to