Modelling Cooperation in Network Games with Spatio-Temporal Complexity


The real world is awash with multi-agent problems that require collective action by self-interested agents, from the routing of packets across a computer network to the management of irrigation systems. Archetypally, such systems have local incentives for individuals, whose behavior has an impact on the global outcome for the group. Given appropriate mechanisms for agent interaction and learning, groups may achieve socially beneficial outcomes, even in the face of short-term selfish incentives. In many cases, collective action problems possess an underlying graph structure, whose topology crucially determines the relationship between local decisions and emergent global effects. Recently, such scenarios have received great attention through the lens of network games. Network games abstract the complexity of real-world scenarios, bringing to bear mathematical tools from game theory. However, this abstraction typically collapses important dimensions relevant to mechanism design, such as geometry and time. In parallel work, multi-agent deep reinforcement learning has shown great promise in modelling the emergence of self-organized cooperation in complex gridworld domains. Here we apply this paradigm to the problem of mechanism design in graph-structured collective action problems. In particular, we introduce \textit{Supply Chain}, a spatio-temporally extended environment in which to study this problem. Using multi-agent deep reinforcement learning, we simulate an agent society for a variety of plausible mechanisms, finding clear transitions between different equilibria over time. We define analytic tools inspired by related literatures to measure the social outcomes, and use these to draw conclusions about the efficacy of different environmental interventions. Our findings have implications for mechanism design in both human institutions and systems of artificial agents.