You are here

Andy Wang (AGPM '18) receives 2018 Koret Scholarship

January 10, 2018

Andy Wang, AGPM '18, has been chosen as one of the recipients of the Koret Scholarship for 2018. Andy and his graduate student mentor, Joseph C. Osborne, will be working on Libretro. The Koret Scholarship provides $1,500 to support undergraduate research and a $2,000 fellowship for the mentor of the project. 

Wang and Osborne describe their research below: 

There is a wealth game design ideas and decisions found from video game to video game, and we want to tap into that pool of information with our project, titled "desAI." We are building a base on top of Libretro, an emulator library that offers us access to a lot of virtual game consoles. That is, our program will be able to run any game found on the NES, N64, Playstation One, PSP, Atari 2600, Nintendo DS, or whatever future platform is added to Libretro's library. From there, the program will interact with the game by sending virtual controller inputs to navigate menus or control the player character. DesAI will also export a screenshot of the game's current state every few frames.

From there, the computer vision component of desAI will be identifying and extracting unique tiles found on screen at any given moment from the game being played. Most of work is currently aiming to program the computer program to be able to recognize enemy sprites from ground tiles, for example. For example, in Super Mario Bros., the program will be able to identify the goomba as an enemy character, lava as dangerous, super mushroom as a power-up, etc. Once desAI has extracted and categorized what it sees on screen in a game, we will be exploring how much we can get a program to learn about game rules, character behaviors, game menu structures, and game states like title screens, game over, etc.
Not only, we are excited to explore the potential of game design information we can get a computer to extract for us, but also gain an understanding how human players can process all the information they see on a screen.