Current Crisis is a vertically integrated project developed at Georgia Tech Institute of Technology in collaboration with the Vertically Integrated Projects (VIP) group. The project focuses on creating a simulation-style video game designed and developed by undergraduate students, which puts the player in the role of a power system operator tasked with managing the grid during extreme events like wildfires and hurricanes. The project aims to use gamified learning to teach about the critical power grid infrastructure and sustainability. Previously, the VIP partnered with Dr. Jessica Roberts and the Tiles lab to display a game demo at the Dataseum in the Crosland exhibition space. Currently, Tiles Lab is collaborating again to measure the learning outcomes and the overall game efficiency towards learning.
Sponsor(s): The video games will complement work for the NSF-funded AI Institute for Advances in Optimization (AI4Opt, www.ai4opt.org) and research efforts at Sandia National Laboratories.
Research Team
VIP Group and Tiles Lab (Jessica Roberts and Stella Quinto Lima)
Contextualized Air Quality Timelines (ContAQT) is an air pollution data platform to visualize local AQI values over time and teach users about the multiple pollutants measured in air quality ratings. The ContAQT platform is part of our AQI Data Literacy project.
A prototype of an interactive sonified map of different habitats in Gray’s Reef National Marine Sanctuary
Project Description
While scientific datasets including ocean data are increasingly shared via publicly accessible online repositories and web-based interactive exploration tools (e.g. https://www2.purpleair.com/ and https://www.ndbc.noaa.gov/) and exhibits at museums and science centers (e.g. Roberts et al., 2019; Ma et al., 2012), these platforms remain difficult for blind and visually impaired (BVI) users to access. Similarly, these materials are inaccessible or difficult to comprehend for sighted individuals with print-related disabilities such as learning disabilities, low visual or text literacy skills, or innumeracy (Roberts et al., 2023).
This project builds off of prior work in data sonification is seeking to create auditory displays representing scientific phenomena for public exploration, including our previous project Accessible Oceans: Exploring Ocean Data through Sound (NSF AISL #2115751) that explored the feasibility of auditory displays for supporting oceanography data interactions by visually impaired visitors in museums. Through a co-design process with oceanographers, teachers of BVI students, and BVI learners, we have created five prototype auditory displays that aimed to support data investigations and communicate oceanography concepts through non-visual representations.
The current Accessible Oceans project is exploring topics of local and regional interest relating to the publicly available data from the Gray’s Reef National Marine Sanctuary to create auditory displays of interest for sighted and BVI learners to address the following research questions:
RQ1: How can we design auditory displays to facilitate conversations about issues affecting Georgia’s coastal and marine ecosystems for both sighted and visually impaired learners in informal environments?
RQ2: What human-data interactions with auditory displays contribute to sensemaking by sighted and visually impaired learners around local marine data in informal learning interactions?
We will answer these questions through the creation and study of one or more auditory displays based on publicly available data collected near and within Gray’s Reef National Marine Sanctuary (GRNMS). We worked with local Sea Grant personnel through a co-design process to decide which datasets and phenomena they feel would be most relevant and captivating for their local audience and potentially help to amplify existing outreach materials. We are now creating and refining auditory displays through an iterative design and feedback loop with Sea Grant personnel, BVI individuals, and science educators. This project will result in 1-3 auditory displays comprising data sonifications and contextualizing information (e.g. narrative tracks, curricular materials) that will be made available to educators in schools and science centers throughout Georgia.
References
Ma, J., Liao, I., Ma, K. L., & Frazier, J. (2012). Living liquid: Design and evaluation of an exploratory visualization tool for museum visitors. IEEE Transactions on Visualization and Computer Graphics, 18(12), 2799-2808. doi: 10.1109/TVCG.2012.244.
Roberts, J., Lowy, R., Li, H.,Bellona, J., Smith, L.,and Bower, J.(2023). Breaking Down the Visual Barrier: Designing Data Interactions for the Visually Impaired in Informal Learning Settings. Paper presented at the annual conference of computer-supported collaborative learning (CSCL 2023). Montreal, Quebec, Canada.
Project Team
Dr. Amy Bower, WHOI [PI]
Dr. Jon Bellona, U Oregon [co-PI]
Dr. Jessica Roberts, Georgia Tech [co-PI]
Dr. Leslie Smith, Your Ocean Consulting, LLC [key personnel]
Stella Quinto Lima, HCC PhD Student
Emily Amspoker, HCC PhD Student
Solyane Berge, MS-CS student
Anuska Giri, Industrial and Product Design undergraduate student
Although artificial intelligence (AI) is playing an increasingly large role in mediating human activities, most education about what AI is and how it works is restricted to computer science courses. This research is a collaboration between the TILES lab, the Expressive Machinery Lab (Dr. Brian Magerko, Georgia Tech), and the Creative Interfaces Research + Design Studio (Dr. Duri Long, Northwestern University) to create a set of museum exhibits aimed at teaching fundamental AI concepts to the public. In particular we aim to reach middle school girls and students from groups who are underrepresented in computer science.
Data Bites asks visitors to build their own training datasets to teach an AI to classify pizzas and sandwiches. After collaboratively building various labeled examples of pizzas and sandwiches using tangible pieces, learners can see how well their classification algorithm is able to distinguish between the two food items.
This 4-year project is funded by the NSF Advancing Informal STEM Learning (AISL) program (NSF DRL #2214463). We are collaborating with the Griffin Museum of Science and Industry in Chicago to conduct focus groups, needs assessments, and pilot testing of exhibit designs based off our prior work.
Knowledge Net asks learners to create semantic networks to collaboratively build up to 4 different characters in a virtual world. Characters’ appearances, likes, dislikes, and abilities can be customized which results in different interactions with the world and other characters.
This research explores how embodiment and co-creativity can help learners make sense of and engage with AI concepts.
Publications
Kafai, Y. B., Proctor, C., Cai, S., Castro, F., Delaney, V., DesPortes, K., Hoadley, C., Lee, V. R., Long, D., Magerko, B., Roberts, J., Shapiro, B. R., Tseng, T., Zhong, V., & Rosé, C. P. (2024). What Does it Mean to be Literate in the Time of AI? Different Perspectives on Learning and Teaching AI Literacies in K-12 Education. In Lindgren, R., Asino, T. I., Kyza, E. A., Looi, C. K., Keifert, D. T., & Suárez, E. (Eds.), Proceedings of the 18th International Conference of the Learning Sciences – ICLS 2024 (pp. 1856-1862). International Society of the Learning Sciences. https://repository.isls.org//handle/1/10828
Yasmine Belghith, Atefeh Mahdavi Goloujeh, Brian Magerko, Duri Long, Tom Mcklin, and Jessica Roberts. 2024. Testing, Socializing, Exploring: Characterizing Middle Schoolers’ Approaches to and Conceptions of ChatGPT. In Proceedings of the CHI Conference on Human Factors in Computing Systems (CHI ’24). Association for Computing Machinery, New York, NY, USA, Article 276, 1–17. https://doi.org/10.1145/3613904.3642332
Our project, Technology for Acquiring Language through Engagement (TALE), focuses on designing a gamified platform to help self-learners acquire foreign language skills, specifically non-native English speakers.
Research question: How might we help intermediate to advanced English learners get more practice speaking their target language with others?
Motivations: Self-learners encounter a number of challenges when attempting to learn a language, including:
Lack of depth in existing free learning materials
Disconnect to native speakers and culture
Struggles with loss of interest or feelings of burnout over time
Difficulties practicing natural conversations
Why gamification? Research has shown that playful environments and gamified approaches can be powerful tools for engaging learners.
People’s quality of life or socioeconomic status can greatly differ based on where they live, whether they live in different cities, different counties, or in different countries. These inequalities and disparities can be and have been measured using indicators related to the different facets of life: wealth, education, health, and even happiness. Communicating where and how those inequalities affect different people and geographical areas to the public in an effective and engaging way is imperative to progress towards a more equitable society. Information visualization, and specifically maps, are a successful mechanism for conveying these differences in quality of life and engaging people in caring about larger issues and other people.
This project investigates how people perceive socioeconomic disparities using an interactive choropleth map visualization where users can change the map to display different socio-demographic indicators, including income inequality, life expectancy, educational attainment, rate of incarceration, and prevalence of HIV/AIDS and obesity. We implemented two versions of the interface: one displaying the United States only by county, and another arranging the United States map and a country-level choropleth map of the world by country, side-by-side. We conducted between-subjects A/B user testing where we surveyed and interviewed users before and after interacting with one version of the map.
Figure 1: Choropleth map of the Secondary Educational Attainment indicator in the United States only, by countyFigure 2: Choropleth map of the Secondary Educational Attainment indicator in the United States, by county (on the right) and in the world, by country (on the left)
We hypothesize that interacting with either map will lead to stronger feelings about the intensity of inequality in the U.S., and more confidence in their opinions on disparities. Additionally, we hypothesize that interacting with a contextualized map of the U.S. within the world will influence the user’s perceptions of the U.S. differently than when interacting with the U.S. map only.
Co-design sessions with oceanographers are helping elicit the aspects of visualized oceanography data that are crucial to convey to BVI and novice audiences.
Project Description
Accessible Oceans: Exploring Ocean Data Through Sound is a pilots & feasibility study in the NSF’s Advancing Informal STEM Learning program (NSF AISL #2115751). The interdisciplinary project team is exploring how oceanography Data from the Ocean Observatories Initiative can be sonified and made interactive in a museum exhibit tailored to visually impaired learners.
The project is a collaboration between the Woods Hole Oceanographic Institute (WHOI), Your Ocean Consulting, the University of Oregon, and Georgia Tech.
Roberts, J., Lowy, R., Li, H., Bellona, J., Smith, L., and Bower, J. (2023). Breaking Down the Visual Barrier: Designing Data Interactions for the Visually Impaired in Informal Learning Settings. Paper presented at the annual conference of computer-supported collaborative learning (CSCL 2023). Montreal, Quebec, Canada. Read the paper here. View our lit review dataset here.
Bower, A., J. Bellona, J. Roberts, L. Smith 2023. Accessible Oceans: Exploring Ocean Data Through Sound, presented at 2023 Sonification World Chat Meeting, 23 May. Online.
Bower, A., J. Bellona, J. Roberts, L. Smith, 2022. Accessible Oceans: Exploring Ocean Data Through Sound (ED12A-03), presented at 2022 AGU Fall Meeting, 12-16 December.
Braun, R., L. Karlstrom, A.S. Bower, M.O. Archer, 2022. Listening to Our World: Sonification Applications in Research, Education, and Outreach Town Hall (TH13H), presented at 2022 AGU Fall Meeting, 12-16 December.
Project Team
Dr. Amy Bower, WHOI [PI]
Dr. Jon Bellona, U Oregon [co-PI]
Dr. Jessica Roberts, Georgia Tech [co-PI]
Dr. Leslie Smith, Your Ocean Consulting, LLC [key personnel]
A virtual museum platform built in Mozilla Hubs provides users a chance to play with shapes and colors in authentic art pieces.
Description
Art is an important factor in child development. Research has highlighted art education’s role in children’s acquisition of the economic, cultural, and civic capital required to sustain a communities’ cultural resources. For K-2 learners, art education also contributes to the development of fine motor skills, cognition, and interpersonal relationships. The incorporation of art museum visits into school curriculum is one of the ways students can have repeated, sustained engagement with art. Recognizing this, many art museums provide digital resources to support the integration of these resources into classrooms, but little research investigates classrooms’ use of these resources. Additionally, little research investigates technology designs that support interactions and needs of K-2 learners, teachers, art museum educators, and docents in fully remote art education settings.
This project uncovers key implications and design requirements for developing effective, remote art education environments for K-2 learners and educators. From these requirements we made novel, instrumented tangible tools that can create beneficial learning opportunities where K-2 learners can practice fine motor skills and age-appropriate art principles. This project also studies how the integration of these tools into virtual environments can support K-2 learners in remote settings.
Understanding K-2 Remote Art Education Needs
Repeated, meaningful art education experiences for young children are often coordinated across school settings, which support daily art encounter opportunities, and museums, which provide more isolated encounters with artworks. To understand the needs of classroom and museum educators in remote K-2 settings, we conducted a needs survey and interview. We developed 3 sets of design requirements covering their needs. We also developed a novel typology of existing art education platforms, identifying where educators’ needs are and are not met.This project is described in our 2022 Interaction Design and Children (IDC) Paper: Ready, Set, Art: Technology Needs and Tools for Remote K-2 Art Education
To satisfy the need for young learners to receive appropriate feedback as they practice fine motor skills in remote environments, we created the Chameleon Clippers. This low cost instrumentation of classic school scissors uses line sensors and a custom built Processing application to alert users when they deviate from the line they are attempting to cut.
This project is described in our 2022 Computer-Supported Collaborative Learning (CSCL) paper:
Mansi, G., Boone, A., Kim, S. & Roberts, J. (2022). Chameleon Clippers: A Tool for Developing Fine Motor Skills in Remote Education Settings. In Proceedings 2022 International Conference on Computer Supported Collaborative Learning (CSCL). Hiroshima, Japan and online. Best design paper nominee
Increasing shifts to online and remote education in recent years — greatly augmented by the Covid-19 pandemic — have created a new challenge for museum-based art educators: How can young children have impactful art engagement experiences on remote museum tours?
In this project, I explore the K-2 art educators’ pedagogical needs in facilitating the remote art tour through co-design, and offer a technological solution, Play and Learn in Virtual Museum (PLVM). PLVM (pronounced as Plum) is a web-based digital platform for K-2 art educators and students to aid the followings:
Simple technology set up for students (3 point setup)
Ability to deeply looking at the art utilizing different interaction models such as discussion, drag-and-drop, point, multiple choice and storytelling
Integration with existing educational tools and different media such as 360 view, youtube, google slides, and images from the museum collection database
Sharable hands-on activities
Texture sound for the 3D elements
This platform offers both moderated and unmoderated versions to make it more accessible for limited resourced art educators.
Transgender and nonbinary (trans/nonbinary) college students face unique challenges as members of Greek Life organizations, as their identities contradict the heteronormative culture of most Greek communities. Despite these differences, trans/nonbinary students still exist within Greek communities, and many Greek organizations have been making efforts to be inclusive of these members. Institutional-level changes, however, do little to prepare individual Greek students to be inclusive of their trans/nonbinary peers, and because GT Greek Life does not mandate LGBTQ+ education, cisgender students struggle to bridge this gap in knowledge, while transgender students are expected to act as educators. Crossroads is an educational mobile application designed for cisgender Greek students. It teaches users about LGBTQ+ concepts, specifically focusing on trans/nonbinary issues within Greek communities.
The Crossroads application has four features: learning modules, community messaging, a glossary, and external resources. In our proof of concept prototype, we designed and tested two modules, one of which focused on basic LGBTQ+ terminology and concepts, and a second that honed in on trans/nonbinary issues. These modules are followed up with ‘daily challenges’ that utilize a spaced repetition model to support long-term learning. We hypothesize that this learning environment will provide cisgender Greek students with an easy-to-access LGBTQ+ learning environment that can be easily integrated into a college student’s schedule and will act as an alternative to relying on trans/nonbinary students for education.
Publications
Stephanie Baione, Yiming Lyu, Audrey Reinert & Jessica Roberts (2022) Crossroads: a transgender education platform for Greek life students, Journal of LGBT Youth, DOI: 10.1080/19361653.2022.2070813 Please contact me if you would like to read the article but do not have access and I will be happy to send you an access link.
When Sulfur Oxides (SOx) are emitted from power plant facilities, they do not fall directly to the ground. They are carried by air currents, sometimes great distances. Modeling of atmospheric transport and dispersion of these particles can estimate fine particulate matter (PM2.5) source impacts attributable to SOx emissions from each of the more than 1,200 coal-fired electricity generating units in operation in the United States between 1999-2018.
The Coal Pollution Impacts Explorer (C-PIE) is a web-based interface designed to visualize and scaffold atmospheric data and modeling for a public audience. Users can investigate the sources of pollution in their home county’s air, examine where pollution from a nearby facility disperses, and explore trends over time as facilities install pollution-mitigating scrubbers in response to legislative actions.
Research on the C-PIE platform investigates how data interactions can be scaffolded to support inquiry and engagement for public audiences.
Below you will find representations of some of our iterative development work on the platform. To read more about the impacts of coal pollution, read our recent article in the Journal Science:
Henneman, L., Choirat, C., Dedoussi, I., Dominici, F., Roberts, J., & Zigler, C. (2023). Mortality risk from United States coal electricity generation. Science, 382(6673), 941-946. https://doi.org/10.1126/science.adf4915