The skull of a helmet vanga (Euryceros prevostii), a species that lives in Madagascar, shows some of the incredible variation in skeletal form found in birds. More advanced 3D representations of this variation promise to yield new insights into the origins of avian diversity around the world and how it may respond to global environmental change.
The skull of a helmet vanga (Euryceros prevostii), a species that lives in Madagascar, shows some of the incredible variation in skeletal form found in birds. More advanced 3D representations of this variation promise to yield new insights into the origins of avian diversity around the world and how it may respond to global environmental change.

From Measuring by Hand to AI-Assisted Computer Vision

By Nayiri Mullinix | 
View Nayiri Mullinix's Profile

An AI system is helping researchers understand the evolution of birds, which contributes to a better understanding of responses to global change.

Our understanding of the evolution of bird bodies is being reshaped thanks to a deep neural network-based computer vision system called Skelevision, which uses artificial intelligence (AI) to identify and measure bird bones from photos of thousands of museum skeletal specimens.

Started in 2019 by University of Michigan School for Environment and Sustainability (SEAS) Associate Professor Brian Weeks and David Fouhey, assistant professor at New York University, Skelevision transforms the once laborious process of identifying and measuring bones of bird skeletal specimens into a fully automated process performed by computers.

The result is the most extensive dataset of skeletal trait measurements in birds to date. This dataset is important because birds are considered a model system, meaning they are often used to understand how ecosystems and species are responding to human alteration of the environment.

Building this technology involved contributions by undergraduate students in PitE and SEAS master’s and PhD students, who have annotated a few hundred examples of every bone element, used those data to train Skelevision to segment out bones from images and then identify and measure them. They then used Skelevision to measure traits on more than 2,000 species of birds, with measurements of 12 skeletal elements from 14,419 individuals mostly held in U-M’s Museum of Zoology, which is among the largest and most diverse skeletal specimen collections in the world.

Contributing to Skelevision’s history has been an incredible privilege. The gravity of how much it can help researchers interested in museum specimens really dawned on me when I gave researchers demonstrations—they often mentioned how it could help improve the efficiency of their research personally."

“Traditionally, the development of large-scale datasets characterizing the skeletal system in birds has been limited compared to other types of traits, presumably because measuring bird bones by hand is really tough work,” says Weeks, an evolutionary ecologist. “We can now look at how skeletal traits have evolved, and we can also combine skeletal traits with existing external traits to better understand how the evolution of birds has been influenced by the demands of thermoregulation, or how organisms maintain their body temperature despite external temperatures.”

This is changing how we understand things like the evolution of flight or centuries-old ideas about the relationship between bird body size and temperature, and may help us predict how species will or will not be able to adapt to human-caused climate change.

SEAS PhD candidate Charlotte Probst says that access to a dataset of this size, which is available to any researcher because it is open source and open access, makes her reflect on the toil of measuring bird specimens by hand, which she was still doing until about five years ago.

“Creating this massive dataset in a short period is game-changing in itself, but it’s also impressive how flexible Skelevision is. If someone is interested in analyzing a bone that wasn’t run in the training the first time, they could go back, make a new annotation data set for it, run it through Skelevision, and get thousands and thousands of measurements for that bone quite quickly,” says Probst.

Weeks adds that, as the image bank grows, “the amount of training data needed will stay the same, making it possible to measure a new trait on thousands or millions of specimens in a few days rather than months or years.”

An image of a skeletal specimen with target bones identified and outlined by Skelevision. These outlines of the bones are then used to measure aspects of the skeletal elements for use in trait-based research.
An image of a skeletal specimen with target bones identified and outlined by Skelevision. These outlines of the bones are then used to measure aspects of the skeletal elements for use in trait-based research.

This past summer, PitE alum and SEAS master’s student Madisyn Guza (BS ’22, MS ’26) worked in the Field Museum in Chicago to collect scans for version 2.0, which will include 3D capability. She says that, in the process, she has realized the magnitude and impact of this project.

“Contributing to Skelevision’s history has been an incredible privilege,” says Guza. “The gravity of how much it can help researchers interested in museum specimens really dawned on me when I gave researchers demonstrations—they often mentioned how it could help improve the efficiency of their research personally. Scanning entire collections as we’re doing provides massive amounts of data, advancing science in unimaginable ways.”

Weeks says that one goal of version 2.0 is to develop detailed 3D models of bones that might be able to inform a classification model that would be able to identify species from fragmentary bones from archaeological sites, called zooarchaeology deposits, which would allow researchers to ask questions about how biological communities have changed at a large scale.

In the age of AI, Weeks emphasizes the importance of collaboration, calling his cross-disciplinary work with Fouhey and his group essential, and explains that Skelevision would not have been possible without centuries of investment and continued support of natural history museum collections.

“Museum collections are core to this work, and will continue to be. They take continued investment and are priceless, irreplaceable resources for basic research and conservation biology.”

More Stories from the Magazine

Table of Contents
A California Department of Forestry and Fire Protection crew work during the Palisades Fire that started in the City of Los Angeles in January 2025. Image credit: Cal Fire

Tackling the Wildfire Problem Using a Holistic Approach

Wildfires have become an increasingly severe and frequent threat across the U.S. and around the globe. SEAS is approaching the wildfire issue holistically through its Berman Western Forest and Fire Initiative. Its goal is to develop policy and management interventions that can mitigate the risk of wildfires and help communities better respond to them.

Driving Toward a Cleaner Future

Driving Toward a Cleaner Future

Despite the rollback of government incentives, the momentum behind electric vehicles (EVs) isn’t slowing. Vehicle electrification is the future, says SEAS Professor Greg Keoleian, whose recent study compares the greenhouse gas emissions of different vehicles based on size, usage, powertrain type and location. SEAS alumni working on EVs also share their insights.
SEAS Assistant Professor Parth Vaishnav poses with SEAS PhD student Mallika Kothari in the Bryant neighborhood in Ann Arbor, where they are studying how residents there cope with power outages of greater than 12 hours. Vaishnav's work is a small slice of a much larger project to make Bryant the nation's first carbon-neutral neighborhood.

Making Ann Arbor’s Bryant Neighborhood a Model for Sustainable Energy

SEAS Assistant Professor Parth Vaishnav and SEAS PhD student Mallika Kothari are studying how residents in Ann Arbor’s Bryant neighborhood cope with power outages of greater than 12 hours. Their work is a small slice of a much larger project to make Bryant the nation’s first carbon-neutral neighborhood.