How scientists are using facial-recognition AI to track humpback whales

After decades of whaling decimated their numbers, humpback whales have made a remarkable comeback. The 50-foot giants, known for their elaborate songs, have become common in parts of the Pacific Ocean they disappeared from.

Now, a new study finds that climate change could be slowing that recovery. Using artificial intelligence-powered image recognition, the survey finds the humpback population in the North Pacific Ocean declined 20% from 2012 to 2021.

The decline coincides with "the blob," a severe marine heat wave that raised water temperatures from Alaska to California. The impacts cascaded through the food web, affecting fish, birds and whales.

"I think the scary part of some of the changes we've seen in ocean conditions is the speed at which they're occurring," says John Calambokidis, a whale biologist at Cascadia Research and a co-author on the study." And that would put long-lived, slow-reproducing species like humpback whales and other large whales as more vulnerable."

Facial recognition for whale tails

Ted Cheeseman is a co–author of the new study, and for 30 years, he worked as a naturalist, guiding trips on boats around Antarctica. That meant looking for whales, which wasn't easy in the early 1990s.

"We saw very, very few whales," he says. "In the 2000s, we saw more. The 2010s – we started seeing quite a few whales."

The whales were making a slow recovery after industrial whaling, which continued into the 1960s for many species. Over years of photographing whales, Cheeseman realized he was collecting valuable data for scientists.

Photographs are key for counting whales. As they dive deep, humpbacks raise their tails out of the water, revealing markings and patterns unique to each individual. Scientists typically identify whales photo by photo, matching the tails in a painstaking process.

Cheeseman figured that technology could do that more quickly. He started Happy Whale, which uses artificial intelligence-powered image recognition to identify whales. The project pulled together around 200,000 photos of humpback whales. Many came from scientists who had built large images catalogs over the years. Others came from whale watching groups and citizen scientists, since the website is designed to share the identity of a whale and where it's been seen.

"In the North Pacific, we have identified almost every living whale," Cheeseman says. "We were just doing this as a study of the population. We didn't expect to see a major impact of climate."

Don't call it a comeback

Humpbacks in the North Pacific Ocean likely dropped to only 1,200 to 1,600 individuals in the wake of whaling. By 2012, they had climbed back to around 33,000 whales. The study finds after that, their numbers started falling again.

The biggest decline was seen in one particular group of humpbacks in the Pacific. As migratory animals, the whales swim thousands of miles, returning to the same sites every year. Some whales spend their summers feeding in Alaska and then head to Hawaii for the winter. The study found this group declined 34 percent, while other groups didn't see as sharp of a drop.

"It tells us something pretty dramatic happened for humpback whales," Calambokidis says. "We are facing a new era of impacts."

Calambokidis says for years, scientists wondered if humpbacks had recovered so well that they'd hit a natural plateau, if the ecosystem couldn't support more animals. He says the study shows something else is at play too.

The Alaska-Hawaii whales may have been more susceptible to the dramatic changes caused by "the blob." Spanning several years, the intense marine heatwave disrupted the food chain, including tiny organisms like krill that feed larger animals like whales. Studies show that marine heat waves are likely to become more common as the climate keeps warming due to the burning of fossil fuels. Humpbacks are also vulnerable to ship strikes and getting entangled in fishing gear off the West Coast.

Calambokidis says the humpback decline was easier to detect because the whales have recovered so strongly. For rarer whales, it's much harder to track and count them, making it difficult to see how marine heat waves may be having an impact. The hope is that new technology, like Happy Whale, will help reveal these changes faster than ever before.