News

Birdfy, a leading brand in smart birdwatching solutions, held a tech webinar on June 26 to introduce the new Birdfy AI and its latest identification capabilities. The webinar, themed "Meet the Mind Behind Birdfy AI: How Bird Identification Works," attracted more than 100 birders.The event featured two speakers, Birdfy CTO Udall Hu and Birdfy AI Consultant Alec Roseto. It was hosted by Brand Ambassador Gary Herritz. Together, they delved into what the updated birdwatching AI can achieve and shared how the birding community contributes to improving the bird identification accuracy. New Birdfy AI Driven by A Vision-Language Model At the webinar, Birdfy CTO Hu introduced that the current birdwatching AI — Birdfy OrniSense — is powered by a vision-language model (VLM), a brand new model completely different from traditional AI models. According to Hu, traditional AI models are like "students who learn by rote memorization.""Our new VLM brain, however, is like a knowledgeable nature guide," Hu said, comparing it with the traditional models. "It doesn’t just recognize pixels — it understands the birds, their habits, and the environment." What distinguishes it from its predecessor resides in several key aspects. The most significant one is that the new model not only identifies the bird species but also tells you why. Moreover, when faced with tricky problems, such as identifying a rare species, it leverages both the visual clues and the ornithological encyclopedia it was trained on, raising accuracy rates.According to Hu, when a traditional model gets a bad photo, it panics and forces a completely wrong guess just to give you an answer. "Our VLM is much smarter than that — it actually knows what it doesn't know," Hu said, introducing another feature called smart fallback. With this, the OrniSense model will output a broader category whenever the photo or video is too blurry for the system to label a bird species.Hu also explained how the vision-language model reads images with a different approach. The new AI system "reads the room” by checking the environment, in stark contrast to traditional AI that only looks at the bird itself."Our VLM looks at the entire story of the video. It takes into account the background — whether it's a wetland, a desert, or an icy branch — and uses this habitat context to eliminate some impossible, silly guesses," Hu said.Before diving into these new AI features, Hu briefly recapped the brand's AI journey in the past five years. He recounted that Birdfy first launched the bird AI identification feature for smart feeders in 2021, and released another AI feature — nesting process identification — exclusively to smart birdhouses in 2022. These core capabilities have so far successfully helped deliver joyful smart birdwatching experiences for every birder. Latest AI Features Elevate Backyard Birding Experiences Since the deployment of Birdfy OrniSense, the team has rolled out three new AI features, aimed at elevating birdwatching experiences with extensive context and enhanced accuracy. These features are sex identification, animal recognition, and geographic location filtering.The most noticeable capability is sex identification. Emphasizing its educational benefits, Birdfy's AI Consultant Roseto introduced that Birdfy products can now identify the sexes of feathered friends among select species."This is very beneficial because it gives the user a better understanding of which birds are visiting their feeder and potential behavioral changes with seasons," he said.Roseto added that the new model can educate users on plumage differences between male and female birds of certain species, such as Brown-headed Cowbirds, House Finches, and Northern Cardinals.Another update is animal recognition. The OrninSense AI can now recognize up to 16 animal species, including squirrels, deer, raccoons, cats, and dogs. This feature makes smart birdwatching even more practical and joyful for nature lovers. It helps birders prevent the targeted wild animals from accessing birdseed. For those who have befriended a squirrel that frequents their backyards, they can get instant alerts of its visits.Roseto shared that the new AI also comes with geographic location filtering — a process that narrows down the bird species pool by using location information.A case in point is the identification of a Magpie. By comparing Birdfy camera's location with the brand's GeoBird Database, the AI system is trained to label a Magpie spotted in the United States as a Black-billed Magpie rather than a Eurasian Magpie. The former is commonly seen in North America, whereas the latter is a resident bird in the Eurasian continent.This powerful feature also enables the model to identify birds with better precision on the local level. According to Roseto, location information is helpful for distinguishing Carolina Chickadees and Black-capped Chickadees, two US-based bird species that are nearly identical in appearance.Roseto noted, "Having an accurate location will separate Carolina from Black-capped Chickadees in most instances." A Close-Knit Community for Smart Birdwatching At the webinar, the Birdfy team also touched on tricky challenges AI bird identification faces, such as birds in weird poses. Brand Ambassador Herritz shared a few tips to resolve the problems and called for birders to submit bird ID corrections to sharpen the model’s identification capabilities.“When you do get an ID that is incorrect, please go ahead and hit the button to put the correct ID in," Herritz said. “It does make a difference!"It turns out bird lovers' efforts do help enhance the capabilities of Birdfy AI. Hu revealed that real-world data is extremely valuable for efficiently training the AI model to boost bird identification.“For a long time, our model struggled to tell the difference between the Common Grackle and Brewer’s Blackbird,” Hu said.Thanks to hundreds of corrections submitted by sharp-eyed birders in the Birdfy community, the AI model can now tell them apart. Hu announced, "I’m happy to share that the updated model is now officially live and successfully distinguishing between the two!" Looking Ahead: More Features to Come Audience members at the webinar also had early glimpses of AI features currently under development. Hu shared that the team will soon roll out a new feature called sick bird identification.He said the model is being trained to recognize avian diseases, such as avian pox and severe feather degradation. Once the illnesses are identified, the Birdfy app will notify users to clean smart feeders, guaranteeing the health and safety of backyard bird community.According to Hu, the team is also training a self-learning personalized AI, designed to possess account-level memory. What's more, Birdfy has been researching bird tag and band recognition. This feature is designed to automatically detect the bands put on birds' legs by conservation groups or scientific institutions and report their sightings back to these entities. It will facilitate the global network of Birdfy products to contribute to citizen science.Hu admitted that it is challenging to tackle this problem, because the ID numbers on the leg bands "are microscopic and often blurred in motion.""But our team loves a hard problem, and we are pushing the boundaries of Ornisense (VLM) to make it happen," Hu said.