How can automated image processing be applied for detecting, counting and identifying animals in natural landscapes? A recent review article published in the Journal of Animal Ecology covers this exact question! Dr Ben Weinstein is a postdoctoral fellow at Oregon State University interested in community ecology, Bayesian modeling and data science. He studies a wide range of taxa including tropical hummingbirds, Antarctic whales, and temperate trees. In March 2018, he will be starting as a Moore Data Fellow in the Weecology Lab at the University of Florida, where he will be studying remote sensing, machine learning and tree species classification. 

The Challenge

Finding, counting, and identifying animals is a central challenge in ecology. Most studies are limited by the time and funding of fieldwork by human observers. To increase the spatial and temporal breadth of sampling, I believe ecologists must turn to greater automation to increase ecological sampling. There simply are too few ecologists to capture the dynamic nature of species presence, behavior and demography.

The potential for automation is made possible by the decreasing cost of remote data capture. From camera traps, to time-lapse videos, to remote sensing, images will play a crucial role in the future of ecological data collection. But how do we analyze this data? How can we extract the needed information from enormous datasets without having to review them by hand? Our ability to gather data has greatly outpaced our ability to make sense of it.

The Tools

Computer vision is a form of image-based artificial intelligence that mimics human vision by generating rules for the form, grouping, and changes of image pixels. Computer vision is a driving force behind the growth in visual intelligence, image search, and self-driving cars. Computer vision algorithms are increasingly accessible, with a number of robust libraries in python and C++. While a human reviewer may be better at finding seals on sea-ice, or have a greater knowledge of bird identification, when confronted with 100,000 images, it is difficult to find the time, organization and concentration to validate each image by hand. A number of computer vision tools have recently been published for the wider ecological community. The goal of my article was to review recent applications of computer vision to ecology and identify areas of potential growth. I divided articles in Measurement, Counting, and Identification tasks to help orient the reader to specific ecological goals.

The Benefits

By embracing automated image analysis for capturing biodiversity, we can greatly expand our sampling of the natural world. Computer vision will help reduce the burden of laborious image review and allow ecologists to focus on our science, not our data. One unexpected benefit of increased data collection is greater statistical rigor. I believe the gold standard of model evaluation is out-of-sample prediction (Hooten et al. 2015). Yet we rarely use prediction to evaluate models. Our lack of data has always made validation datasets a necessarily casualty. We often cannot afford to withhold a portion of the dataset used to fit our models, we barely had enough data as it is! With greater automation, not only will we have more data, but we will greater confidence in our inferences.

The Obstacles

The challenge of staying abreast of developments across multiple fields is daunting. Applications of computer vision may spring up in biomedical sciences, engineering, robotics and a dozen other fields. Converting new research into practical tools is time-consuming. At some point, this leads to the question, to what extent is it our responsibility as ecologists to master these tools? Should this be part of the ever-expanding cannon of expected skills for new researchers? The alternative is to seek out collaborations with computer scientists and engineers interested in challenging problems. I admit I am torn on this issue. If we don’t arm ourselves with these tools, we will have to wait for others to solve our problems. Ecologists are unlikely to have the economic power to fund new solutions, as the demand for computer vision developers is too great. In addition, many of our challenges are more of application than of theory, and as such, are not as attractive to academics on the cutting edge of computer vision. I was once told that 30 years ago, if you wanted a complex Bayesian model, you invited a statistician as a collaborator. Now it’s something we just teach our students. Why not image processing? Where is that line? It’s a hard one.

The Future

Penguins from space! Whales from space! A flower-counting drone. Photo mark-recapture of individuals from their patterns. The explosion of deep learning neural networks, accompanied by easy to use python libraries, will help bring biodiversity science into the machine learning age. My own work is focusing on machine learning models for detecting animals in time-lapse video. Finally, I fervently hope that all researchers will work hard to make their advances available to ecologists of all computational backgrounds. Sharing this work will allow ecology to move towards a new future of automated data collection.

 

More Info:

Weinstein, B.G. (2017) A computer vision for animal ecology. Journal of Animal Ecology. https://doi.org/10.1111/1365-2656.12780

References

Hooten, M. B., A. N. T. Hobbs, and N. T. Hobbs. (2015) A guide to Bayesian model selection for ecologists. Ecological Monographs 85:3–28.

Torney, C. J., A. P. Dobson, F. Borner, D. J. Lloyd-Jones, D. Moyer, H. T. Maliti, M. Mwita, H. Fredrick, M. Borner, and J. G. C. Hopcraft. (2016) Assessing Rotation-Invariant Feature Classification for Automated Wildebeest Population Counts. Plos One 11:e0156342.

Weinstein, B. G. (2015) MotionMeerkat: Integrating motion video detection and ecological monitoring. Methods in Ecology and Evolution 6:357–362.