MBL, UChicago Scientists Roll Out New Tool for Viewing 3D Images
The excited buzz on campus about a new imaging tool is usually heard first from the MBL courses. Last summer, it was all about “napari,” a platform for viewing and analyzing big, multidimensional images:
But in the case of napari, a community-built, open-source platform, MBL imaging scientists were already fast on its trail. Over the past year, Senior Scientist Rudolf Oldenbourg has been developing a plug-in that allows the napari image viewer to be used with light-field microscopes.
“Light-field imaging is a method I’ve been working on with Patrick La Rivière, professor of Radiology at University of Chicago, together with Patrick’s graduate student in Medical Physics, Geneva Schlafly,” Oldenbourg said.
The cool thing about light-field is it can capture 3D information about fast-moving processes or organisms in a single snapshot. “If you want to image a network of neurons firing, for example, that’s 3D and it’s happening very, very fast. Light-field is really a godsend for that,” said Oldenbourg.
Oldenbourg and La Rivière received a grant from the Chan Zuckerberg Initiative (CZI) to add light-field imaging to the napari platform in fall 2021, and their plug-in is now available on GitHub.
CZI is investing deeply in the napari community, having awarded more than $1.8 million to 75+ scientists to extend napari to new capacities and microscopy systems.
“After a user captures images on their microscope, they can upload them to napari and in particular they can choose to use our plug-in to upload their light-field images. And then we have a method that will analyze that raw data from the microscope and turn it into something more interpretable by the user,” Schlafly said.
Light-field microscopy, first developed at Stanford University, has yet to catch on widely among biologists. “One reason is there hasn’t been a reasonable way to view the images,” said Schlafly. “Generally, if experimentalists have a light-field microscope, the images they capture are not 3D. They're in this strange format that has information about the 3D structure, but that is not at all immediately evident. Our napari plug-in is an accessible way for scientists to see the analogous 3D structure that is formed from their images.”
Napari uses the Python programming language, which has become dominant for imaging applications – from microscopy to astronomy -- over the past decade. Biologists and other non-programmers can learn it quickly, and Python also has robust algorithms for applying machine learning to image analysis. While the Oldenbourg group’s plug-in doesn’t yet incorporate machine learning, this is in development with the support of Josué Page Vizcaino, a graduate student with Tobias Lasser at the Technical University of Munich.
“Our napari plug-in can be adjusted to many different types of image reconstruction methods, such as iterative or machine learning approaches,” Schlafly said. “In order to compare and improve reconstruction methods, it is very helpful to be able to visualize the outcome. And our tool enables scientists to do that.”
Our eventual goal is to bring polarized light microscopy to three dimensions.
The team views the light-field plug-in as an important step in a grander scheme. “Our eventual goal is to bring polarized light microscopy to three dimensions,” said Oldenbourg, referring to the microscopy approach developed at MBL since the 1950s by the late Shinya Inoué and others, including Oldenbourg.
“One of the ways we can get from 2D to 3D in polarized light microscopy is to use light-field microscopy and image reconstructions, and later add polarization,” said Grant Harris, a consultant with Oldenbourg. “We call it polarized light-field microscopy.”
Along the way, the team also has assistance from Amit Verma, a scientific informatics analyst in the MBL’s Bell Center. Verma helped design the napari plug-in’s framework and graphical user interface.
