Gigapixel optical microscopy for meteorite characterization

Abstract

We present an automated microscopy system for the optical characterization of meteorite thin sections. The system employs focus-stacking and high-dynamic range imaging to facilitate high-contrast and unpolished samples. Images are acquired at ∼385 nm/pixel and automatically stitched together to create a billion-pixel image for a typical ∼1 cm² thin section. This image can be viewed in a web browser (with smooth panning and zooming) using a free browser plugin. The software we created to acquire and assemble these images is made freely available for others to create a similar system. Large optical digital images of meteorite sections make it possible to collaboratively inspect and characterize the sample by using the web browser interface as a “virtual microscope”. The system can be employed on any optical microscope with a computerized stage and consumer-grade digital camera, and can be used in a wide range of applications.

Background

Optical microscopy is an important first step to characterize geologic samples. A petrographic description of the sample can be made by using a relatively simple and inexpensive optical microscope. A detailed optical characterization of the sample should be made before carbon coating for electron-beam analyses or destructive techniques, such as isotopic analyses, are applied. The relatively recent development of digital photography in optical microscopy has allowed for rapid and high-quality recording of optical micrographs.

Optical microscopy is the first method used to characterize meteorite thin sections. Meteorites are classified into groups that have common mineralogical, petrographical, chemical, and isotopic properties. Meteorites within a given group are thought to originate from the same parent body. Although optical microscopy cannot be used to measure quantitative chemical or isotopic composition, it is vital for determining the terrestrial weathering grade, stage of shock, and basic petrography of a meteorite.

Upon recovery of a meteorite, hand samples are typically observed, weighed, and described at a curation facility, such as those at the Johnson Space Center or the Smithsonian National Museum of Natural History. Meteorites are then sectioned and observed at the facility under an optical microscope (and possibly by other techniques) for a preliminary characterization. Sections may then be distributed to interested scientists and curators for further analysis. To properly classify a meteorite, modal abundances of features (such as chondrules, clasts, inclusions, or matrix material) and the size distribution of chondrules are typically acquired under reflected light microscopy to make a preliminary identification of the meteorite group (for typical chondritic meteorite characteristics, see [1]).

The degree of terrestrial weathering is determined with the optical microscope employing the methods of [2]. During the process of terrestrial weathering, oxidized iron phases replace metals and sulfides, often forming rims around primary mineral grains. Under reflected light, metal, sulfides, and iron-oxides can be distinguished to determine the percent of mineral replacement by weathering. Further observations of terrestrial weathering can be made in transmitted light, where the terrestrial oxides appear rusty or red in color, often forming a network of veins through the sample.

The stage of shock is another attribute that is determined primarily by optical microscopy. Methods such as those in [3] are typically used, identifying characteristic shock features such as planar fractures in olivine or recrystallization of glassy material.

Although complete characterization of meteorites also necessitates the use of other techniques such as the scanning electron microscope, electron microprobe, or ion microprobe, optical qualities must be characterized before the sample is carbon coated. Digitally recording optical information allows for further analysis of this data after the sample is coated, and the high-resolution optical imaging provides direction for these other analytic techniques. The system we describe here is no replacement for a skilled petrographer with active control of a petrographic microscope, rather it is a valuable tool for static image documentation. An automated system to record very large image mosaics at fine (e.g. ∼1°) intervals of stage rotation with polarized light is theoretically possible, but practically extremely slow and data-intensive.

Commercial systems for automated slide scanning for biological applications are available [4], however these are specialized, expensive systems that cannot be adapted to suit a meteoriticists needs regarding, for example, illumination options (e.g. cross-polarized light) and variable types of sample mounts. Additionally, these commercial systems run compiled, closed-source software that cannot be modified for different applications. For geological samples, Open University has developed a petrological virtual microscopy system ( [5], http://virtualmicroscope.org/) which is used primarily for educational purposes.

In this paper we describe an automated microscopy setup for the acquisition and creation of optical micrographs at ∼385 nm/pixel for typical meteorite thin sections up to a few cm in size, resulting in a total image size of up to a few billion pixels for the entire thin section. Our system employs focus-stacking and exposure fusion to properly image variable-height and high-contrast samples. Using free software available for Windows and Macintosh, these images can be viewed easily in a web browser, allowing for smooth zooming and panning. The software for our system is entirely open-source and can be improved upon and adapted for a wide range of applications. Our gigapixel imaging system can be used on any optical microscope with a computer-controlled stage and modern consumer-grade digital camera.

Methods

The optical microscope we use is a Nikon AZ100 Multizoom Research Microscope System with 0.5 ×, 1 ×, and 4 × objective lenses. We chose this microscope for its capabilities as a multi-purpose instrument – its zoom capabilities and long working distance objectives facilitate picking presolar grains from meteorite acid-resistant residues [6] and sample preparation for returned cometary material [7]. The microscope is capable of both reflected and transmitted light, in addition to plane-polarized and cross-polarized illumination using custom-designed polarizing-film holders.

We installed a Prior Optiscan II motorized stage and focus control on the AZ100. This stage can be moved with 1 μ m step size in x- and y- dimensions with better than 5 μ m repeatability. The motorized focus control provides z-axis movement with step size as small as 100 nm. The stage can be computer controlled via a controller box and RS232 serial port connection.

We attached an LMScope DSLR photo adapter to a Canon T2i digital SLR for use in the AZ100’s photo port. This camera captures still images at 5184 × 3456 pixels (∼18 megapixels), videos at 1920 × 1080 pixels at 30 fps, and a live video feed at 1056 × 704 pixels (0.7 megapixels). The camera can be computer controlled through a USB 2.0 connection using the libgphoto2 library and the gphoto2 command-line client.

The microscope and camera are controlled with a PC running Fedora Linux version 3.6.6-1.fc17.x86_64. The PC is a modern machine circa 2012: AMD FX-8120 Eight-Core processor with 16 gigabytes of RAM (useful for blending large images).

The microscope, stage, camera, and computer described here can be easily replaced with other models which are compatible with the software used in our gigapixel imaging system. The microscope must support the digital camera and stage, the stage must be capable of 3-axis computer-controlled movement, the camera must be compatible with libgphoto2, and any computer running a standard Linux distribution will work.

Control of camera and stage

Our goal is to acquire high-resolution optical images of meteorite thin sections. The Canon T2i camera can capture ∼18 megapixel still images, but capturing and downloading the full-size images is slow (∼7/s per image) and puts stress on the camera’s mechanical shutter and mirror mechanism. The optical plane of the microscope is flattest at full zoom (using the 4x objective and 8x zoom on the AZ100), meaning the entire 407 μ m × 271 μ m field of view is in focus. At this magnification the 1056 × 704 pixel preview frame yields 385 nm/pixel, which is beyond the diffraction limit, so a larger image size would not yield a significantly more detailed picture. Preview frame capture has the added benefit of being very fast (∼0.2 s/image) and does not stress the camera because it requires no mechanical movement of the camera’s shutter or mirror.

To capture individual images in a mosaic plane, the camera and stage should communicate with each other. We facilitated communication between the camera and stage through the Tcl scripting language. The camera is controlled inside a gphoto2 shell and the microscope stage is controlled inside a minicom terminal emulator. The Tcl script uses send to move the stage to a location and listens for its confirmation of reaching that position with expect. After the confirmation is received by expect, the image is captured by sending the appropriate command to the gphoto2 shell using send, and then the script listens for confirmation that the image was captured with expect.

The camera can occasionally freeze during the acquisition of a large mosaic. Rather than trying to prevent freezing, we simply listen for an error in the gphoto2 shell and when there is one, we reboot the camera, reestablish communication, and continue capturing images.

Our method of capturing mosaics is very efficient and robust since there are no built-in wait times– the images are captured as fast as they can be and the camera and stage are always synchronized.

The microscope-camera-stage setup is shown in Figure  1.

Figure 1

Microscope setup. Photograph showing the microscope (Nikon AZ100), camera (Canon T2i), automated stage with controller (Prior Optiscan II), vibration isolation table, and computer.

Focus stacking

The depth of field for the 32 × magnification of our microscope used to acquire individual mosaic images is only several μ m. Small errors in defining the plane of the sample to be imaged can lead to acquiring out-of-focus images. Additionally, unpolished sections may contain topography outside of the depth-of-field that the user wishes to record in a single image. Transmitted light imaging for thin sections, ∼30 μ m thick, can show different features depending on what depth in the section is in focus. For these reasons it is frequently useful to acquire a “focus stack” of images at each point in the plane of the microscope’s stage. A focus stack combines the in-focus parts of a number of images captured at different focal plane depths into one image.

We employ the command-line utility enfuse which uses the Mertens-Kautz-Van Reeth exposure fusion algorithm [8] to merge the in-focus regions of a stack of images taken at various depths into a single image. This algorithm assesses the quality of each pixel based on certain characteristics (most commonly: saturation, contrast, and exposure) to determine the amount a given pixel contributes to the final image. Since in-focus regions of an image have high contrast between neighboring pixels and out-of-focus regions have low contrast, we assign full weight to pixel-by-pixel contrast in enfuse and zero weight to saturation and exposure.

It typically takes ∼3 seconds to combine a focus stack of ten 1056 × 704 pixel images into one focus-stacked image using enfuse. An example of focus-stacking for an unpolished meteorite is shown in Figure  2.

Figure 2

Focus stacking. Single-focus optical micrographs at varying focus depths of an unpolished piece of the achondritic diogenite meteorite Tatahouine are shown at lower-left, upper-left, and upper-right. The focus-stacked image, which combines 40 individual images taken at steps of 5 μ m in focus, is shown at lower-right.

Exposure fusion

What we see through the eyepieces of a microscope spans a larger dynamic range of brightness than what is possible to capture by a single digital camera exposure (typically 8-bits per color channel in the processed output image). This is also the amount of dynamic range available on a typical computer monitor. However, we can capture several digital images with varying exposure compensation and use an algorithm to assemble these into one image with a perceived increase in dynamic range.

At each point in the plane of the microscope’s stage, in addition to focus-stacking as described above, we acquire a range of images with different values of exposure compensation, typically −2, −1, 0, + 1, and + 2 eV. We use exposure blending as employed by enfuse to merge the best-exposed (not too dark and not too bright) parts of these individual images together into one well-exposed image. A single field of view in a meteorite thin section can contain metal grains (very bright in reflected light) and fine-grained matrix (very dark in reflected light). A single digital image capture will saturate the pixels in either or both the metal (recorded as pixel value = 255,255,255) and matrix (recorded as pixel value = 0,0,0), whereas an exposure-fused image will record information in both these constituents.

We perform exposure fusion on images that have already been focus stacked. It typically takes 3 seconds to perform exposure fusion on a stack of five (−2, −1, 0, + 1, and + 2 eV) 1056 × 704 pixel images. An example of exposure fusion is shown in Figure  3.

Figure 3

Exposure fusion. Optical micrographs of the CO3 meteorite EET 90043 with exposure compensation values of −2, 0, and + 2 eV are shown at lower-left, upper-left, and upper-right, respectively. The exposure-blended image, which is blended from five individual images with exposure compensation values of −2, −1, 0, + 1, and + 2 eV is shown at lower-right. The five individual images are each focus-stacked.

Gigapixel mosaic

Using the optical microscopy system described above, we have written Matlab code that writes a Tcl script to acquire the individual mosaic images. The image acquisition and mosaic creation is entirely automated. The only input required from the user is to set the mosaic coordinates, decide how many focus steps and exposure compensation values to acquire, and name the final image. The user then starts the script with a single terminal command, and the mosaic is created and posted to a webpage (see “Viewing the mosaic”) when it is finished.

Defining the mosaic points

To define the plane over which the mosaic is to be captured, the user focuses the camera view on the four exterior corners of the sample by moving the stage with the joystick. Using Matlab, for each corner we query the Prior stage for its position. We fit a plane to these four points using the Matlab function lsplane then we compute the grid of points (x,y,z) over which the stage moves. The spacing of points in the grid depends on the fixed field-of-view of each image and the predefined image overlap (40% overlap in both dimensions is a good compromise between speed and ensuring each image has overlapping features). We raster scan the mosaic points to avoid large movements of the stage.

Figure  4 shows grid points for an example mosaic.

Figure 4

Mosaic grid. The user defines the four corners of the plane over which the mosaic images will be acquired, as indicated by black circles. The mosaic grid is then calculated based on a least-squares fit plane to these four points. Each vertex of the grid indicates the position at which an image will be acquired (additional images at varying focus depths are gathered at each vertex point if the focus-stacking option is selected).

Stitching the mosaic

Individual mosaic images are captured with filenames indicating their x- and y- coordinates within the total mosaic. The images then need to be stitched together and blended to produce one large seamless image.

For stitching we use the free, open-source code package Hugin ( http://hugin.sourceforge.net). The major steps in building the final stitched image are performed by various functions in the Hugin package: matching features (“control points”) in the individual image tiles (using cpfind), optimizing the positions of the individual photos in the final mosaic based on matching features in overlapping images (autooptimiser), remapping image tiles into components of the final image (nona), and blending remapped image tiles together to create the final mosaic image (enblend and multiblend). The final stitched and blended mosaic image shows neither stitching seams nor brightness changes across the mosaic. All of these functions (along with others that crop the mosaic, refine control points, etc.) are scriptable Linux shell commands and many can take advantage of multiple processor cores on modern computers.

Assembling mosaics is computationally intensive and time-consuming. The remapping step (nona) for assembling thousands of individual image tiles into a gigapixel mosaic requires the creation of a temporary gigapixel image for each image tile. This would require unrealistic computer resources and computation time. Instead, we stitch together each row of image tiles after that row of image tiles has been acquired. The row is stitched by background processes while the next row’s images are being acquired. Prealigned Hugin project files (.pto files) for each row, and for the final mosaic, are written by the main Matlab script to simplify feature-matching between images. After all of the image tiles have been acquired and row images stitched, we then stitch together these individual rows to create the final mosaic. A row of 60 images takes ∼13 min to stitch (conveniently, a bit shorter than the time it usually takes to acquire a row of images, each with 10 focus steps and 5 exposures). The final gigapixel mosaic (60 rows) takes ∼3 hrs to stitch.

Our strategy is to be as error-tolerant as possible in creation of the optical mosaic image. A pair of images fails to stitch together usually because overlapping features cannot be identified. In this case, we try four alternative matching strategies to find matches. If a row of images fails to stitch together (e.g., if no matching features could be found between two rows) our script copies the image alignment parameters of a neighboring row that was successfully stitched.

Viewing the mosaic

The final stitched mosaic image is a ∼1 gigapixel tiff image that is ∼3.0 gigabytes in size. Viewing such an image on even a modern computer would consume a lot of resources. We would like to smoothly pan across the image and zoom which is difficult to do if the entire image is loaded into a graphics program. Additionally, we would like to share this large digital image easily, a difficult task for such a large file.

There are several options for viewing gigapixel-sized images over the web. We chose Microsoft HD View ( http://research.microsoft.com/en-us/um/redmond/groups/ivm/HDView/) because it is free to use for both creation and viewing of large images, and it has a simple Windows command-line interface for creating HD View content, hdmake. HD View uses tiles at different resolutions (created by hdmake) and only displays the tiles within a particular field of view. With this software we can easily view our gigapixel mosaics and share them with collaborators. Zooming and panning is smooth and it is possible to save a specific view as a web link.

The HD View Control provides javascript callback functions that can be called when the zoom level of the image is changed. We use this callback to create a scale bar visible at the bottom of the web page that changes as the user zooms in and out of the image. This is possible because the per-pixel physical dimension of the final image is fixed and known (385 nm).

An example of a webpage showing a meteorite mosaic in HD View is shown in Figure  5.

Figure 5

HD View. Web page screen capture of Microscoft HD View which is used to smoothly display the gigapixel images online. The scale bar shows 500 μ m and changes as the user zooms in and out of the image (or resizes the browser window). The “Save Specific View” button displays a URL that can be copied and returns the view to the location on the large image and zoom level currently displayed. Buttons in the top-right have the following functions— Hand: translate the image view; Line: toggle between the following tone adjustments: no adjustment (horizontal line), exposure only adjustment (diagonal line), compress or expand the dynamic range as necessary (curve); Grid: toggle between auto lens and fisheye lens (this should be left as the grid icon, auto lens); Fullscreen: toggle between normal window and fullscreen (scale bar and “Save Specific View” button are not visible in fullscreen); Home: return to the fully zoomed out view, so the entire image is visible in the browser window; Magnifying glass: zoom in and out of the image. With a scroll wheel mouse, one can zoom into and out of a place on the image by scrolling the wheel forwards or backwards while the mouse is over the image.

Results and discussion

We have imaged several meteorite thin sections of various types using this gigapixel microscopy setup. These images can be viewed online at the follwing url: http://www.higp.hawaii.edu/~ogliore/mosaics/. Sample images are shown in Figures 6, 7, 8.

Figure 6

NWA 7402 meteorite thin section. Optical micrographs of a currently unclassified chondrite, provisional name NWA 7402. At low magnification (top), the heterogeneity of terrestrial weathering is apparent. Patches of rust-colored alteration to the rock caused by iron oxidation distinguishes regions of extensive weathering from more pristine gray-toned rock. At higher magnification (bottom), one is able to identify individual weathering veins, and can assess whether metal grains have been completely or partially oxidized.

Figure 7

GRO 95577 meteorite thin section. Reflected light optical micrographs of CR1 chondrite GRO 95577. Brighter materials in reflected light tend to be metal or opaque minerals such as oxides and sulfides. At low magnification (top) we can see that the opaque minerals are mostly fine-grained and scattered thoroughly about the groundmass, an indication that this meteorite has experienced heavy aqueous alteration. Higher magnification (bottom) reveals zoning in magnetite (an iron oxide), due to variation in chemical composition. Such zoning indicates that the mineral grew in a disequilibrium setting.

Figure 8

MET 00426 meteorite thin section. Transmitted light optical micrographs of CR3 chondrite MET 00426. The low magnification view (top) allows for primary modal abundance analysis for large features such as chondrules, clasts, and matrix material. The transmitted light also helps to distinguish the types of chondrules (e.g. porphyritic olivine, radial pyroxene, etc.). This image above shows a high abundance of chondrules, with much less groundmass or matrix material. The higher magnification image (below) allows the user to observe crystal morphology, such as in this euhedral olivine grain. Here, the mesostasis surrounding the grain is glassy and shows no signs of recrystallization, indicating that this sample has experienced only minor shock or aqueous alteration.

Availability of mosaic creation software

We provide the scripts used to acquire and assemble mosaics as open source software (free to use, modify, and share). The code is stored at http://www.higp.hawaii.edu/~ogliore/mosaics/software/. Familiarity with Linux scripting, Matlab, and the Hugin package is recommended to use and modify the software effectively.

Conclusions

We have described a relatively inexpensive and versatile setup to acquire and view extremely high-resolution images of meteorite thin sections. All of the computer code needed to run this system is freely available for use and adaptation at http://www.higp.hawaii.edu/~ogliore/mosaics/software/.

Our gigapixel microscopy system can, of course, be used for many other purposes. As another application for imaging of extraterrestrial samples, this system can be combined with a feature identification algorithm to search for impacts on foils flown on NASA’s Stardust mission [9]. Obviously, petrographic thin or thick sections of terrestrial rocks can be imaged. In neuroscience, our gigapixel microscopy system can be used to acquire high-resolution images of brain tissue [10].