11 Night Heron Drive, Stony Brook, NY 11790, USA ● 631-882-1415
The PixonÒ Method: A New way of Looking at Data
Contact: info AT pixon.com
Website: http://www.pixon.com
NEW!! We now include software simulations of the performance of our full color PixonVision hardware card that has just been completed under NASA SBIR Phase 2 support (program complete, June 2002). Go there.
Last updated 12 September 2002
What Does the Pixon Method Do?
PixonVision--Quick Pixon Video Hardware
Examples of Pixon Method Image Reconstruction
2. Image Reconstruction of 'Lena'
4. Astronomical Far-Infrared Satellite Imaging (IRAS imaging of M51)
5. Astronomical Near-Infrared Satellite Imaging (Hubble NICMOS imaging of the Pistol Star)
6. Astronomical X-Ray Satellite Imaging (Chandra image of the Crab Nebula)
7. Astronomical Gamma-Ray Satellite Imaging (OSSE imaging in the direction of Virgo)
11. Highly Sampled Battle Scene Imagery
13. Super-resolved Closely Space Objects
14. Subdiffraction Limited Imaging
15. Subdiffraction Resolution Electron Microscopy
PixonVisionTM Examples: performance simulations of our real-time color video hardware
Movies: real-time video image reconstructions for radar
Pixon LLC was established
by the inventors of the powerful Pixon method, which represents a new and
innovative way of modeling an image in terms of its information content. This
technique allows remarkable new capabilities in the fields of image
reconstruction and data compression. Recently, an Accelerated Pixon method has
been developed that is faster than the original method by many orders of
magnitude. In addition to the Accelerated Pixon method, there is now a Quick
Pixon method that is even faster. The Quick Pixon method sacrifices local
photometric accuracy for the sake of computational speed. Nonetheless, its
photometric performance rivals that of Wiener-filter Fourier deconvolution and
produces artifact-free images. With the addition of special-purpose hardware,
the Quick Pixon method is expected to reach real-time image restoration at
video speed.
A patent was approved on 15
June 1999 (US Patent No. 5,912,993) for the original Pixon method. A
continuation in part patent covering the numerical acceleration of the method
has also been granted. Two other
patents are currently pending. The
first covers the generalization of the Pixon method to the algerbraic modeling
of complex systems (the AlgebronTM
method). The second application covers
the even faster Quick Pixon method (see below).
As the exclusive licensee
of this technology, Pixon LLC is actively looking for business opportunities
(sub-licensing, strategic partnerships) in the areas of medical imaging,
microscopy, industrial inspection, military applications, and commercial
satellite imaging. We have develop of a hardware implementation of the Quick
Pixon method under NASA SBIR sponsorship that can perform video-rate (30 Hz)
reconstruction of 720x480 pixel images. We are also developing a hardware
implementation of the Full Pixon method under US Missile Defense Agency SBIR
support. In addition to image
processing hardware, Pixon offers versatile commercial software for a wide
range of applications.
What Does the Pixon Method Do?
The Pixon method has very
broad applications for geometric-based data sets such as images. The first application
of this method was to image reconstruction in astronomy. However, it provides
strong benefits to all kinds of imaging systems, e.g., medical, geophysical,
surveillance, radar, and sonar, as well as to data archiving and communication,
e.g., image compression, and voice and video encoding and playback. In order to
acquaint you with the advantages provided by the Pixon method, we first
describe the general problem addressed by this technique, then the method by
which the solution is obtained. This is demonstrated with several practical
examples of Pixon reconstructions shown below.
The problem: How do you fit a model to the signal in a general
data set, such as an image, when its functional form is not known beforehand?
Tracing a curve through the data would be fitting the noise and would not apply
to another realization of the signal. A forced parametric fit to a small set of
arbitrarily chosen basis functions is typically unsatisfactory. This is either
because the functions used may not be suitable, or too many parameters are
used, again leading to noise fitting. To compound the problem, instrumental and
atmospheric limitations typically result in blurring of the image, and
deblurring (deconvolution) techniques tend to amplify the noise, particularly
at high frequencies, exacerbating the difficulties of model fitting.
The solution: The Pixon method provides a solution to this
problem. First, it assigns a priori probabilities to different signal models,
so a function that zigzags through noisy data has low probability, while a
smooth function has a high a priori probability. Then, in Bayesian
fashion, it maximizes not the usual maximum likelihood of the data, given the
model, but the product of that likelihood and the prior probability of the
model.
The Pixon Method: The Pixon method provides a solution to this
problem. First, it develops a set of basis functions into which the images is
decomposed. This set of basis functions (Pixon elements) depends on the
problem. It is composed of a large set of elements, whose number typically
greatly exceeds the number of data points, is selected to be rich enough to
encompass all the foreseen possibilities, yet does not include functions which
would zigzag through noise. The Pixon method then demands minimum complexity,
or Algorithmic Information Content, as it is known in information theory. This
means that it selects the smallest number of basis functions that still
adequately fit the data. Information theory shows that the most efficient
(minimal description) model of the signal also does the best job of separating
it from noise. This is clearly true since if the data are adequately fit with n
parameters, an additional parameter must begin to fit the noise.
The innovation behind the
breakthrough of the Pixon method is its new method of quantifying the
information content in a data set. Each basis function, or Pixon-element, is a
quantum unit of the Algorithmic Information Content. The collection of Pixon-elements
represents the minimum parameterization necessary to describe the data within
the accuracy allowed by the noise, and constitutes the natural representation
of the data. As demonstrated below, use of this representation has tremendous
advantages for image reconstruction/restoration and image compression over
other, arbitrary, representations. Relative to the performance of other
methods, the Pixon method can increase the spatial resolution by factors of
several, increase the sensitivity to low contrast sources by orders of
magnitude, and ensure robust rejection of spurious sources. For image
compression, the Pixon method can be shown, from first principles, to provide
optimal signal compression.
An important problem in a
number of fields is the reconstruction/restoration of imaging data. For this
application one seeks the best estimate of some underlying quantities which
have been encoded into a data set, typically with some image deterioration, and
almost always with noise. For example, a picture may become blurred by
unfocused optics or the passage of light through a turbulent atmosphere, and
noise may be added by the recording device. From the blurry picture one seeks
the best estimate of what the unblurred object looks like.
Use of a Pixon basis allows
a very efficient mathematical inversion of the image encoding. This is because
in most data sets, there is noise that is unrelated to the properties of the
encoded image, or there are details, which are of no interest. Inversion algorithms
that do not take this into account, overestimate the required number of
parameters required to model the data and attempt to invert the problem with
all of them. The additional parameters are underconstrained by the data and
result in modeling of spurious sources. Furthermore, the extra parameters
increase the numerical difficulty of extracting the encoded signal. By
contrast, the Pixon representation uses exactly the correct number of
parameters, each of which is fully constrained by the data. This results in
optimal information extraction and the elimination of spurious sources.
Our implementation of the
Pixon method uses a multi-resolution language to efficiently
describe the spatial structure of generic images. This method has now been
tested for a wide variety of imaging applications by the inventors as well as
numerous outside groups (e.g., the University of Hawaii, University of Texas,
University of California at Riverside, the University of Chicago, the National
Radio Astronomy Observatories, and the Lawrence Livermore National Laboratory).
In all cases tested so far, the performance of the Pixon method far exceeded
those of any other method.
As described above, the
Pixon representation of an image is the one with the smallest number of
parameters necessary to completely specify the signal in the data.
Consequently, this is the optimal compression of the signal.
The Quick Pixon method is
an extremely fast, non-iterative Pixon method. While the Quick Pixon method
doesn't precisely preserve local radiometric accuracy, it is of the same
accuracy as that of Wiener-filter Fourier deconvolution. Moreover, the Quick
Pixon method is in the same speed class as linear methods. In addition, the
method is highly parallelizable, and it can achieve video rate image
reconstructions of modest sized images (720x480 pixels) with dedicated
hardware—see below.
PixonVision—Color Quick Pixon Hardware
In December of 1999, Pixon
LLC was awarded a NASA SBIR grant to design dedicated hardware to perform quick
Pixon image reconstructions at video rates.
This resulted in a successful Phase 2 program that completed in June
2002. The product of this effort was
the VP-100 video card, a prototype video board that enhances blurry, noisy video
in real-time. Typical enhancement include factor of 3 to 4 reduction in noise
and factor of 1.5 to 2 increase in linear spatial resolution. The VP-100 was designed and built in
collaboration with DigiVision, Inc. of San Diego, a recognized leader in real-time
video hardware. Examples of
representative color image processing by the quick Pixon video method appear
below.
Examples of Pixon Method Image
Reconstruction
To demonstrate the great
advantages of the Pixon method for practical problems, we present below a
number of sample image reconstructions.
Example 1: A Synthetic Problem
In the first example of
Pixon image reconstruction we show a synthetic data set in which the true,
unblurred image and the blurring function (point-spread function, or PSF) are
known. These functions are shown in Figure 1. To the blurred image we add
Gaussian noise (also shown) to create an input data set with a peak signal to
noise ratio of 100. The Pixon reconstruction, along with the residual error, is
shown in the third column of Figure 1. For comparison, the results of a maximum
entropy reconstruction are shown in column 4. The algorithm used for this
reconstruction is the powerful MEMSYS 5 algorithm, which is probably the
highest performance, commercial maximum entropy package available. Nick Weir, a
recognized maximum entropy expert, performed the MEMSYS 5 reconstruction in
Figure 1, supplementing it by his multi-channel method, which is known to
enhance the performance of the MEMSYS code.
|
|
Click to enlarge |
Comparison of
the Pixon and MEMSYS 5 image reconstructions for a synthetic data set for which
the true (unblurred) image, the blurring function (PSF), and the input noise
are known a priori. Top row (from left to right): images of (1) the true image,
(2) the blurred and noisy input data to the reconstruction, (3) the Pixon
reconstruction, and (4) the MEMSYS 5 reconstruction. Middle row: surface plots
of the top row images. Bottom row (from left to right): images of (1) the PSF,
(2) input noise, and (3) residuals for the Pixon and (4) MEMSYS 5
reconstructions. As can be seen from the figure, the MEMSYS 5 reconstruction
has strong signal-correlated residuals and systematically underestimates the
source strength. In addition, it is plagued with many spurious sources. By
contrast, the Pixon reconstruction has statistically perfect residuals and
hence faithful source strength determination. Furthermore, all sources are real
since the Pixon method robustly rejects false sources.
Example 2: Image Reconstruction
of "Lena"
This example compares the
performance of the Pixon method with reconstructions by other techniques for a
famous image reconstruction test image: "Lena". For this example the
true undistorted image of Lena was blurred with a Gaussian blurring function
with a full width at half maximum of 6 pixels. To this Gaussian noise was
added. The resulting input data set had a maximum signal to noise ratio of 40
and a minimum signal to noise ratio of 20. In addition to the Pixon method
result, we shown results for a Wiener filtered Fourier deconvolution and a
Maximum Likelihood reconstruction. As can be seen, relative to the other
methods, the Pixon reconstruction is extremely artifact free. Unlike the
results from the other methods, all of the features in the Pixon method
reconstruction have firm statistical evidence of their existence and the
measured brightness of these features are statistically faithful. For this
example the Pixon method achieved a resolution improvement of a factor of 3 over
the input data.
|
|
Click to enlarge |
Comparison of
the performance of the Pixon method to Wiener filtered Fourier deconvolution
and Maximum Likelihood deconvolution for the "Lena" image.
Example 3: Quick Pixon
Performance
In this example we present
a comparison of a Quick Pixon method reconstruction and the accelerated Pixon
reconstruction of the "Lena" image presented in example 2. As can be
seen, even though the Quick Pixon method does not accurately preserve local
photometry, it looks visually quite similar to the full Pixon method
reconstruction. Hence the Quick Pixon method is an excellent choice for image
identification and classification applications. This is especially true given
its great speed advantage. For the example here, the Quick Pixon reconstruction
took only 200 second for the 512x512 pixel image of "Lena" on a
moderately fast desktop computer (200 MHz Pentium Pro).
|
|
Click to enlarge |
Comparison of
the performance of the Quick Pixon method and the full Pixon method. As can be
seen, despite the approximate local photometry of the Quick Pixon method, it
gives results that are visually quite similar to the full Pixon method. Since
it is in the same speed class as linear methods, this makes the Quick Pixon
method an excellent choice for image identification and classification uses. It
provides high image reconstruction speed without introducing the artifacts
normally associated with non-Pixon method techniques.
Example 4: Astronomical Far-Infrared Satellite
Imaging
The fourth example uses
data from the Infrared Astronomical Satellite (IRAS) at the far-infrared
wavelength of 60 microns. This instrument gathered "imaging data" by
scanning discrete detectors across the sky and recording their temporal
responses. The detectors were rectangular in shape with the narrow dimension
being in the scan direction and the wide dimension in the cross-scan direction.
This makes for rather poor images. (IRAS was initially intended as a survey to
detect point sources, not as an imaging instrument.) As can be seen from the
reconstruction (performed by the inventors), the Pixon method dramatically
improves the resolution of the IRAS instrument (by a factor of 20 in the cross
scan direction). It also increases the sensitivity of IRAS by a factor of
roughly 100 over the best previous efforts to detect faint sources from the
data. There is no other survey at 60 microns against which the reconstructions
may be compared, but the overlap with high-resolution radio image contours, and
the easy identification of many of the sources revealed in the reconstruction,
demonstrates the reliability of the method. For comparison to competing method,
click here.
|
|
Click to enlarge |
A Pixon reconstruction
of the 60-micron IRAS satellite survey scans of the galaxy M51 (''whirlpool
galaxy''). The resolution of the reconstructed image is 20 times finer than the
raw co-added scans (in the cross-scan direction) and the sensitivity to the
detection of faint sources is improved by a factor of 100. Overlaid on the
reconstruction are the 5 GHz radio contours of van der Hulst et al., Astronomy
& Astrophysics, 195, 328 (1988). Also identified are a number of
optical sources and Ha knots.
Example 5: Astronomical Near-IR Imaging: Hubble
NICMOS Imaging of the Pistol Star
In this example we present results
from the NICMOS instrument aboard the Hubble Space Telescope. The NICMOS
instrument is a near infrared camera. Its optical performance is nearly perfect
and is limited only by the fundamental diffractive properties of light. Indeed,
the diffraction pattern from the telescope is strongly evident in the raw data.
The data shown are of the
"Pistol Star". The Pistol star is the most luminous star ever
discovered, lying near the center of our own Milky Way galaxy. While the NICMOS
instrument provides near optimal performance from the Hubble Space Telescope,
diffraction effects clearly hide much of the detail of the underlying nebula.
The Pixon method provides the capability of correcting for the natural
diffractive properties of light and allows all the diffracted light to be
regathered back to the points from which they came, the stars. This reveals the
properties of the underlying nebula in unprecedented detail.
|
|
Click to enlarge |
Pixon
reconstruction of near infrared imaging of the "Pistol Star", the
most luminous star ever discovered. The raw image shows the strong spreading of
the light around stars due to diffraction from the Hubble Space Telescope.
Rather than being a problem with the telescope (as was suffered before Hubble
was repaired), the diffraction spikes around the stars show the high quality of
the NICMOS instrument with which the data was taken. These artifacts result
from the fundamental nature of light. No instrument can avoid them.
Nonetheless, the Pixon method offers a solution to this fundamental limit in
our ability to see clearly. Through the use of minimal complexity modeling, the
Pixon method is able to reveal the underlying structure with greatly enhanced
clarity. It sorts out the light and puts it back where it belongs. Consequently
all the stars are properly brought back to points of light and the structure of
the underlying nebula is seen in unprecedented detail.
Example 6: Astronomical X-ray Satellite
Imaging: Chandra Imaging of the Crab Nebula
The next example presents a Pixon reconstruction of imagery of the Crab nebula obtained with NASA's Chandra X-ray satellite. The energy coverage of this image is the entire bandpass of the ACIS instrument. The Crab pulsar lies in the middle of the image and produces a very over exposed image that is "blacked out" in both the data and the reconstruction.
|
|
Click to enlarge |
A Pixon reconstruction of broad-band imagery of the Crab nebula taken by NASA's Chandra satellite. Because the Crab is so bright, it was necessary to image it through the ACIS spectrometer at zeroth order. Consequently the sharp narrow rays coming out of the crab are spectra of the bright central object (small black circle in center of image where the data is over-exposed) in the orders n=-1 and n=1. The Crab image is dominated by noise due to Poisson photon counting statistics. The reconstruction improves the resolution and fits an optimal minimal complexity model to the structure of the Crab's emission, thereby avoiding the "eye-fooling" signal dependent noise in the data and revealing a wealth of fine spatial structure.
Example 7: Astronomical Gamma-Ray Satellite
Imaging: OSSE Imaging in the Direction of Virgo
In the next example, we
present gamma-ray imaging of objects in the direction of the constellation of
Virgo from the Oriented Scintillation Spectrometer Experiment (OSSE) aboard
NASA's Compton Gamma Ray Observatory satellite. Since g -ray imaging is so difficult, and the sources are so
faint, the resolution of g -ray images is very poor (the
pixels are very large) and the images are very noisy. (All images must be
processed by some method in order to see anything at all.) The first image,
presented in the top panel, is that produced by the Non-Negative Least Squares
(NNLS) method developed by the University of California at Riverside. The
second is a variant of the Pixon method developed by UC Riverside in
collaboration with the inventors. As can be seen, the NNLS image is incredibly
noisy. From this image one would be forced to say that there is no clear
evidence of any real sources in this direction. The Pixon reconstruction
clearly and sensitively reveals two prominent sources, the quasar 3C273 and the
active galaxy NGC 4388.
|
|
Click to enlarge |
OSSE
gamma-ray images (50-150 keV) in the direction of the constellation Virgo. The
dark region contains the entire field with significant exposure time. Regions
with insignificant exposure are shown as white surrounding area. The standard
non-negative least squares (NNLS) reconstruction produces essentially a random
noise field, while the Pixon reconstruction reveals two sources in this
direction, 3C273 and NGC 4388, standing out prominently against a background
with a mild brightness gradient. (The galaxy M87 also falls into the pixel
containing NGC 4388, but it is not expected to contribute significantly to the
gamma-ray emission.) Dave Dixon, UC Riverside, performed both the NNLS and the
Pixon reconstructions.
Example 8: X-Ray Mammography
We present in Figure 7 the
results of a Pixon reconstruction of a standard mammogram phantom from the
American College of Radiology. In this example a 400 micron diameter fiber was
placed in material with X-ray absorption properties similar to the human breast.
The raw X-ray mammogram is displayed in the left-hand column. The right-hand
column presents the Pixon-filtered data. As can be seen from the figure, the
Pixon-filtered image greatly increases the ability to see even the very low
signal to noise features in this mammogram.
|
|
Click to enlarge |
Pixon
reconstruction of a standard X-ray mammogram phantom from the American College
of Radiology. Left panel: raw X-ray mammogram. Right panel: Pixon
reconstruction of the mammogram data. As can be seen from the figure, the Pixon
reconstruction provides excellent resolution improvement and noise rejection
and allows the detection of even very low signal-to-noise features in the
mammogram.
Example 9: Nuclear Medicine
Imaging
In this example, we present
Pixon processing of data from nuclear medicine applications. Shown is a
gamma-ray image of the chest of a patient that has taken a radioactive
substance. Clearly evident in the image is the person's collarbone and spine.
The bright spots are from areas in the body in which the radioactively tagged
material has accumulated.
As can be seen from the
figure, the Pixon method provides significantly enhanced resolution and greatly
reduces the noise caused by photon counting statistics in the raw data. The
enhanced performance provided by the Pixon method both increases the utility of
such imagery and allows diagnosis to be carried out at lower radiation dosages
to the patient.
|
|
Click to enlarge |
Pixon
reconstruction of digital nuclear medicine imagery. In this example the raw
image is significantly blurred and plagued with noise artifacts from photon
counting statistics. The Pixon reconstruction improves the resolution and
reduces the noise. This enhances the diagnostic capabilities of such imagery
for radiologists and physicians and allows a reduction in radiation dosage for
the same image quality.
Example 10: Aerial
Reconnaissance
In this example, we present
an aerial photograph of a portion of New York City. The image has been blurred
and noise added to simulate data that might be obtained from high altitude
aircraft or a satellite. Compared are the results for the Quick Pixon method,
and non-negative least square (NNLS) and Wiener-filter Fourier deconvolution.
As can be seen from the figure, the Pixon method gives the best results. It has
high spatial resolution and a very low artifact level. The absence of strong
artifacts is especially useful in distinguishing structures in crowded portions
of the image.
|
|
Click to enlarge |
Comparison of
the performance of the Quick Pixon method to a non-negative least square (NNLS)
and a Wiener filter Fourier deconvolution. Shown is an image of a portion of
New York City that has been blurred and noised to simulate data that might be
taken from high altitude aircraft or a satellite. The blurring PSF was a
Gaussian with FWHM of 4 pixels. The additive noise was Gaussian and provided a
data set that had a maximum SNR of 38 and a minimum SNR of 4.5.
Example 11: Highly Sampled
Battlescene Imagery
Optimal resolution gain,
e.g. super-resolution, is only achievable with excellent sampling of the
data. To demonstrate this effect, we compare in this example the reconstructions
achievable with sampling corresponding to 2.5 pixels per PSF Full Width Half
Maximum (FWHM) and 10 pixels per PSF FWHM. High signal to noise ratio
(SNR) is also necessary for optimal performance. The SNR values for each
of the two examples is 500. The quality of the results begin to noticeably
decline for SNR values below about 200.
Super-resolution is achieved in both Pixon reconstructions. However the poorer sampling in the 2.5 pixels per PSF FWHM example prevents a clear image of the main gun of the tank. Rather it is lost in the clutter associated with background objects. The better sampled data provides a good, identifiable image of the main gun. It is now clearly separated from the background clutter. In addition, the two objects atop the tank are also more easily identified as soldiers.
|
|
Click to enlarge |
The Pixon method achieve sub-diffraction resolution imaging of a tank in a typical battle scene. This level of resolution requires both good signal-to-noise ratio data and excellent PSF sampling. The PSF sampling in this case was 10 pixels per FWHM (see PSF inset in the image of the blurred data) and the SNR was about 500 per pixel. This capability reveals detail, e.g., the tank's main gun, otherwise undetectable in a more poorly sampled image (compare 2.5 pixel/FWHM reconstruction to 10 pixels/FWHM reconstruction).
Example 12: Radar Imaging
In the final example we
display results for inverse synthetic radar (ISAR) imaging. Imaging radar uses
reflection of broad band radar pulses. The returned pulse amplitudes and phases
as a function of frequency allow the construction of 2-dimensional models of
the target. The data shown here are for a commercial airliner.
A common technique for such
image processing uses Fourier methods to reconstruct the image. As can be seen
from the image, the Quick Pixon method provides a superior solution with
greatly reduced noise. In addition, use of the Accelerated Pixon method offers
the ability for "super-resolution", i.e. seeing structures with
resolution on the order of, or less than, the wavelength of the radar pulse.
This is the same type of capability that is demonstrated in the Pistol star
example given above.
|
|
Click to enlarge |
Standard Fourier versus a
Quick Pixon reconstruction of aircraft Inverse Synthetic Aperture Radar (ISAR) imaging
data. The Pixon inversion provides excellent artifact (noise) rejection and
greater interpretability. The horizontal stripes in the tail area are the
result of multiple reflections and are actually useful in identifying the type
of aircraft.
Example 13: Super-Resolved Closely Spaced Objects
The Pixon method is also of
great benefit for the identification of "closely spaced objects"
(CSOs). This is an important problem in missile defense where one wishes to
identify as early as possible the number and types of launch vehicles. We
present in the accompanying figure and example of the use of the Pixon method
for the reconstruction of a CSO data set. The example chosen here is for a low
signal case. The data set was synthetically generated and has a maximum
signal-to-noise ration of 10 per pixel on the brightest source. Shown are the
truth image, the blurring function (PSF), and the blurred, noisy data. The noise
model chosen for this case was Gaussian noise. Both the result for the Pixon
method and Wiener-filtered Fourier deconvolution are shown.
As can be seen from the
figure, the Pixon method provides a great improvement over the Wiener
reconstruction, offering both increased resolution and remarkable artifact
(noise) rejection. Even at an SNR of 10 the Pixon method provides
superresolution performance for this case. Furthermore, unlike the Wiener
method, as the SNR increases, the resolution obtained by the Pixon method
continues to grow. Examples showing the relative performance of the Pixon
method and Wiener deconvolution are provided in the figures, SNR=25 and SNR=100. A summary
figure showing the performance of the Pixon method for all three SNR values is
given in the figure: Performance
versus SNR.
|
|
Click to enlarge |
Wiener-filtered
Fourier deconvolution versus Accelerated Pixon reconstruction of a synthetic
"Closely-Spaced-Object" (CSO) example. In this example the maximum
SNR is the data is 10. The Pixon inversion super-resolves the data, provides
excellent artifact (noise) rejection, and much greater interpretability.
Example 14: Subdiffraction
Limited Imaging
The next example provides a
graphic illustration of the ability of the Pixon to recover high frequency
information which is absent in the imaging data. In this case, we have taken a
synthetic image containing several closely spaced objects and blurred it with a
diffraction pattern appropriate to a circular aperture (Airy function). This was
further convolved with a small Gaussian function to simulate additional
atmospheric turbulence or Gaussian jitter of the imaging platform during data
collection. To this a small amount of Gaussian noise was added to create a data
set with a maximum signal-to-noise ratio of 300 per pixel. The resulting data
set has a sharp cutoff in information content in frequency space due to the
finite aperture of the simulated data collecting aperture. Nonetheless, the
Pixon reconstruction is able to accurately recover Fourier power well beyond the
cutoff frequency in the data, demonstrating that the imposition of a minimum
complexity constraint on the image model is remarkably powerful. As advertised,
the application of "Ockham's Razor" not only is the
best means of separating the noise from the true information content, but by
deducing the most probable model for the real image, it accurately predicts the
values of missing information in the data set.
|
|
Click to enlarge |
The Pixon
method demonstrates the ability to perform super-resolution for a synthetic
"Closely Space Object" (CSO) problem. Shown is the truth image along
with its true power spectrum. The power spectrum of the PSF blurring function
has a sharp cutoff in frequency space. This results in a sharp cutoff in the
power spectrum of the data, i.e. there is no spectral power present in the data
beyond this cutoff frequency. Nonetheless, the Pixon method recovers spectral
power outside of the frequency content of the data (compare the power spectrum
of the true image with that of the reconstruction). The ability to recover this
lost spectral power is provided by the powerful minimal complexity image model
constraint used by the Pixon method.
Example 15: Subdiffraction
Resolution Electron Microscopy
The next example presents a Pixon reconstruction of electron microscopy data for a nanocrystal of CdSe. Of interest here is in the properties of real nanocrystals for semiconductor research. The Pixon reconstruction achieve subdiffraction resolution and cleanly separates the signal from the noise to provide an image with greatly enhanced interpretability. Relative to competing techniques, the Pixon reconstruction provides superior artifact reduction and an accurate rendition of the resolution achievable with the data. (For a comparison of the Pixon reconstruction with Maximum Entropy, click here.)
|
|
Click to enlarge |
Raw and Pixon
reconstructed Z-Contrast Scanning Transmission Electron Micrograph (Z-STEM) of
a Cd Se (cadmium selenide) nanocrystal embedded in MEH-PPV polymer. The
image was obtained on the VG HB603 STEM in the Electron Microscopy Group,
Oak Ridge National Lab., Stephen Pennycook, PI. The nanocrystal composite
sample was prepared in the group of Prof. Sandy Rosenthal, Vanderbilt
University. Tadd Kippeny: noncrystal synthesis; Meg Erwin: polymer
synthesis; Peter Nellist: assistance with STEM operation; Andreas Kadavanich:
fabrication of the composite and microscopy. The bright objects are Cd
atoms. The darker objects adjacent to the Cd atoms are the Se atoms,
roughly 1.5 Angstroms away. CdSe pairs are separated by approximately 3.6
Angstroms. The scanning beam PSF had a FWHM of 1.5 Angstroms and was sampled
with approximately 11 to 12 pixels across the FWHM.
PixonVisionTM—Simulation
of the performance of our color quick Pixon video hardware
Imaging of a color image
of a Mandrill. The images below present
simulations of the performance of the quick Pixon method as it has been
tailored for dedicated read-time video hardware (PixonVisionTM card). The PixonVision electronics will be a single printed circuit
board capable of processing standard NTSC color video in real-time and
outputting the processed imagery to and NTSC compatible monitor and/or a high
resolution monitor, e.g., SVGA. The
degree of sharpening and artifact suppression is user adjustable in
real-time. This allows the user to
select a wide range of image processing options, for pass-through of the video,
straightforward deconvolution, and full quick Pixon processed video. Real-time user selectable deconvolution kernels
support a wide range of de-blurring conditions.
(These examples use TIFF files. GIF files are unable to fully display the image quality improvements. To get your browser to display TIFF files requires the installation of a plug-in. AlternaTIFF is a Netscape-style browser plug-in that displays most of the common types of TIFF files. It is compatible with Netscape Navigator 3.0 and higher, Internet Explorer 3.0 and higher, and Opera 3.51 and higher. It is a 32-bit Windows program, and requires Windows 95 or higher (98, NT 4.0, 2000), and a 32-bit browser. You can obtain this plug-in on the web at http://www.mieweb.com/alternatiff/)
|
Blurry data vs Pixon |
Data vs Deconvolution |
Click to enlarge |
|
Truth vs quick Pixon |
Deconvolution vs Pixon |
Four examples
of simulated PixonVision hardware performance.
For this example, a picture of a Mandrill was blurred and noise was
added. This made up the input data set
for the image processing. Upper-left:
blurry, noisy image (left half) versus quick Pixon processed image (right half
of image). Upper-right: blurry, noisy
data (left half) versus straight deconvolution without the benefit of Pixon
artifact removal (right half).
Lower-left: truth image, i.e. the original image which was blurred and
noised (left half) versus the Pixon reconstruction (right half). Lower-right: the straight deconvolution
result (left half) versus the quick Pixon reconstruction (right half).
Clicking on any of the images produces an enlarged version for closer study.
-- Movies
--
In this section we
provide a Pixon processed video of Navy ISAR radar imagery to demonstrate the
utility of the method for video applications. This clip was made with the Quick
Pixon method and can be implemented in custom hardware to provide real-time (30
Hz), on-the-spot processing of video or other real-time data.
Note: Because of their size, these
videos may have poor performance on slower machines.
Radar imaging of a
commercial airliner (8.3 Mbytes AVI). The data is the source of the single
still frame of the radar example. In the clip the Pixon processed imagery
appears in the top frame and the standard Fourier processed data appear in the
bottom frame.
As can be seen from the
examples, the Pixon method provides a revolutionary leap forward in our ability
to process and interpret data. In the field of astronomical imaging, the Pixon
method is now being used to increase the scientific capability and sensitivity
of NASA's multi-million dollar satellite observatories by factors of 10 or
more. These same capabilities can be used in commercial applications to
increase the sensitivity of surveillance cameras, medical imaging, earth
resource satellite imaging, radar, sonar, geophysical surveying, video and
audio recording and decoding, high definition TV, communications, and a variety
of other applications.
Inventors Drs. Richard
Puetter and Amos Yahil, and attorney Michael Shaham are the managing directors
of Pixon LLC. A brief biographical sketch of each person appears below.
Dr. Richard Puetter
developed the original Pixon method with his student Robert Piña at the
University of California, San Diego. He is a well-known infrared astronomer,
instrument builder, and computational physicist. He has built a variety of
infrared instruments for ground-based and airborne astronomy and is currently
building an imaging mid-infrared spectrometer for the world's largest
telescopes, the Keck Telescopes in Hawaii. He is also well known for his work
in computational physics which includes supercomputer modeling of radiative
transfer processes in quasars and active galaxies, magneto-hydrodynamics of the
earth's magneto-tail, and, most recently, image processing.
Dr. Amos Yahil is a
leading cosmologist at the State University of New York at Stony Brook, known
for pioneering work on dark matter in massive galactic halos, the large-scale
structure in the universe, primeval galaxies, and exploding stars (supernovae).
In addition to work in computational physics, he is intimately familiar with
large astronomical data sets. Both require the manipulation of extensive
amounts of data, and the development of efficient numerical methods for this
purpose. Two years ago he proposed an acceleration technique for the Pixon
method originated by Puetter and Piña, and teamed with Puetter to refine the
method and to develop fast and efficient codes for it. This has resulted in an
Accelerated Pixon method, which increases the computational efficiency by many
orders of magnitude without compromising the quality of the original Pixon
method.
Attorney Michael Shaham
is an attorney based in Tel Aviv specializing in international transactions and
intellectual property. He has more than twenty years of experience as counsel
to Israeli, American, Japanese and European technology-based businesses.
Return to:
"The Pixon
Page" (Supplemental gallery of Pixon method Imagery)