The SSIM Index for Image Quality Assessment

Zhou Wang, Alan C. Bovik, Hamid R. Sheikh and Eero P. Simoncelli

MAD_fixmse_iters

about this image

The Structural SIMilarity (SSIM) index is a method for measuring the similarity between two images. The SSIM index can be viewed as a quality measure of one of the images being compared, provided the other image is regarded as of perfect quality. It is an improved version of the universal image quality index proposed before. Detailed description is given in the following paper:

1.      Z. Wang, A. C. Bovik, H. R. Sheikh and E. P. Simoncelli, "Image quality assessment: From error visibility to structural similarity," IEEE Transactions on Image Processing, vol. 13, no. 4, pp. 600-612, Apr. 2004.

A Matlab implementation of the SSIM index (ssim_index.m) is available here. You can download it for free, change it as you like and use it anywhere, but please refer to its original source (cite the above paper and this web page).

Download ssim index.m

Before using the code, please go through the suggested usage and demo tests below to get an idea on how to use it and how it works.

 

Suggested Usage

The above (ssim_index.m) is a single scale version of the SSIM indexing measure, which is most effective if used at the appropriate scale. The precisely “right” scale depends on both the image resolution and the viewing distance and is usually difficult to be obtained.  In practice, we suggest to use the following empirical formula to determine the scale for images viewed from a typical distance (say 3~5 times of the image height or width): 1) Let F = max(1, round(N/256)), where N is the number of pixels in image height (or width); 2) Average local F by F pixels and then downsample the image by a factor of F; and 3) apply the ssim_index.m program. For example, for an 512 by 512 image, F = max(1, round(512/256)) = 2, so the image should be averaged within a 2 by 2 window and downsampled by a factor of 2 before applying ssim_index.m.

The Matlab code (ssim.m) that includes the suggested downsampling process described above is given here:

Download ssim.m

Other Useful Downloads

The SSIM values are computed (using ssim.m) for 5 publicly available subject-rated image databases, including LIVE database, Cornell A57 database, IVC database, Toyama database and TID2008 database. When the images are in RGB color, an Matlab function “rgb2gray” is used to convert the images to gray scale. The results (in Matlab .mat format) are provided here for future comparisons:

 

Download SSIM results for LIVE database

Download SSIM results for Cornell A57 database

Download SSIM results for IVC database

Download SSIM results for Toyama database

Download SSIM results for TID2008 database

 

A tutorial paper is published at

 

2.      Z. Wang and A. C. Bovik, “Mean squared error: love it or leave it? - A new look at signal fidelity measures,” IEEE Signal Processing Magazine, vol. 26, no. 1, pp. 98-117, Jan. 2009.

 

Powerpoint and PDF figures of the above paper are available here for references:

 

Download Powerpoint figures

Download PDF figures

 

C++ and Java implementations of the SSIM index have also been made available online by other authors:

·         C++ implementation by unknown author(s) at http://perso.wanadoo.fr/reservoir/

·         C++ implementation by Mehdi Rabah at http://mehdi.rabah.free.fr/SSIM/

·         Java implementation by Gabriel Prieto at http://rsb.info.nih.gov/ij/plugins/ssim-index.html and http://www.ucm.es/info/fismed/SSIM_family/SSIM_archivos/frame.htm

·         Java implementation of Multi-scale SSIM by Gabriel Prieto at http://rsb.info.nih.gov/ij/plugins/mssim-index.html and http://www.ucm.es/info/fismed/MSSIM/MSSIM_archivos/frame.htm

Extensions and Related Papers

The following papers discuss the extensions and improvements of the SSIM index approach:

3.      Z. Wang and Q. Li, "Video quality assessment using a statistical model of human visual speed perception," Journal of the Optical Society of America A, vol. 24, no. 12, pp. B61-B69, Dec. 2007.

4.      Z. Wang and X. Shang, “Spatial pooling strategies for perceptual image quality assessment,” IEEE International Conference on Image Processing, Atlanta, GA, Oct. 8-11, 2006.

5.      Z. Wang and E. P. Simoncelli, “Translation insensitive image similarity in complex wavelet domain,” IEEE International Conference on Acoustics, Speech and Signal Processing, vol. II, pp. 573-576, Philadelphia, PA, Mar. 2005.

6.      Z. Wang, L. Lu, and A. C. Bovik, “Video quality assessment based on structural distortion measurement,” Signal Processing: Image Communication, special issue on “Objective video quality metrics”, vol. 19, no. 2, pp. 121-132, Feb. 2004.

7.      Z. Wang, E. P. Simoncelli and A. C. Bovik, “Multi-scale structural similarity for image quality assessment,” Invited Paper, IEEE Asilomar Conference on Signals, Systems and Computers, Nov. 2003. [Java Code 1, Java Code 2]

 

Other related papers:

 

8.      Z. Wang and E. P. Simoncelli, “Maximum differentiation (MAD) competition: A methodology for comparing computational models of perceptual quantities,” Journal of Vision, vol. 8, no. 12, Sept. 2008.

9.      Z. Wang and E. P. Simoncelli, “An adaptive linear system framework for image distortion analysis,” IEEE International Conference on Image Processing, Genoa, Italy, Sept. 11-14, 2005.

10.  Z. Wang, A. C. Bovik and E. P. Simoncelli, “Structural Approaches to image quality assessment,” in Handbook of Image and Video Processing (Al Bovik, ed.), 2nd edition, Academic Press, June 2005.

11.  Z. Wang and E. P. Simoncelli, “Stimulus synthesis for efficient evaluation and refinement of perceptual image quality metrics,” Human Vision and Electronic Imaging IX, Proc. SPIE, vol. 5292, Jan. 2004.

12.  Z. Wang, H. R. Sheikh and A. C. Bovik, “Objective video quality assessment,” in The Handbook of Video Databases: Design and Applications (B. Furht and O. Marqure, eds.), CRC Press, pp. 1041-1078, Sept. 2003.

13.  Z. Wang, A. C. Bovik and L. Lu, “Why is image quality assessment so difficult?” IEEE International Conference on Acoustics, Speech, & Signal Processing, May 2002.

14.  Z. Wang, and A. C. Bovik, “A universal image quality index,” IEEE Signal Processing Letters, vol. 9, no. 3, pp. 81-84, March 2002.

 

Demonstration

In the following, all distorted images have roughly the same mean squared error (MSE) values with respect to the original image, but very different quality. SSIM gives a much better indication of image quality.

 

einstein

Original, MSE = 0; SSIM = 1

meanshift

MSE = 144, SSIM = 0.988

contrast

MSE = 144, SSIM = 0.913

impulse

MSE = 144, SSIM = 0.840

blur

MSE = 144, SSIM = 0.694

jpg

MSE = 142, SSIM = 0.662

 

The following example illustrates the maximal/minimal SSIM images synthesized along the equal-MSE hypersphere in the space of all images. All images along the hypersphere have the same MSE values with respect to the reference image, but have drastically different perceptual quality.

 

fig04.gif

 

The following example demonstrates the SSIM index map, which provides a measurement of local image quality over space. Note that severe JPEG compression produces annoying pseudo-contouring effects (in the sky region) and blocking artifacts (along the boundaries of the building), which are successfully captured by the SSIM index map, yet poorly predicted by the absolute error map (in both distortion/quality maps, brighter indicates better quality).

 

imo.tif

Original image

imjpg.tif

JPEG compressed image

emapjpg.tif

Absolute error map

qmapjpg.tif

SSIM index map

 

Test on LIVE Image Database

The SSIM indexing algorithm has been tested on the LIVE database created at the Lab for Image and Video Engineering (LIVE) at the University of Texas at Austin. The database is subject-rated and available for free download.

 

First Stage LIVE database test

In the first stage, the database contained 460 images, where 116 were original images and the rest 344 are JPEG and JPEG2000 compressed. Two sample images (cropped from 768X512 to 256X192 for visibility) are shown below. Note that quantization in JPEG and JPEG2000 algorithms often results in smooth representations of fine detail regions (e.g., the tiles in the upper image and the trees in the lower image). Compared with other types of regions, these regions may not be worse in terms of pointwise difference measures (as shown in the absolute error map). However, since the structural information of the image details are nearly completely lost, they exhibit poorer visual quality. Close piece-by-piece comparison of the SSIM index and the absolute error maps, we observe that the SSIM index is more consistent with perceived quality measurement. Note: in both distortion/quality maps, brighter means better quality.

 

Original Images

JPEG/JPEG2000

Compressed Images

Absolute Error Map

SSIM Index Map

crop_jpg_oimg_low

crop_jpg_img_low

crop_jpg_rsemap_low

crop_jpg_ssimmap_low1

crop_jp2_oimg_low

crop_jp2_img_low

crop_jp2_rsemap_low

crop_jp2_ssimmap_low1

 

The scatter plots of the subjective measurement (mean opinion score, MOS) versus the objective predictions (PSNR and MSSIM) are shown below, where each point represent one test image. Clearly, MSSIM is much better in predicting the perceived image quality.

 

PSNR vs. MOS

MSSIM vs. MOS

 

Full LIVE database test

The LIVE database was later developed to contain more diverse distortion types, including JPEG2000 compression, JPEG compression, Gaussian noise contamination, Gaussian blur, and JPEG2000 compressed images undergoing fast fading channel distortions. The full database contains 982 images, with 203 original and 779 distorted images. The SSIM index was computed with the full database and the SSIM values for all images are provided in the following file (in Matlab .mat format) for future comparisons:

 

SSIM results for LIVE database

 

Zhou Wang's Homepage

Created Feb. 2003

Last updated Nov. 20, 2009

 

http://u1.extreme-dm.com/i.gif