The SSIM Index for Image Quality Assessment

Zhou Wang, Alan C. Bovik, Hamid R. Sheikh and Eero P. Simoncelli

Description: MAD_fixmse_iters

about this image


 

*    Visit the webpage of SSIMplus, a novel perceptual video quality-of-experience (QoE) index designed for practical use and in many ways goes far beyond what SSIM can do.

 


The Structural SIMilarity (SSIM) index is a method for measuring the similarity between two images. The SSIM index can be viewed as a quality measure of one of the images being compared, provided the other image is regarded as of perfect quality. It is an improved version of the universal image quality index proposed before. Detailed description is given in the following paper:

1.      Z. Wang, A. C. Bovik, H. R. Sheikh and E. P. Simoncelli, "Image quality assessment: From error visibility to structural similarity," IEEE Transactions on Image Processing, vol. 13, no. 4, pp. 600-612, Apr. 2004.

A Matlab implementation of the SSIM index (ssim_index.m) is available here. You can download it for free, change it as you like and use it anywhere, but please refer to its original source (cite the above paper and this web page).

Download ssim index.m

Before using the code, please go through the suggested usage and demo tests below to get an idea on how to use it and how it works.

 


Suggested Usage

The above (ssim_index.m) is a single scale version of the SSIM indexing measure, which is most effective if used at the appropriate scale. The precisely “right” scale depends on both the image resolution and the viewing distance and is usually difficult to be obtained.  In practice, we suggest to use the following empirical formula to determine the scale for images viewed from a typical distance (say 3~5 times of the image height or width): 1) Let F = max(1, round(N/256)), where N is the number of pixels in image height (or width); 2) Average local F by F pixels and then downsample the image by a factor of F; and 3) apply the ssim_index.m program. For example, for an 512 by 512 image, F = max(1, round(512/256)) = 2, so the image should be averaged within a 2 by 2 window and downsampled by a factor of 2 before applying ssim_index.m.

The Matlab code (ssim.m) that includes the suggested downsampling process described above is given here:

Download ssim.m


Other Useful Downloads

The SSIM values are computed (using ssim.m) for 6 publicly available subject-rated image databases, including LIVE database, Cornell A57 database, IVC database, Toyama database, TID2008 database, and CSIQ database. When the images are in RGB color, an Matlab function “rgb2gray” is used to convert the images to gray scale. The results (in Matlab .mat format) are provided here, together with performance evaluations based on Spearman’s rank correlation coefficient (SRCC) and Kendall’s rank correlation coefficient (KRCC),  for future comparisons:

 

Database

Toyama

SRCC

KRCC

LIVE

Result

0.9479

0.7963

Cornell-A57

Result

0.8066

0.6058

IVC

Result

0.9018

0.7223

Toyama-MICT

Result

0.8794

0.6939

TID2008

Result

0.7749

0.5768

CSIQ

Result

0.8756

0.6907

 

A tutorial paper is published at

 

2.      Z. Wang and A. C. Bovik, “Mean squared error: love it or leave it? - A new look at signal fidelity measures,” IEEE Signal Processing Magazine, vol. 26, no. 1, pp. 98-117, Jan. 2009.

Powerpoint and PDF figures of the above paper are available here for references:

Download Powerpoint figures

Download PDF figures

 

MATLAB, C, C++, Java and LabView implementations of the SSIM index have also been made available online by other authors:

·         C++ implementation by Lefungus at http://perso.wanadoo.fr/reservoir/

·         C++ implementation by Mehdi Rabah at http://mehdi.rabah.free.fr/SSIM/

·         C++ implementation by Naru Kinjo at http://denshika.cc/open/20100630/, http://denshika.cc/open/20100630/batch.php

·         C++ OpenCV implementation (wrapper of Mehdi Rabah and Naru Kinjo’s work) by Norishige Fukushima at http://nma.web.nitech.ac.jp/fukushima/opencv/ssim/ssim-e.html

·         C implementation of RGBA SSIM for comparing PNG images by porneL at http://pornel.net/dssim

·         C implementation of SSIM and MS-SSIM by Tom Distler at http://tdistler.com/iqa/

·         Java implementation by Gabriel Prieto at http://rsb.info.nih.gov/ij/plugins/ssim-index.html and http://www.ucm.es/gabriel_prieto/ssim

·         Java implementation of Multi-scale SSIM by Gabriel Prieto at http://rsb.info.nih.gov/ij/plugins/mssim-index.html and http://www.ucm.es/gabriel_prieto/ms-ssim

·         LabView implementation at LIVE: http://live.ece.utexas.edu/research/quality/SSIM.llb

·         More efficient MATLAB implementation of SSIM by Adam Turcotte at http://ssim.rivetsforbreakfast.com/

·         Python implementation of SSIM by Antoine Vacavant at http://isit.u-clermont1.fr/~anvacava/code.html

·         Python implementation of SSIM (improved upon Antoine Vacavent’s version) by Helder C. R. de Oliveira at http://helderc.net/ 


Extensions and Related Papers

The following papers discuss the extensions and improvements of the SSIM index approach:

3.      D. Brunet, E. R. Vrscay and Z. Wang, “On the mathematical properties of the structural similarity index,” IEEE Transactions on Image Processing, vol. 21, no. 4, pp. 1488-1499, Apr. 2012.

4.      Z. Wang and Q. Li, "Information content weighting for perceptual image quality assessment," IEEE Transactions on Image Processing, vol. 20, no 5, pp. 1185-1198, May 2011. [Web and code]

5.      M. P. Sampat, Z. Wang, S. Gupta, A. C. Bovik and M. K. Markey, "Complex wavelet structural similarity: A new image similarity index," IEEE Transactions on Image Processing, vol. 18, no. 11, pp. 2385-2401, Nov. 2009.

6.      Z. Wang and Q. Li, "Video quality assessment using a statistical model of human visual speed perception," Journal of the Optical Society of America A, vol. 24, no. 12, pp. B61-B69, Dec. 2007.

7.      Z. Wang and X. Shang, “Spatial pooling strategies for perceptual image quality assessment,” IEEE International Conference on Image Processing, Atlanta, GA, Oct. 8-11, 2006.

8.      Z. Wang and E. P. Simoncelli,Translation insensitive image similarity in complex wavelet domain,” IEEE International Conference on Acoustics, Speech and Signal Processing, vol. II, pp. 573-576, Philadelphia, PA, Mar. 2005.

9.      Z. Wang, L. Lu, and A. C. Bovik, “Video quality assessment based on structural distortion measurement,” Signal Processing: Image Communication, special issue on “Objective video quality metrics”, vol. 19, no. 2, pp. 121-132, Feb. 2004.

10.  Z. Wang, E. P. Simoncelli and A. C. Bovik,Multi-scale structural similarity for image quality assessment,” Invited Paper, IEEE Asilomar Conference on Signals, Systems and Computers, Nov. 2003. [Matlab Code] [Java Code 1, Java Code 2]

 

Other related papers:

 

11.  A. Rehman, Y. Gao, J. Wang, Z. Wang, “Image classification based on complex wavelet structural similarity,” Signal Processing: Image Communication, special issue on Biologically Inspired Approaches for Visual Information Processing and Analysis, vol. 28, no. 8, pp. 984-992, Sep. 2013.

12.  S. Wang, A. Rehman, Z. Wang, S. Ma and W. Gao, “SSIM-inspired divisive normalization for perceptual video coding,” IEEE Transactions on Image Processing, vol. 22, no. 4, pp. 1418-1429, Apr. 2013.

13.  A. Rehman and Z. Wang, “Reduced-reference image quality assessment by structural similarity estimation,” IEEE Transactions on Image Processing, vol. 21, no. 8, pp. 3378-3389, Aug. 2012.

14.  S. Wang, A. Rehman, Z. Wang, S. Ma and W. Gao, “SSIM-motivated rate distortion optimization for video coding”, IEEE Transactions on Circuits and Systems for Video Technology, vol. 22, no. 4, pp. 516-529, Apr. 2012.

15.  A. Rehman, M. Rostami, Z. Wang, D. Brunet and E. R. Vrscay, “SSIM-inspired image restoration using sparse representation,” EURASIP Journal on Advances in Signal Processing, special issue on Image and Video Quality Improvement Techniques for Emerging Applications, Jan. 2012.

16.  Z. Wang and E. P. Simoncelli, “Maximum differentiation (MAD) competition: A methodology for comparing computational models of perceptual quantities,” Journal of Vision, vol. 8, no. 12, Sept. 2008.

17.  Z. Wang and E. P. Simoncelli,An adaptive linear system framework for image distortion analysis,” IEEE International Conference on Image Processing, Genoa, Italy, Sept. 11-14, 2005.

18.  Z. Wang, A. C. Bovik and E. P. Simoncelli, “Structural Approaches to image quality assessment,” in Handbook of Image and Video Processing (Al Bovik, ed.), 2nd edition, Academic Press, June 2005.

19.  Z. Wang and E. P. Simoncelli,Stimulus synthesis for efficient evaluation and refinement of perceptual image quality metrics,” Human Vision and Electronic Imaging IX, Proc. SPIE, vol. 5292, Jan. 2004.

20.  Z. Wang, H. R. Sheikh and A. C. Bovik, “Objective video quality assessment,” in The Handbook of Video Databases: Design and Applications (B. Furht and O. Marqure, eds.), CRC Press, pp. 1041-1078, Sept. 2003.

21.  Z. Wang, A. C. Bovik and L. Lu, “Why is image quality assessment so difficult?IEEE International Conference on Acoustics, Speech, & Signal Processing, May 2002.

22.  Z. Wang, and A. C. Bovik, “A universal image quality index,” IEEE Signal Processing Letters, vol. 9, no. 3, pp. 81-84, March 2002.

 


Demonstration

In the following, all distorted images have roughly the same mean squared error (MSE) values with respect to the original image, but very different quality. SSIM gives a much better indication of image quality.

 

Description: einstein

Original, MSE = 0; SSIM = 1

Description: meanshift

MSE = 144, SSIM = 0.988

Description: contrast

MSE = 144, SSIM = 0.913

Description: impulse

MSE = 144, SSIM = 0.840

Description: blur

MSE = 144, SSIM = 0.694

Description: jpg

MSE = 142, SSIM = 0.662

 

The following example illustrates the maximal/minimal SSIM images synthesized along the equal-MSE hypersphere in the space of all images. All images along the hypersphere have the same MSE values with respect to the reference image, but have drastically different perceptual quality.

 

Description: fig04.gif

 

The following example demonstrates the SSIM index map, which provides a measurement of local image quality over space. Note that severe JPEG compression produces annoying pseudo-contouring effects (in the sky region) and blocking artifacts (along the boundaries of the building), which are successfully captured by the SSIM index map, yet poorly predicted by the absolute error map (in both distortion/quality maps, brighter indicates better quality).

 

Description: imo.tif

Original image

Description: imjpg.tif

JPEG compressed image

Description: emapjpg.tif

Absolute error map

Description: qmapjpg.tif

SSIM index map

 


Test on LIVE Image Database

The SSIM indexing algorithm has been tested on the LIVE database created at the Lab for Image and Video Engineering (LIVE) at the University of Texas at Austin. The database is subject-rated and available for free download.

 

First Stage LIVE database test

In the first stage, the database contained 460 images, where 116 were original images and the rest 344 are JPEG and JPEG2000 compressed. Two sample images (cropped from 768X512 to 256X192 for visibility) are shown below. Note that quantization in JPEG and JPEG2000 algorithms often results in smooth representations of fine detail regions (e.g., the tiles in the upper image and the trees in the lower image). Compared with other types of regions, these regions may not be worse in terms of pointwise difference measures (as shown in the absolute error map). However, since the structural information of the image details are nearly completely lost, they exhibit poorer visual quality. Close piece-by-piece comparison of the SSIM index and the absolute error maps, we observe that the SSIM index is more consistent with perceived quality measurement. Note: in both distortion/quality maps, brighter means better quality.

 

Original Images

JPEG/JPEG2000

Compressed Images

Absolute Error Map

SSIM Index Map

Description: crop_jpg_oimg_low

Description: crop_jpg_img_low

Description: crop_jpg_rsemap_low

Description: crop_jpg_ssimmap_low1

Description: crop_jp2_oimg_low

Description: crop_jp2_img_low

Description: crop_jp2_rsemap_low

Description: crop_jp2_ssimmap_low1

 

The scatter plots of the subjective measurement (mean opinion score, MOS) versus the objective predictions (PSNR and MSSIM) are shown below, where each point represent one test image. Clearly, MSSIM is much better in predicting the perceived image quality.

 

PSNR vs. MOS

MSSIM vs. MOS

 

Full LIVE database test

The LIVE database was later developed to contain more diverse distortion types, including JPEG2000 compression, JPEG compression, Gaussian noise contamination, Gaussian blur, and JPEG2000 compressed images undergoing fast fading channel distortions. The full database contains 982 images, with 203 original and 779 distorted images. The SSIM index was computed with the full database and the SSIM values for all images are provided in the following file (in Matlab .mat format) for future comparisons:

 

SSIM results for LIVE database

 


Zhou Wang's Homepage

Created Feb. 2003

Last updated Nov. 9, 2014

 

Description: http://u1.extreme-dm.com/i.gif