MH Predictions Extended Results

Compressed-Sensing Recovery of Images and Video Using Multihypothesis Predictions

Chen Chen, Eric W. Tramel and James E. Fowler

Abstract

Compressed-sensing reconstruction of still images and video sequences driven by multihypothesis predictions is considered. Specifically, for still images, multiple predictions drawn for an image block are made from spatially surrounding blocks within an initial non-predicted reconstruction. For video, multihypothesis predictions of the current frame are generated from one or more previously reconstructed reference frames. In each case, the predictions are used to generate a residual in the domain of the compressed-sensing random projections. This residual being typically more compressible than the original signal leads to improved reconstruction quality. To appropriately weight the hypothesis predictions, a Tikhonov regularization to an ill-posed least-squares optimization is proposed. Experimental results demonstrate that the proposed reconstructions outperform alternative strategies not employing multihypothesis predictions.

Download

[PDF], [MATLAB CODE], [PRESENTATION]

Experiment Results

1. Video
The performance of multihypothesis predictions and residual reconstruction is evaluated on 4 different video sequences. More results can be found at http://www.ece.msstate.edu/~ewt16/publications/

**Table 1. PSNR in dB of Residual Reconstruction (RR) of Video**
Algorithm	Subrate
Algorithm	0.1	0.2	0.3	0.4	0.5
	Foreman
RR w/ MH-TIK	31.5	33.8	35.0	35.9	36.7
RR w/ MH-l₁	28.0	31.0	32.9	34.5	36.3
BCS-SPL	25.1	27.5	29.9	31.8	33.7
	Susie
RR w/ MH-TIK	34.2	36.3	37.3	38.0	38.6
RR w/ MH-l₁	30.7	33.4	35.2	36.7	38.1
BCS-SPL	29.4	31.9	33.6	35.1	36.4
	Football
RR w/ MH-TIK	25.5	27.6	29.1	30.3	31.4
RR w/ MH-l₁	24.6	26.7	28.3	29.7	31.1
BCS-SPL	24.0	25.9	27.2	28.5	29.9
	News
RR w/ MH-TIK	30.9	31.6	32.0	32.1	32.2
RR w/ MH-l₁	23.7	27.0	29.5	31.6	34.1
BCS-SPL	20.2	23.4	26.3	28.9	31.8

2. Still Images
The performance of the MH-BCS-SPL and MH-MS-BCS-SPL is evaluated on 13 grayscale images of size 512 x 512.

**Table 2. Image Reconstruction PSNR (dB) at Various Subrates**
Algorithm	Subrate
Algorithm	0.1	0.2	0.3	0.4	0.5
	Lenna
BCS-SPL	27.93	31.27	33.41	35.15	36.71
MH-BCS-SPL	29.85	32.85	34.73	36.34	37.82
MS-BCS-SPL	31.48	34.59	36.61	37.82	38.94
MH-MS-BCS-SPL	31.61	34.88	36.79	38.32	39.74
MS-GPSR	30.30	33.60	35.29	36.29	37.73
TV	29.84	32.93	35.05	36.82	38.43
	Barbara
BCS-SPL	22.28	23.78	25.32	26.88	28.51
MH-BCS-SPL	27.89	31.46	33.63	35.68	37.29
MS-BCS-SPL	23.84	25.12	26.05	27.27	28.83
MH-MS-BCS-SPL	24.28	26.42	27.98	32.95	36.21
MS-GPSR	24.03	25.24	26.06	27.50	29.64
TV	22.95	24.48	26.30	28.42	30.78
	Goldhill
BCS-SPL	26.57	28.92	30.42	31.72	33.03
MH-BCS-SPL	27.67	30.28	31.82	33.26	34.62
MS-BCS-SPL	28.97	31.06	32.75	33.67	34.64
MH-MS-BCS-SPL	29.07	31.35	33.06	34.55	36.10
MS-GPSR	28.39	30.55	32.10	32.89	34.20
TV	27.52	29.87	31.63	33.23	34.81
	Mandrill
BCS-SPL	20.45	21.80	22.90	23.96	25.10
MH-BCS-SPL	20.47	22.36	24.03	25.36	26.67
MS-BCS-SPL	21.45	23.07	24.69	25.54	26.46
MH-MS-BCS-SPL	21.66	23.20	24.82	25.81	27.10
MS-GPSR	21.52	22.92	24.27	25.07	26.21
TV	20.53	22.02	23.44	24.94	26.52
	Peppers
BCS-SPL	28.83	31.98	33.73	35.12	36.36
MH-BCS-SPL	30.28	32.82	34.32	35.63	36.87
MS-BCS-SPL	31.75	34.59	35.76	36.77	37.67
MH-MS-BCS-SPL	32.08	34.73	35.96	37.15	38.37
MS-GPSR	29.25	31.90	33.03	34.22	35.75
TV	30.36	33.14	34.70	35.91	37.05
	Clown
BCS-SPL	24.46	29.65	31.87	33.70	35.37
MH-BCS-SPL	28.78	32.80	35.02	36.83	38.40
MS-BCS-SPL	29.06	32.72	35.65	36.94	38.02
MH-MS-BCS-SPL	29.48	33.89	36.20	38.16	39.76
MS-GPSR	27.93	31.58	33.95	35.00	36.56
TV	27.94	31.18	33.48	35.41	37.24
	Crowd
BCS-SPL	24.49	28.87	31.16	33.11	34.98
MH-BCS-SPL	26.62	30.11	32.35	34.38	36.22
MS-BCS-SPL	29.15	32.72	35.71	37.03	38.32
MH-MS-BCS-SPL	29.20	32.76	35.78	37.57	39.33
MS-GPSR	27.99	31.20	34.13	35.14	36.96
TV	26.81	30.45	33.26	35.81	38.32
	Couple
BCS-SPL	24.78	27.01	28.65	30.14	31.60
MH-BCS-SPL	25.82	28.85	30.67	32.21	33.76
MS-BCS-SPL	26.87	29.51	31.47	32.55	33.65
MH-MS-BCS-SPL	27.04	30.05	31.79	33.83	35.57
MS-GPSR	26.28	28.82	30.60	31.61	33.15
TV	25.73	28.48	30.64	32.61	34.51
	Man
BCS-SPL	26.25	28.63	30.30	31.79	33.24
MH-BCS-SPL	26.75	29.36	30.95	32.47	34.03
MS-BCS-SPL	28.64	31.05	33.03	34.05	35.10
MH-MS-BCS-SPL	28.68	31.22	33.18	34.59	36.12
MS-GPSR	27.78	30.15	31.85	33.01	34.51
TV	27.25	29.94	32.05	33.98	35.98
	Boat
BCS-SPL	25.14	27.78	29.57	31.12	32.60
MH-BCS-SPL	26.17	29.30	31.18	32.89	34.45
MS-BCS-SPL	27.35	30.08	31.98	33.12	34.22
MH-MS-BCS-SPL	27.46	30.38	32.23	33.89	35.46
MS-GPSR	26.52	29.21	30.84	32.06	33.64
TV	26.42	29.24	31.28	32.97	34.53
	Cameraman
BCS-SPL	26.02	30.36	33.65	36.42	38.90
MH-BCS-SPL	29.86	33.97	36.57	39.28	41.48
MS-BCS-SPL	31.05	36.54	39.98	42.91	44.91
MH-MS-BCS-SPL	31.71	38.08	42.85	45.63	48.15
MS-GPSR	29.53	34.76	39.59	41.82	45.08
TV	30.87	35.08	38.32	41.09	43.73
	Girl
BCS-SPL	29.63	32.69	34.76	36.46	38.07
MH-BCS-SPL	31.74	34.84	36.68	38.41	39.85
MS-BCS-SPL	33.49	36.29	38.87	39.70	40.63
MH-MS-BCS-SPL	33.58	36.75	39.08	40.43	41.81
MS-GPSR	32.18	35.24	37.70	38.59	39.89
TV	30.26	33.53	35.87	37.77	39.51
	Barbara2
BCS-SPL	23.64	25.56	27.19	28.75	30.38
MH-BCS-SPL	26.83	30.19	32.03	33.71	35.27
MS-BCS-SPL	25.14	27.31	29.12	30.34	31.66
MH-MS-BCS-SPL	25.61	28.87	31.06	33.74	36.30
MS-GPSR	25.28	27.34	28.82	30.19	32.21
TV	23.90	26.24	28.51	30.74	32.97

Visual Comparison

The following example shows a visual comparison of the various algorithms. All the images are reconstructed using different algorithms at sampling subrate = 0.1.

*Lenna*

BCS-SPL Reconstructed Image, PSNR = 28.03 dB	MH-BCS-SPL Reconstructed Image, PSNR = 29.76 dB

MS-BCS-SPL Reconstructed Image, PSNR = 31.43 dB	MH-MS-BCS-SPL Reconstructed Image, PSNR = 31.55 dB

MS-GPSR Reconstructed Image, PSNR = 30.28 dB	TV Reconstructed Image, PSNR = 29.83 dB
*Barbara*

BCS-SPL Reconstructed Image, PSNR = 22.35 dB	MH-BCS-SPL Reconstructed Image, PSNR = 27.91 dB

MS-BCS-SPL Reconstructed Image, PSNR = 23.91 dB	MH-MS-BCS-SPL Reconstructed Image, PSNR = 24.31 dB

MS-GPSR Reconstructed Image, PSNR = 24.03 dB	TV Reconstructed Image, PSNR = 22.95 dB
*Cameraman*

BCS-SPL Reconstructed Image, PSNR = 26.17 dB	MH-BCS-SPL Reconstructed Image, PSNR = 29.77 dB

MS-BCS-SPL Reconstructed Image, PSNR = 31.01 dB	MH-MS-BCS-SPL Reconstructed Image, PSNR = 31.75 dB

MS-GPSR Reconstructed Image, PSNR = 29.52 dB	TV Reconstructed Image, PSNR = 30.77 dB