 Research article
 Open Access
 Published:
Cryoelectron microscope image denoising based on the geodesic distance
BMC Structural Biologyvolume 18, Article number: 18 (2018)
Abstract
Background
To perform a threedimensional (3D) reconstruction of electron cryomicroscopy (cryoEM) images of viruses, it is necessary to determine the similarity of image blocks of the twodimensional (2D) projections of the virus. The projections containing high resolution information are typically very noisy. Instead of the traditional Euler metric, this paper proposes a new method, based on the geodesic metric, to measure the similarity of blocks.
Results
Our method is a 2D image denoising approach. A data set of 2243 cytoplasmic polyhedrosis virus (CPV) capsid particle images in different orientations was used to test the proposed method. Relative to Blockmatching and threedimensional filtering (BM3D), Stein’s unbiased risk estimator (SURE), Bayes shrink and Kmeans singular value decomposition (KSVD), the experimental results show that the proposed method can achieve a peak signaltonoise ratio (PSNR) of 45.65. The method can remove the noise from the cryoEM image and improve the accuracy of particle picking.
Conclusions
The main contribution of the proposed model is to apply the geodesic distance to measure the similarity of image blocks. We conclude that manifold learning methods can effectively eliminate the noise of the cryoEM image and improve the accuracy of particle picking.
Background
The theory of threedimensional (3D) reconstruction of electron cryomicroscopy (cryoEM) was defined in the 1960s when Aron Klug and his research group reconstructed the lowersolution 3D structure of a biological macromolecule by means of Transmission Electron Microscopy (TEM) [1]. Aaron Klug won the Nobel Prize for Chemistry in 1982 for his groundbreaking work. The use of cryoEM images is considered to be the most efficient method for obtaining a 3D density map of a complex biological structure [2]. A cryoEM image is a projection of a biological sample taken by the electron microscope. Environmental factors typically produce noise, and Gaussian noise is generated in the process of digitalizing images. Denoising can eliminate cryoEM image noise originating from the processes of collection, transmission, and storage of images [3] and can also improve the signaltonoise ratio of cryoEM images and the quality of single particles. The highquality particle picking are used for 3D reconstruction to obtain the 3D structure of the biological samples in real space. Therefore, cryoEM image denoising has great significance for 3D reconstruction [4, 5].
The noise in cryoEM images will affects the procedure of adjustment and single particle extraction in 3D reconstruction [4]. In image processing, since most of the noise comes from interference from electronic devices and the like, the Gaussian and Poisson noise models are often used in actual modeling. At present, most of the denoising algorithms are designed to process white Gaussian noise in images. The probability density function has a normal distribution, and the power spectral density function is a constant [5].
In recent years, with the development of structural biology, cryoEM images have become increasingly more important. A large amount of multidimensional data generated by standard biomedical imaging modalities, such as electron microscope images and denoising nuclear magnetic resonance image processing, has been used to analyze the distribution of noise in cryoEM images [6, 7], explore methods for electron microscope image denoising, to reduce noise in images and provide highquality images for subsequent image processing.
The image block matching algorithm, which is based on the redundancy and correlation of the image information, finds the reference block class to which the candidate block belongs by calculating the distance between the candidate block set X and the reference block set R [8]. The image is segmented according to the noise level in the image [9]. Because images in the image block can be used as a reference block several image blocks are randomly selected as reference blocks. For each image block, the similarity to the reference block is calculated separately, which involves calculating the distance from the reference block. Image blocks that are less than a certain threshold distance from the reference block are considered similar blocks and are classified accordingly until all the image blocks find the corresponding similar block group [10].
Noise type, noise intensity, and image block size are the key factors that affect the performance of denoising in the design of the denoising algorithm based on the image block [8]. Therefore, taking into account the effect of noise on the effective information of the image block, the process analyzes the noise sources of the cryoEM image and sets the image block size according to the noise standard deviation. The primary methods for cryoEM image denoising are as follows: twodimensional (2D) projection image classification, which focuses on denoising in the transform domain but encounters challenges distinguishing single particles and background noises, and the Taneli Mielikäinen method [11], which introduces the radon transform for image denoising of the single particles but relies on the accurate determination of the commonline of singleparticle projection and which cannot extract a single particle from the original cryoEM image successfully due to the noise. WANG [12] proposed to combine the Zernike matrix and nonlocal means for cryoEM image denoising, but this method is not suitable for biological macromolecules with nonicosahedral symmetry structure. A nonparametric denoising method combines the contourlet transform and Bayesian estimation, but this method does not take into account the structural characteristics of identical particles in the cryoEM image.
The denoising method above is based essentially on a Euler metric. Prompted by the increasingly wide use of manifold learning, we introduce the geodesic metric to improve the picture quality effectively. Related research has shown that the structure feature of image blocks can be used to improve the performance of image processing [9]. However, as mentioned, cryoEM image denoising methods are not integrated into the structure of the image block, so there is still some room for improvement to denoise images [8, 13, 14]. The present paper presents a new method to denoise cryoEM images by using image blocks. First, blocks that are similar to reference blocks are searched for in the entire image through nonlocal selfsimilarity prior learning and a similar blockbased matching algorithm based on geodesic distance. Second, images are processed with additional Gaussian noise through prior learning; then, image blocks with sparse representation and nonlocal means are denoised. Finally, all the denoised images are reconstructed to obtain the denoised cryoEM image.
Results
Experiment configuration
The image block, noise type, noise intensity, and image block size are the key factors that affect the performance of denoising in the associated algorithm design. Therefore, taking into account the effect of noise on the effective information of the image block, the algorithm analyzes the noise sources of the cryoEM image and sets the image block size according to the noise standard deviation. Table 1 shows the value of the image block size p and the noise standard deviation σ = 50.
We used real cryoEM image data to test our method. This data set contains 2243 images of CPV in total. Particles were randomly selected from the data set and divided into five groups, whose number was different in each group. The size of the images of the five data groups ranged from 320 × 320 to 4096 × 4096. The increasing image size scaled as 4, 9, 16 and 163.4 times that of the original. The specific parameters are shown in Table 2. (Fig. 1).
Experiments on similar block selection
Two kinds of methods are widely used to extract image blocks. One is nonrepetitive block extraction using the ‘blkproc’ function in MATLAB, and another is overlapping extraction, which allows the existence of repeated pixels in image blocks [6]. To improve the denoising effect, overlapping extraction is typically used in actual image processing. As shown in Fig. 2, this technique is similar to that of the partial block in Fig. 1(d).
The green box represents similar blocks located on the edge of the particle, and the red box indicates similar blocks whose middle point is at the center of the particle. Because the identities of the CPV virus particles used in the experiment are known, we ignore the noise and other factors; the single particles in the cryoEM image have similar structure information.
Experiments on different image block matching algorithms for CryoEM images
Results and analysis of CryoEM image Denoising based on the geodesic distance
For four cryoEM images with different sizes using the Euclidean distance and geodesic distance to select similar blocks, we demonstrate that the proposed similar block matching algorithm based on the geodesic distance is efficient, and we analyze the influence of the image size and the number of blocks on denoising. For denoising of four cryoEM images, through similar block matching, we use the Euclidean distance and the measured distance to measure the similarity blocks separately and then recorded the PSNR and SSIM value and the time required for denoising. The experimental parameters are set to p = 8 and σ = 50, and the experimental results are shown in Table 3.
As the size of the image increases, the PSNR and SSIM values are improved after denoising because when the image block size is fixed, the larger the image, the greater is the number of similar blocks that can be used to learn, and the better the Gaussian component obtained by prior learning can describe the structural features of the image block. For the same image, higher PSNR values and SSIM values were achieved when using the geodesic distance to measure the similarity between the image blocks. Which indicates that accuracy of similar blocks is improved by using similarblockbased matching to enhance the effect of denoising.
Table 4 shows that as the image size scales by 4 times, 9 times, and 16 times, the original denoising time scales by the same amounts. Thus, as the image size becomes larger, the time consumption increases. For the same image, the denoising time when using the geodesic distance is slightly longer than that using the Euclidean distance because the Euclidean distance considers only the gray value of the image. However, the proposed geodesic distance takes into accounts both the gray value and gradient values of the image. As before, the geodesic distance is more accurate (albeit while increasing the denoising time), and a comparison of PSNR and SSIM shows that using the geodesic distance can effectively improve these metrics.
In this paper, the geodesic distance is used to replace the Euclidean distance to measure the similarity between image blocks. At the same time, the computation time increases with increasing accuracy. Relative to other methods, when the noise standard deviation is the same, the proposed method can achieve a higher PSNR value. In addition, the 3D reconstruction of single particle cryoEM image is a computationally intensive task; in the complete data processing of obtaining biological macromolecules, denoising represents only a small proportion of data processing. The denoising of cryoEM images effectively improves the image quality and is helpful to obtain the highresolution 3D structure of biological macromolecules.
Experiments on different noise standard deviations for CryoEM images
The proposed method was applied to the denoising of the image shown in Figs. 1(b), and 3(b)(f) shows the experimental results under different noise standard deviations. The image size is 640 × 640, and the experimental parameters are set as shown in Table 1.
When the electron microscope produces real imaging data, noise is generated during the process; however, as shown in Fig. 3(c), the effect of real data denoising is very effective. With increasing noise in the image, the visual effect of the image is reduced after denoising. Table 5 shows the specific PSNR value and denoising time, where ΔPSNR is the difference between the final PSNR value and the initial PSNR value.
As shown in Table 5, with the noise standard differential increases, the initial image PSNR value begins to decrease, the PSNR of the denoised image declines gradually, and ΔPSNR values increases significantly, which indicates that the proposed method can effectively remove the noise. Furthermore, with increasing noise in the image, the denoising time becomes longer, which indicates that the noise intensity is related to the denoising time.
Comparative experimental analysis
To reconstruct a highresolution 3D model, cryoEM images with a low signaltonoise ratio (SNR) and a complex particle structure must be processed effectively. In this aspect of image processing, the block denoising method represented by BM3D can effectively aggregate similar blocks into a 3D array and implement cofiltering in the transform domain, which has a favorable effect on image denoising. CryoEM images can be regarded as gray images, which can be processed by the BM3D [15], SURE [16], Bayes shrink [17] and KSVD [18, 19] methods.

(a)
BM3D [15]: This method is a denoising method based on image blocks; it aggregates the similar blocks into a 3D array and executes collaborative filtering in the transform domain.

(b)
SURE [16]: Stein’s unbiased risk estimate transforms the denoising process into solving the linear equations in the wavelet domain by minimizing the MSE, and the solutions can be used to denoise the image.

(c)
Bayes shrink [17]: The image is processed with the wavelet transform, and the threshold value of each subband is selfadaptively solved by Bayes estimation. The wavelet coefficients are transformed using a soft threshold function; finally, the noise is removed.

(d)
KSVD [18, 19]: This method is an effective and complete method of training sparse signal representation that can achieve image denoising based on a dictionary.
The proposed method is first compared to BM3D, SURE, Bayes shrink and KSVD, and then all five methods are used to denoise a cryoEM image to verify the superiority of the method proposed in this paper. In the experiment, the parameters are set to \( p=8\kern0.5em and\kern0.5em \sigma =30 \).
Figure 4 shows that the use of the proposed denoising method results in better visual effects. The method can change the original cryoEM image limitations with low contrast and single particles with a clear outline and edge. The method also facilitates the designs of automatic particle selection algorithms [5], the accuracy of single selected particles, and the resolution of 3D reconstruction. Moreover, it reduces the total time of 3D reconstruction.
Table 6 indicates that the proposed method exhibits the highest PSNR value; this result arises because the proposed method takes full advantage of the structural features of the image blocks. The image block averaging can effectively suppress the noise, and through the prior learning of structural information of blocks and sparse presentation, the method denoises image blocks while accurately preserving the details of the image. With increasing noise in the image, the PSNR value is decreased, which indicates that the noise intensity affects the denoising time.
Testing the method on an existing publicly available benchmark data set
To test the validity of the proposed method, public data sets—which are available on the 3D Electron Microscopy Benchmark website—are used for experimental evaluation. In the experiment, we selected the Ad2 ts 1 Data Set for comparison, in which the parameters were set to \( p=8\kern0.5em and\kern0.5em \sigma =40 \), and the graphs made in Ad2_ts1_Data Set_I_Test were used.
The experimental results are shown in Fig. 5. For the different methods, the experimental results of the PSNR and time consumed are displayed in Tables 7 and 8, respectively. Figure 5 shows that the proposed denoising method results in better visual effects.
Experimental results and analysis of 3D reconstruction
CryoEM image Denoising
According to the method described in the paper, the denoising experiment is carried out on 7000 CPV capsid particle images of 4096 × 4096 in size on multiple servers; the parameters are set to \( p=12\kern0.5em and\kern0.5em \sigma =10 \). The partial results after the denoising experiment are shown in Fig. 6. When the noise standard is different, the experimental results of PSNR and SSIM are shown in Table 9.
Table 9 represents the experimental data showing the PSNR and SSIM of the four images in Fig. 6 on different noise levels. The PSNR and SSIM decrease dramatically, and the noise increases and denoising effects worsen as the noise standard deviation increases, which verifies that the noise level strongly affects the performance of the denoising method. For a constant noise standard deviation, the PSNR values of the four images after denoising are similar, as are those of the SSIM, which indicates that the proposed method can remove noise from cryoEM images effectively.
In Fig. 7, the horizontal coordinates represents the noise standard deviation, and the vertical coordinates represent the PSNR value of the denoised image. With increasing noise standard deviation, the PSNR value decreases significantly. For a constant noise level, the four images have similar PSNR values after denoising, which again verifies that the proposed method can effectively remove noise from cryoEM images.
The results and analysis of the extraction experiment of single particles
The fundamental principle of selecting single particles [5] in their 3D reconstruction under cryoEM is that the particles selected must be fine and isolated particles that are unaffected by ice crystals. The use of images with a higher PSNR and contrast obtained after the denoising of the cryoEM images (as discussed in CryoEM image Denoising section) contributes to quick and accurate particle selection. Over the many years of development of electron cryomicroscopy, multiple automatic particle selection algorithms have been proposed. EMAN [20] is an example of an automatic particle selection algorithm. However, due to the limitation of these algorithms and larger deviation of automatic selection, automatic selection is inefficient. However, manual particle selection can achieve satisfactorily accurate performance only when the number of particles is small. With an increasing number of particles, the large consumption of labor becomes prohibitive, and omission and other mistakes can easily occur because of the complex distribution of particles when manual particle selection is selected. Therefore, the false positive rate (FPR) and true negative rate (TNR) are used to judge the quality of singleparticle selection, the true positive rate (TPR) is adopted to indicate whether the selection algorithms can accurately recognize all particles, and the time consumed during selection is used to show the time complexity of the selective methods. Table 10 displays the results of the selection of single particles on a CPV virus image by using different methods.
The TPR of the manual selection is the highest, but this method also consumes the highest time. In the actual application, the images are automatically selected at first, and then partial particles are manually adjusted when the number of particles is large; thus, more particles can be recognized to the largest extent in less time. Figure 8 shows a projection image of the CPV virus used for particle selection.
Some particles at the edges of the image are incomplete, some particles in the purple frames are affected by ice crystals, impurities are mixed with particles in the red frame while freezing samples, and the blue frames in the image indicate that these particles would influence the final results of reconstruction. These are all sources of large deviations in the experiment of extracting single particles. To increase the accuracy of singleparticle extraction, the above mentioned particles are abandoned at the stage of selecting particles, and only homogeneously distributed and isolated particles remarked with green frames (only partial particles are marked) in the image are selected to carry out 3D reconstruction. Figure 9 shows a schematic diagram of the manual selection of partial particles.
The distributive rules of mutually selected particles are reviewed. Each single particle can accurately falls into a square frame, in which the selected particles can be stored. Figure 10 shows the schematic diagram of the storage results of 40 single particles through manual selection.
Parts of single particles in the figure are used in 3D reconstruction. To obtain the precise structure of the CPV virus, more than thousands of single particles are needed. After the selection of particles, the center and orientation of each particle need to be measured. However, the data of different projections are different and depend on the properties of samples and the resolution requirements of the reconstruction. For the 3D reconstruction with low resolution, hundreds of icosahedral particles are sufficient. However, 3D reconstruction with high resolution requires additional particles, and 3D reconstruction with subnanometer resolution requires thousands of particles. To obtain a structure with nearatomic resolution, approximately 50,000 to 100,000 icosahedral virus particles need to be imaged. In the experiment, a data set of 7000 CPV capsid particle images are used for reconstruction, on which superimposed averaging was performed in the Fourier transform domain. After eliminating parts of particles from which no easily detectable rings could be obtained through the superimposition, 6500 fine particles are used for reconstruction eventually, corresponding to 92.86% of all single particles. The same number of single particles selected from nondenoised cryoEM images are used to conduct the experiment, from which only approximately 6000 single particles are found that can be used for the reconstruction calculation, accounting for 85.8% of the total number of selected particles, and almost 500 single particles are eliminated, corresponding to a decrease of 7.06%. The extracted single particles are reconstructed by using the central section theorem to obtain the structure of the CPV virus, as shown in Fig. 11. The experimental data show that the extraction accuracy and overall quality of single particles can be improved by first denoising the cryoEM images and then conducting semiautomatic selection of single particles; in this way, the 3D reconstruction resolution can be enhanced.
Discussion
We evaluate the competing methods from four aspects: PSNR, contrast, time and visual quality.
PSNR
The results are presented in Table 6. We have compared the PSNR of five methods on five noise levels: Δ = 10, 20, 30, 40, and 50. The KSVD method performs worst; BM3D, SURE and Bayes shrink obtain results similar to those of PSNR, and the method we proposed performs better than PSNR, with an improvement of approximately 14 dB relative to BM3D, SURE and Bayes shrink. These results validate that the proposed method has a significant ability to denoise images.
Contrast
The proposed method of denoising can significantly improve the PSNR of the image, which is indicated from the visual effect of the image; the contrast of the cryoEM image is improved, and a better visual effect is obtained because high similarity exists among each single particle in the cryoEM image used in the experiment. The substitution of the geodesic distance in place of the Euclidean distance improves the accuracy of searching similar blocks when similar blocks are matched and when the denoising effect is implemented through prior learning.
Time
The method proposed in this paper requires a longer time to denoise the cryoEM images because the data type of the cryoEM image is 32bit floating point numbers; consequently, as the image size increases, the time taken to read the images increases, and the time required to calculate the distance between image blocks with the increase of the numbers of image blocks, which need to be denoised during the matching of image blocks. Additionally, in this paper, the prior learning is first performed on similar blocks, and then the groups of image blocks are denoised; therefore, the denoising time is longer than that of the other four methods.
Table 11 shows that with increasing noise in the image, the denoising time also shows an increasing trend. The time required for SURE and Bayes shrink is relatively low, and the time required for the Bayes shrink is approximately 1.7 times that of SURE. When the noise level is the same, SURE requires a slightly longer denoising time than Bayes shrink. Relative to SURE and Bayes shrink, the BM3D algorithm takes significantly more time. The proposed method takes the longest time, approximately 10 times that of the BM3D algorithm.
Efficiency is another factor to evaluate the methods. We have compared the speed of the 5 methods under the same environment as presented in Table 2. In Table 11, we show the runtime results on the five noise levels Δ = 10, 20, 30, 40, and 50. Considering the application of parallel computing and the highspeed development of computation modules, the runtime makes up a smaller proportion of evaluation than before.
Visual quality
The visual quality plays an important role in the evaluation of any denoising method because human beings are the ultimate judge of image quality. Figure 4 shows the images denoised by the five method. The image processed by the KSVD remains fuzzy. In the images processed by SURE and BM3D, the edges cannot be clearly distinguished. Comparing the two images associated with Bayes shrink and the proposed method, we find that higher contrast is achieved by the latter. In general, the proposed method demonstrates a strong ability to denoise images.
Conclusions
A similar block matching algorithm based on the geodesic distance has been proposed in the paper and applied to the design of a denoising algorithm based on image blocks. The method is based on similar block matching, using the geodesic distance to measure the similarity of image blocks. The method searches for the similar blocks in the whole image field, enhancing the performance of denoising by improving the accuracy of similar blocks and denoising each group of similar blocks separately.
Similar blocks with additional Gaussian noise were treated by prior learning. Finally, the entire denoised image block was used to reconstruct the denoised cryoEM image. The experiments show that the proposed method can effectively eliminate noise in the cryoEM image.
Methods
State of the art on image Denoising based on image blocks
Dictionary learning has broad applications, including image recognition, denoising and restoration [18, 19]. The goal of dictionary learning is to find a sparse approximation solution to represent a class of signals under an appropriate measure. Moreover, sparseness can often be used to avoid overtraining. Current dictionary learning algorithms focus on selecting the vector on the Euclidean space. However, data points often modeled by a Riemannian manifold are critical to applications involving image denoising [20, 21].
In the paper, the image is divided into blocks according to the noise level in the image. Any image block in the image domain can be used as a reference block, and some image blocks are randomly selected as the reference blocks. For each candidate image, we calculate the similarity of each block to that of the entire reference block, that is, we calculate the distance between it and the reference block. When the distance is less than a fixed threshold value, we classify these two image blocks as similar and place them in the same similar block group; the process continues until all of the candidate image blocks are placed into a corresponding block group.
Similar block matching algorithm based on the geodesic distance
In the existing similar block matching algorithm [21], the Euclidean distance between the candidate block S_{x} and the reference block set \( {\left\{{\mathrm{S}}_{\mathrm{x}}\right\}}_{\mathrm{k}=1}^{\mathrm{K}} \) is typically used to calculate the similarity. However, the Euclidean distance does not take into account the local connectivity. Moreover, the image block subspace is not entirely Euler space. To overcome this limitation, in the paper, the Euclidean distance is replaced by the geodesic distance [22] to evaluate the similarity of image blocks. The geodesic distance considers the intrinsic influence on the image space, and its computation is not complex.
The image block is composed of pixels. For the two given image blocks, S_{A} and S_{B}, their sizes are both p × p; \( {\mathrm{d}}_{\mathrm{R}}\left({\mathrm{S}}_{{\mathrm{A}}_{\mathrm{i}}},{\mathrm{S}}_{{\mathrm{B}}_{\mathrm{i}}}\right) \) is used to represent the geodesic distance between the two image blocks in the ith pixels.
Calculate the weight of the ith pixel point weight1 and weight2:
where value [Ai] represents the gray values of the ith pixel point in an image block S_{A} and value [i] represents the gray values of the ith pixel point in image block S_{B}.
where tAi represents the gradient value of the ith point in the image block S_{A}. tBi represents the gradient value of the ith point in the image block S_{B}. α represents the angle of the ith point of the image block S_{A}, which is the angle between the direction of the maximum change of the gray value and the minimum direction. β represents the angle of the ith point of the image block S_{B}, which is the angle between the direction of the maximum change of the gray value and the minimum direction. Figure 12 shows a view of an angle.
Here, \( \upalpha =0,\frac{\uppi}{4},\frac{\uppi}{2},\frac{3\uppi}{4},\uppi \), and eight pixels exist around pixel A_{i, j}. The gray value changes are calculated to find the direction of the maximum change in the guidance of the maximum change and the direction of the smallest change, and the angle is defined as α. In the same way, we know that\( \upbeta =0,\frac{\uppi}{4},\frac{\uppi}{2},\frac{3\uppi}{4},\uppi \). Consequently, the geodesic distance between two points Ai and Bi, which are located at the same position in different image blocks, is defined as
Therefore, the geodesic distance between the image blocks S_{A} and S_{B} is
where i is the ith pixel point of the image block and i = 1, 2, …, p × p. Comparing d(S_{A}, S_{B}) with the fixed threshold T, if d(S_{A}, S_{B}) < T, we define the image blocks A and B as similar blocks; otherwise, the image blocks A and B are not similar. The value of the fixed threshold T is related to the size of the image blocks. The larger the image block, the larger the value of T.
In the image domain, we can use a 2D discrete function to represent the image, and the gradient direction is the direction of the maximum change of the gray value, so this paper uses the gradient value and the gray value of the image to describe the geodesic distance. When the distance between two image blocks is less than the threshold, the two image blocks are considered to be similar blocks. The value of the threshold is related to the size of the selected image block. Figure 13 shows the flow chart of using the geodesic distance to measure the similarity of the image block.
In the paper, we use the proposed method to evaluate the similarity of the image blocks; then, the geodesic distance is used to select the similar block for the reference blocks. The detailed similarblockbased matching algorithm based on the geodesic distance is shown in Table 12.
The accuracy of similar blocks has a strong impact on the accuracy of the subsequent learning of similar blocks and affects the accuracy of the dictionary and sparse coding of the similar blocks, which could affect the final denoising result. The proposed block matching method is based on the geodesic distance, which enhances the accuracy of the similar block group and lays a foundation for establishing a more precise denoising model.
Evaluation Indicator of the Denoising effect
A suitable denoising algorithm would remove the noise to the maximum extent, maintaining the integrity of the valid information of the original image while having relatively low computational time complexity. It is typical to evaluate the performance of denoising methods with objective evaluation indexes and visual effects of images.
When evaluating the denoising performance of a specific algorithm, in addition to comparing the visual effect, indicators such as the PSNR, mean square error (MSE), SNR and execution time are used to measure the advantages and disadvantages of the algorithm.\( MSE=\frac{1}{N^2}\sum \limits_{i,j=1}^N{\left(\overset{\wedge }{X_{i,j}{X}_{i,j}}\right)}^2 \), where X is the original image with noise and \( \hat{X} \) is the estimate of the original image X that is the denoised image.
\( PSNR=10\log 10\frac{\max \left({x}^2\right)}{MSE} \), where \( SNR=10\log 10\frac{P_s}{P_n} \), P_{s} denotes the effective signal power, and P_{n} denotes the noise power.
For image processing, apart from PSNR and MSE used to evaluate an algorithm, the structural similarity index measurement system (SSIM) is also a reliable indicator that is based on the correlation of neighboring pixels in natural images. SSIM avoids tallying up different kinds of errors to depict the image differences before and after denoising. The closer to one the SSIM value between the noise image X and the denoised image \( \overset{\Lambda}{X} \) is, the more similar in structure they are.
\( SSIM\left(X,\overset{\wedge }{X}\right)=\frac{\left(2{\mu}_1{\mu}_2+{c}_1\right)\left(2{\sigma}_{12}+{c}_2\right)}{\left({\mu}_1^2+{\mu}_2^2+{c}_1\right)\left({\sigma}_1^2+{\sigma}_2^2+{c}_1\right)} \), where μ_{1}and μ_{2} denote the average, \( {\sigma}_1^2 \) and \( {\sigma}_2^2 \) denote the variance, σ_{12} denotes the covariance, and c_{1} and c_{2} are constants close to 0. PSNR and MSE are based on the statistical model of the image grayscale value, while SSIM is based on differences of image structures. In practical use, it is typical to combine subjective evaluation and objective evaluation to evaluate an algorithm.
The CTF plays a significant role in the comparison of cryoEM data. To obtain a highresolution 3D reconstruction of a virus by a cryoEM image, it is necessary to implement a CTF correction for the micrographs. It is hard to distinguish the positions of the CTF zeros accurately due to the low signaltonoise ratio of the cryoEM image. To avoid an inaccurate measure of the positions of the CTF zero blurred by attenuation at high frequency, we use a Gaussian curve to compensate for the attenuation of the Fourier transform of the image at high frequency; in this way, the amplitudes of the two CTF zeros at the curve are the same value.
Proposed CryoEM image Denoising
The purpose of cryoEM image denoising is to remove the noise in the image, improve the contrast and the SNR of the picture, and provide sufficient information for the following singleparticle selection and 2D projection image classification. Our method implements prior learning of the image block and sparse representation and then uses the dictionary representation to denoise the image block. We use the method of learning from similar blocks to obtain the dictionary, which avoids the limitation of using the discrete cosine transform (DCT) dictionary [23, 24]. The dictionary denoising theory is based on the ideal image having sparse representation under the appropriate overcomplete dictionary; the noise can destroy the sparse representation. By choosing or designing appropriate dictionaries, the sparse representation of natural images in the dictionary can be achieved to reduce or eliminate the noise.
A significant number of identical particles exist in the cryoEM images. The image block can effectively use the characteristics of these identical particles and achieve improved experimental results. In the paper, the similar block matching method based on the geodesic distance is combined with the nonlocal selfsimilarity (NSS) prior knowledge of image blocks [9] to search the similar blocks of the reference blocks in the whole image domain, followed by the process of prior learning with the image blocks. The proposed method takes into account the distance in the manifold space and uses the geodesic distance to select similar blocks accurately. In addition, the proposed method gives the prior internal knowledge and the external prior knowledge of the similar blocks.
The hypothesis observation image is y, the freenoise image is x, the noise is v, and y = x + v; here, \( \mathrm{PSNR}=10\log 10\frac{255^2}{\mathrm{MSE}} \). Thus, the cryoEM image denoising problem is transformed into obtaining an estimate \( \widehat{x} \) of an image x by observing the image y so that \( {\left\Vert \mathrm{x}{}_{\mathrm{x}}^{\hat{\mkern6mu}}\right\Vert}^2 \) the denoising problem can be minimized. This process determines the minimum MSE, which can be used to obtain the maximum PSNR value and optimal denoising result.
A cryoEM image was decomposed into image blocks; N reference blocks were chosen. In our process, the geodesic distance is used to select the similar blocks, and then all the similar blocks are clustered into N similar blocks, each of which contains M similar blocks. y_{m} represents the image block in the picture y, and x_{m} represents the image block in the picture x. According to the formula \( \mathrm{PSNR}=10\log 10\frac{255^2}{\mathrm{MSE}} \), to make the image PSNR as large as possible, the MSE must be as small as possible. Therefore, the image denoising problem can be converted to the minimum MSE problem, that is,
Here, \( \left\langle u\right\rangle =\frac{1}{p^{\ast }p}\sum \limits_{i=1}^{p^2}{u}_i \) because the nonlocal means can suppress the noise, and the dictionary can effectively represent the nonnoise signal in the image. Therefore, combined with the nonlocal mean and dictionary representation, we can obtain the denoised image block. By solving the dictionary D and sparse coding coefficient, the purpose of denoising image blocks can be achieved, and the image blocks can ultimately be used to reconstruct the denoised cryoEM image. The denoising flow chart is shown in Fig. 14.
\( {\left\{{\mathrm{y}}_{\mathrm{m}}\right\}}_{\mathrm{m}=1}^{\mathrm{M}} \)refers to M similar blocks with size p × p in the image y; here, \( {\mathrm{y}}_{\mathrm{m}}\in {\mathrm{R}}^{{\mathrm{p}}^{2\ast 1}} \). The mean value of M image blocks is expressed by μ_{y}, where \( {\mu}_y=\frac{1}{M}{\sum}_{m=1}^M{y}_m,\overline{y_m}={y}_m{\mu}_y \), \( \overline{Y}\underline {\underline{\Delta}}\left\{\overline{y_m}\right\},m=1,\dots, M \), and \( \overline{Y}\underline {\underline{\Delta}}{\left\{\overline{y_{n,m}}\right\}}_{m=1}^M,n=1,2,\dots, N;m=1,\dots, M \).
The next step is to implement a priori learning of \( \overline{{\mathrm{Y}}_{\mathrm{n}}} \) to calculate the K Gaussian distribution.
According to the sparse redundancy of images, the probabilistic representation is calculated by\( \left\{\overline{{\mathrm{Y}}_{\mathrm{n}}}\right\}:\mathrm{P}\left(\overline{{\mathrm{Y}}_{\mathrm{n}}}\right)={\sum}_{\mathrm{k}=1}^{\mathrm{K}}{\uppi}_{\mathrm{k}}{\prod}_{\mathrm{m}=1}^{\mathrm{M}}\mathrm{N}\left(\overline{{\mathrm{y}}_{\mathrm{n},\mathrm{m}}}{\upmu}_{\mathrm{k}},{\sum}_{\mathrm{k}}\right) \).
Therefore, the global target likelihood function can be expressed as \( \mathrm{L}=\prod \limits_{\mathrm{n}=1}^{\mathrm{N}}\mathrm{P}\left(\overline{{\mathrm{Y}}_{\mathrm{n}}}\right) \) for the convenience of the following calculation, namely, the logarithmic function of the target likelihood
Through GMM learning [25], we can obtain the K Gaussian distribution, which can describe the structural characteristic of the image block. In this context, the maximum posterior probability of each block is obtained by using the Bayesian method, and the most suitable Gaussian component for each group of similar blocks is obtained. Then, the dictionary of the similar block group is obtained to denoise the image block. The algorithm is described in Table 13.
According to the sparse representation of the image, \( {\overline{\mathrm{y}}}_{\mathrm{m}}=\mathrm{D}\upalpha +\mathrm{v} \), where v is noise. Dictionary D is known. By solving α, the sparse representation of the image block can be obtained, and the image block’s denoising can be realized. In the paper, the constraint conditions of the sparse coding model are expressed as
where α is the sparse coding coefficient and w is the weight of the α vector. According to the method provided by PGPD [13, 25],
where \( {\mathrm{w}}_{\mathrm{i}}=\frac{\mathrm{c}\ast 2\sqrt{2}{\upsigma}^2}{\uplambda_{\mathrm{i}}+\upvarepsilon} \) and \( {\mathrm{D}}^{\mathrm{T}}{\overline{\mathrm{y}}}_{\mathrm{m}}=\mathrm{z} \),
which is typically written in the following form:
\( \widehat{\mathrm{a}}=\operatorname{sgn}\left({\mathrm{z}}_{\mathrm{i}}\right){\left(\left{\mathrm{z}}_{\mathrm{i}}\right\frac{{\mathrm{w}}_{\mathrm{i}}}{2},0\right)}_{+} \), where (a)_{+} = max(a, 0)and sgn(∙) is a symbolic function. We define a function SoftMAP:
SoftMAP(g_{i}, τ_{i}) = sgn(g_{i})( g_{i} −τ_{i})_{+} is the sparse coding of the similar block group
to minimize the impact of noise in the image blocks and inaccurate similar blocks grouping in Gaussian mixture models (GMMs). The proposed geodesic distance can improve the accuracy of the similar block group. Moreover, this process uses a combination of the dictionary D and weighted sparse coding \( \widehat{\upalpha} \) to attain the denoised image block \( {\widehat{\mathrm{x}}}_{\mathrm{m}} \):
First, the estimated value of the image blocks in each similar block group is obtained, and then the denoised image \( \widehat{\mathrm{x}} \) is reconstructed by aggregating all the denoised image blocks. When some estimated values appear in a position of the image, the final estimate is obtained by using the weighted average. Through the formula \( {\left({\upsigma}^{\left(\mathrm{t}\right)}\right)}^2=\upeta \ast \left({\upsigma}^2{\left\Vert \mathrm{y}{\mathrm{y}}^{\left(\mathrm{t}1\right)}\right\Vert}_2^2\right) \) the noise is updated, and the standard deviation for several iterations η is a constant.
The difference between the proposed algorithm and PGPD [13, 25] is as follows: In this paper, the geodesic distance is used to replace the Euclidean distance in the PGPD algorithm to select the similar blocks and avoid the limitations of the Euclidean distance. Moreover, the proposed algorithm can search similar blocks in the whole image domain, while PGPD search for similar blocks in a slightly larger search window than the reference block. Here, the proposed algorithm is more accurate. PGPD adds no noise to image blocks for prior learning. In the proposed method, image blocks with additional Gaussian noise are used for prior learning. This technique gives full consideration to the noise affecting the image block sufficient information and to prior learning of additional noise image block directly. By using the property of the similar block group means value, the proposed method can reduce the noise in the image block, and the image block sparse representation can eliminate the noise signal; thus, the additional Gaussian noise in the image block can be eradicated.
Prospects
Based on multilayer neural network, deep learning with the feature of selflearning, which is input with massive data, has greater constructive and reasoning ability, thus, it can handle a variety of complex intelligent problems more effectively. In addition, deep learning also has more powerful learning ability and efficient feature expression ability Its more important advantage is that it can extract information layer by layer from pixellevel raw data to abstract semantic concept [26, 27], which makes it extracting the global features and context information of images more powerful and brings new ideas to solve traditional computer vision problems such as image segmentation and key point detection.
Therefore, it is believed that the application of deep learning in the 3D reconstruction of cryoEM images can exhibit better effects than the traditional methods, thereby it can enhance the resolution of biological macromolecules in 3D reconstruction process. The next step is to use the parallel and deep learning method to realize the denoising algorithm put forward in the paper, which can reduce the time used to denoise the cryoEM image.
Abbreviations
 2D:

Twodimensional
 3D:

Threedimensional
 BM3D:

Block matching and threedimensional filtering
 CPV:

Cytoplasmic polyhedrosis virus
 CryoEM:

Electron cryomicroscopy
 CTF:

Contrast transfer function
 DCT:

Discrete cosine transform
 FPR:

False positive rate
 GMM:

G
 KSVD:

Kmeans singular value decomposition
 MSE:

Mean square error
 NMR:

Nuclear magnetic resonance
 NSS:

Nonlocal selfsimilarity
 PDF:

Probability density function
 PGPD:

Patchgroupbased nonlocal selfsimilarity prior learning for image denoising
 PSNR:

Peak signaltonoise ratio
 SNR:

Signaltonoise ratio
 SSIM:

Structural similarity index measurement system
 SURE:

Stein’s unbiased risk estimator
 TEM:

Transmission electron microscopy
 TNR:

True negative rate
 TPR:

True positive rate
 WGN:

White Gaussian noise
References
 1.
De Rosier DJ, Klug A. Reconstruction of three dimensional structures from electron micrographs. Nature. 1968;217(5124):130–4.
 2.
Crowther RA, Derosier DJ, Klug A. The reconstruction of a threedimensional structure from projections and its application to Electron microscopy. Proc R Soc A Math Phys Eng Sci. 1970;317(1530):319–40.
 3.
Bai XC, et al. How cryoEM is revolutionizing structural biology. Trends Biochem Sci. 2015;40(1):49–57.
 4.
Ahmed S, et al. Nonparametric Denoising methods based on Contourlet transform with sharp frequency localization: application to low exposure time Electron microscopy images. Entropy. 2015;17(5):3461–78.
 5.
Joubert Paul, et al. “Automatic particle picking using diffusion filtering and random forest classification.” Revue Dhistoire De La Pharmacie 80(2011):108–109.
 6.
Li B, Shi L, Liu J, et al. A Method of Raster Data Mining Based on Multi Dimension Data Set[C]// International Conference on Fuzzy Systems and Knowledge Discovery. IEEE Press, 2009:392–6.
 7.
Herman GT, et al. “Threedimensional display of nuclear magnetic resonance images.” application of optical instrumentation in medicine IX International Society for Optics and Photonics; 1981. p. 35–40.
 8.
Kudo M, et al. Simple termination conditions for knearest neighbor method. Pattern Recogniti Lett. 2003;24(9):1203–13.
 9.
Luo H, Chu H, Xu Y. Clusteringbased image sparse Denoising in wireless multimedia sensor networks. Circuits Systems Signal Process. 2015;34(3):1027–40.
 10.
Jerrard RL, Maor C. Vanishing geodesic distance for rightinvariant Sobolev metrics on diffeomorphism groups; 2018.
 11.
Mielikäinen T, Ravantti J. Sinogram Denoising of CryoElectron microscopy images. Lect Notes Comp Sci. 2005;3483:1251–61.
 12.
Jia W, Yin CC. A Zernikemomentbased nonlocal denoising filter for cryoEM images. Sci China Life Sci. 2013;56(4):384.
 13.
Bhuiyan MIH, Ahmad MO, Swamy MNS. Spatially adaptive waveletbased method using the Cauchy prior for Denoising the SAR images. IEEE Trans Circ Syst for Video Technol. 2007;17(4):500–7.
 14.
Ye QZ. The signed Euclidean distance transform and its applications. Int Conf Pattern Recognit IEEE. 1988;1:495–9.
 15.
Dabov K, et al. Image Denoising by sparse 3D transformdomain collaborative filtering. IEEE Trans Image Process. 2007;16(8):2080–95.
 16.
Luo H, Chu H, Xu Y. Clusteringbased image sparse Denoising in wireless multimedia sensor networks. Circ Syst Signal Process. 2015;34(3):1027–40.
 17.
Daniel Zoran, Yair Weiss. From learning models of natural image patches to whole image restoration[J]. 2011;6669(5):479–86.
 18.
Aharon M, Elad M, Bruckstein A. $rm K$SVD: an algorithm for designing Overcomplete dictionaries for sparse representation. IEEE Trans Signal Process. 2006;54(11):4311–22.
 19.
Rubinstein R, Bruckstein AM, Elad M. Dictionaries for sparse representation modeling. Proc IEEE. 2010;98(6):1045–57.
 20.
Ludtke SJ, Chiu WBaldwin PR. EMAN: semiautomated software for highresolution singleparticle reconstructions. J Struct Biol. 1999;128(1):82.
 21.
Gu S, Zhang L, Zuo W, et al. Weighted Nuclear Norm Minimization with Application to Image Denoising[C]// IEEE Conference on Computer Vision and Pattern Recognition. IEEE Computer Society, 2014:2862–2869.
 22.
Cherian A, Sra S. Riemannian dictionary learning and sparse coding for positive definite matrices. IEEE Trans Neural Net Learn Syst. 2017;28(12):2859–71.
 23.
Mohimani H, BabaieZadeh M, Jutten C. A fast approach for overcomplete sparse decomposition based on smoothed l 0, norm[M]. IEEE Press; 2009.
 24.
Olshausen BA, Field DJ. Emergence of simplecell receptive field properties by learning a sparse code for natural images. Nature. 1996;381(6583):607–9.
 25.
Xu J, Zhang L, Zuo W, et al. Patch Group Based Nonlocal SelfSimilarity Prior Learning for Image Denoising[C]// IEEE International Conference on Computer Vision. IEEE, 2015:244–52.
 26.
Badrinarayanan V, Kendall A, Cipolla R. SegNet: A Deep Convolutional EncoderDecoder Architecture for Scene Segmentation. IEEE Trans Pattern Anal Mach Intell. 2017;99:1–1.
 27.
Feng W, et al. DeepPicker: a deep learning approach for fully automated particle picking in cryoEM. J Struct Biol. 2016;195(3):325–36.
Acknowledgments
The authors thank Professor Liu Hongrong of Hunan Normal University and his team for their cryoEM images and instructions. We thank the creators of methods such as PGPD and KSVD and those who provide code block learning, dictionary learning, and related experimental ideas and comparison experiments.
Funding
This research has been supported by NSFC(61672495), Scientific Research Fund of Hunan Provincial Education Department (16A208), Project of Hunan Provincial Science and Technology Department (2017SK2405), and in part by the construct program of the key discipline in Hunan Province.
Availability of data and materials
The results database is available at the following link: https://pan.baidu.com/s/1qjgeXJ21YIpDhibBFMHwZg
Author information
Affiliations
Contributions
Conceived and designed the experiments:JO ZL CC.Performed the experiments: JO ZL CC YZ. Analyzed the data: HL ZL ZF JO. Contributed reagents/materials/analysis tools: HL CC YZ. Wrote the manuscript: ZL JO ZF. All authors read and approved the final manuscript.
Corresponding author
Correspondence to Jianquan Ouyang.
Ethics declarations
Ethics approval and consent to participate
Not applicable.
Consent for publication
Not applicable.
Competing interests
The authors declare that they have no competing interests.
Publisher’s note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.
About this article
Received
Accepted
Published
DOI
Keywords
 Electron cryomicroscopy
 Geodesic distance
 Similar block
 Image denoising
 Particle picking
 Manifold learning