- Research article
- Open Access
Alternating evolutionary pressure in a genetic algorithm facilitates protein model selection
BMC Structural Biology volume 8, Article number: 34 (2008)
Automatic protein modelling pipelines are becoming ever more accurate; this has come hand in hand with an increasingly complicated interplay between all components involved. Nevertheless, there are still potential improvements to be made in template selection, refinement and protein model selection.
In the context of an automatic modelling pipeline, we analysed each step separately, revealing several non-intuitive trends and explored a new strategy for protein conformation sampling using Genetic Algorithms (GA). We apply the concept of alternating evolutionary pressure (AEP), i.e. intermediate rounds within the GA runs where unrestrained, linear growth of the model populations is allowed.
This approach improves the overall performance of the GA by allowing models to overcome local energy barriers. AEP enabled the selection of the best models in 40% of all targets; compared to 25% for a normal GA.
Impressive progress in protein structure modelling has been achieved over the last decade; however, improvement between subsequent rounds of the Critical Assessment of Techniques for Protein Structure Prediction (CASP) is often considered to be modest [1, 2]. Given the current accuracy, protein models are useful for qualitative analysis and decision-making in support of a wide range of experimental work. High accuracy modelling is essential for important applications such as, molecular replacement experiments [3–5], function predictions  and virtual drug screening . Modelling techniques, however, are still not accurate enough to close the gap between known protein sequences (approximately 5 million non redundant) and solved protein structures (approximately 50,000).
Regardless of the current limitations of modelling, two very encouraging observations have been made from the CASP7 results [1, 8, 9]. First, the gap between the quality of fully automated and manual modelling techniques has narrowed and second, improvement beyond the best template is achieved more frequently.
Modern template-based modelling pipelines can be divided into a number of common steps. A typical pipeline starts with template identification and alignment construction. In the next step, models are built using single templates, multiple templates or template fragments. The resulting models are then often refined, and finally the models are ranked and the "best" model selected.
Template search and alignment algorithms are showing significant improvements in accuracy and are becoming increasingly efficient. Well established sequence alignment algorithms such as FASTA , BLAST  and PSI-BLAST  are often replaced, or enhanced by more sensitive algorithms. These sensitive algorithms are based on multiple sequence alignments, sequence profiles or Hidden Markov models and take other information such as secondary structure prediction into account [13–17]. The impact of better alignments on the final model's quality is substantial, as errors made at this stage are not likely to be recovered during the subsequent modelling process.
Once a single template or several templates have been selected and the alignments constructed, models can be built. It is common practice to search the conformational space in order to further refine the structures [1, 18, 19]. Several different approaches have been developed for this task, using conserved constraints , genetic algorithms [21–25], Monte Carlo sampling , Molecular Dynamics , principle components analysis  or a combination of techniques [29–33]. Previous studies have shown that techniques combining several approaches perform best, when the different steps are carefully balanced .
For quality control and to reduce computational costs, protein model ranking and filtering can be applied at almost any stage of a modelling pipeline. Energy functions or statistical potentials are used to select a final model and usually form an integral part of the refinement method itself. Given their importance, the ability to select the best model based on energy alone is still relatively poor . Moreover, most energy scoring methods are optimised in the context of specific modelling approaches, and applying them in a different environment may produce less reliable results.
Model selection has become an important field of protein modelling, and a separate category has been introduced in the 4th Critical Assessment of Fully Automated Structure Prediction (CAFASP4) named Model Quality Assessment Programs (MQAP). The importance of this field was further underlined by a category called quality assessment (QA) introduced in CASP7 .
Several independent methods have been established and widely used to differentiate between models of high and poor quality [36–42]. Two different approaches can be distinguished; MQAPs scoring models in the context of model ensembles and MQAPs scoring single models independently. However, most of the top ranking MQAPs are dependent on the information of model ensembles [35, 38].
Despite all of the above efforts and improvement in protein model construction, ranking and selection, it is still not possible to consistently produce models of high quality. To further progress template-based modelling, it is necessary to carefully evaluate each step and minimise the accumulated errors. Here, we describe a hierarchical modelling approach (template-based modelling), where each step has been carefully evaluated, giving new insights into generating and selecting better models. A known limitation of Genetic Algorithm (GA) approaches in protein modelling is that models tend to end up in local minima, not exploring the conformational landscape enough to be able to find the global energy minima. As a way to alleviate this situation we have implemented the novel concept of Alternating Evolutionary Pressure (AEP) into our Genetic Algorithm (GA) search engine. In AEP intermediate rounds of unrestricted linear growth are introduced, enabling the models to overcome small energy barriers. The AEP approach is shown to promote greater sampling of the conformational landscape thereby enabling better structures to emerge, thus facilitating final model selection.
In the following, the dataset, the algorithms applied and the pipeline of the modelling approach are described. The core of the method is an optimization protocol based on a Genetic Algorithm (GA). This approach mimics the principles of evolution, combining and mutating protein model ensembles. Details of the GA approach used to model and refine protein structures can be found in a previous publication . To assure maximum yield, each step in the modelling pipeline has been evaluated separately. For an overview of the pipeline see Figure 1.
As the main objective of this work is to highlight problems in protein modelling and to extract potential solutions, the performance of the approach was benchmarked against well established modelling methods. We considered all sequences from the seventh round of CASP  which were downloaded from the Protein Structure Prediction Center webpage (http://www.predictioncenter.org/). The 104 protein sequences comprise 77 single and 27 multi-domain proteins. The final dataset consists of the 75 targets out of the 104 targets, for which reasonable templates could be identified.
Template identification, sequence alignments and initial model building
Templates were identified and sequence alignments constructed using the Hidden Markov model based algorithm HHpred  in conjunction with PSI-BLAST  results and the pdb70 database, downloaded from the HHsearch webpage ftp://ftp.tuebingen.mpg.de/pub/protevo/HHsearch. Standard values were used for HHsearch, as provided by the software distribution. In order to allow a fair and unbiased comparison with all template-based CASP7 servers, the data set was restricted to the information available at the time of CASP7. Therefore, the PSI-BLAST results, PSIPRED  secondary structure predictions and selected templates were created using time-stamped data.
For the initial model building, all side chains were stripped off the templates, and the query sequence assigned to the backbone, according to the HHpred alignment. At this stage, neither insertions nor deletions were modelled or side chains added.
For several targets, we were not able to identify the substantially better templates used by the HHpred servers. This might be due to the fact that only the pdb70 database is available for download; the HHpred server uses a combination of the pdb70 and pdb90 database. It seems reasonable that better templates can only be found once both databases are used in conjunction, especially in cases where single, isolated, templates of higher quality are available.
Due to deletions and insertions in the alignment, initial models are likely to be fractured. Missing residues within β-strands are especially hard to insert since it is very likely that any closing process will disrupt the precise hydrogen-bond network. On the other hand, substantial progress can be made focusing on coil and helical regions. We have developed a novel protein model repair protocol, which enables the modelling of incomplete coil and helical secondary structure elements with the correct length, thereby helping to further break away from the initial templates.
The new loop conformations are restricted to highly populated Φ/Ψ angles of the Ramachandran Plot  and backbone clashes are not allowed. The GA conformational search engine is applied to all initial models after backbone completion; it is, therefore, not necessary to extensively sample conformational space at this stage.
During processing, backbone bonds with non-standard length are first identified and fixed. For models with missing backbone elements, fragments are then created according to the PSIPRED secondary structure prediction, using internal coordinates with standard bond and angle values (IUPAC). These fragments are spliced into the backbone of the incomplete model and subsequently adjusted using a mechanism for closure, based on the cyclic coordinate descent algorithm . The procedure is fully automated and only requires a protein model and the PSIPRED prediction. All parameters used in the closing algorithm's procedure were derived from our GA algorithm, which was trained on the CASP6 and CAFASP4 datasets. For algorithmic details [see Additional file 1].
Several energy functions are used throughout the modelling pipeline. Preliminary investigations indicated that poor quality input models are not selected in the GA optimization process. Using model pre-ranking to remove these models at an early stage, allows more computational time to be spent on the better models. In the present approach, the best models are selected after optimization based on their energy scores. To quantify the ranking ability of our energy-scoring scheme for models, we calculate the Pearson correlation-coefficient between the energy and SC scores, defined below.
Structure Comparison (SC) score
In order to assess the quality of the models generated, a measure describing the conformational similarity between models and the known native structure (target) was required. We used a structural comparison scoring scheme, defined as the mean of the scaled TM , GDT  and maxsub  scores. Since all three scores are scaled to the range [0, 1], the final SC score also ranges from 0 to 1.
Energy scoring schemes
For each target all models were ranked using several different scoring schemes. A novel Fast Scoring Function (FSF) was used in the pre-ranking step. The FSF scoring scheme is composed of the following terms:S FSF = w1S pp + w2S cl + w3S ss
Where S pp , the residue-residue pair potentials, is a score of the internal packing according to an empirically derived mixed backbone atom-centroid potential, as described previously [21, 22]. S cl , the clash penalty, clashes are counted between two residues if any two backbone-atoms from any pair of non-consecutive residues are closer than 2 Å. S ss , the secondary structure score, is a sum of PSIPRED confidence scores for matches between predicted  and assigned secondary structure. The weights for the FSF were selected using the simplex algorithm  on the CASP6 and CAFASP4 datasets and are given by: w1 = 1.0, w2 = 2.4 and w2 = -2.4.
For scoring of the model populations during the GA optimization, the coarse energy score is used to preselect the models and reduce the population size to 100 models. Subsequently the fine energy score is used to further reduce the population to 50 models that are used for the next round.
The coarse scoring scheme includes the following terms:S coarse = w1S pp + w2S cl + w3S ss + w4S hb + w3ΔS comp
Where S pp , S cl and S ss are defined as for the FSF. S hb , the number of hydrogen bonds calculated using the software STRIDE . ΔS comp is the compactness reference score . The weights for the coarse scoring scheme are: w1 = 1.0, w2 = 2.07, w3 = -4.20, w4 = -0.46 and w5 = 1.37.
The fine scoring scheme includes the following terms:S fine = w1S eef + w2S se + w3ΔS comp
Where S eef is calculated in the following way. SCWRL 3.0 is used to replace all the side chains  for the energy calculation (standard parameters given by the program are used). All scored models are minimised and then scored using the effective energy function  (EEF) in CHARMM. S se , the solvent accessibility is calculated using the software POPS_A  and the solvation free energy . ΔS comp , as previously defined . The weights for the fine scoring scheme are: w 1 = 1.0, w2 = 0.20 and w3 = 0.20.
After optimization with the GA protocol, the final models are ranked with a combination of the fine and coarse energy scores, and an all atom pair-potential score, DFIRE . In the combined energy function the coarse and fine energy components are weighted according to the best template's sequence identity. Sequence identities were binned into three ranges: 0 – 0.3, 0.3 – 0.5, 0.5 – 1.0. For each of these ranges, the weights were optimised using the simplex algorithm on the CASP6 dataset, see Table 1.
The most representative structure of the final ensemble, which is taken from the middle of the largest cluster, was used as a control. This was done to investigate how consistently the top ranked solutions are selected, compared to the most representative conformation of the final ensemble.
To investigate whether full Cartesian space minimization facilitates protein model selection, all models, repaired and un-repaired, were minimised. The steepest descent and adopted basis Newton-Raphson methods were applied, as implemented in CHARMM  until the value of the gradient dropped below 1.0 kcal mol-1 Å -2.
We previously described an efficient move-set used to search conformational space of protein models . This move-set includes three global operators: the single and double crossover and the protein mutation operator; and two local operators: the helix and the coil mutation operators. A quick protein health check is performed during and after the application of the operators: Φ/Ψ angles must lie within the highly populated areas of the Ramachandran Plot and the change in energy-score is subject to a pseudo Metropolis criterion. The protocol was optimised using the CASP6 and CAFASP4 datasets.
The following modifications have been made to the protocol. Firstly, the input models are clustered using the nearest neighbour method. The metric for this clustering approach is based upon overall protein model similarity weighted with the secondary structure scores. Only the largest two clusters are used for further optimization, thereby removing a few, poor outliers. Secondly, the range of movement for mutations has been changed, to allow finer movements. This was achieved by allowing all values within the highly populated areas of the Ramachandran plot.
After applying the closing algorithm to all selected models, the models were submitted to five parallel runs of the GA protocol. The optimization is run for at least five, and a maximum of 10 rounds, dependant on the population convergence. Running the GA for longer was found to increase the probability of ending in incorrect local minima conformations (data not shown).
Alternating evolutionary pressure
In a GA where Alternating Evolutionary Pressure (AEP) is applied, a number of non-scored rounds are allowed between each scored sampling round. In these non-scored rounds, the population grows linearly and the structures in the ensemble are allowed to sample energetically unfavourable states. Although energy evaluation is not applied, to ensure reasonable sampling, basic protein health checks associated with the operators are still in place. Four different setups were applied: a normal GA and a GA with one to three non-scored intermediate rounds. In each setup, 10 fully ranked rounds were performed where the population was reduced to the top 50 models.
The complete modelling procedure can be accessed via a web server interface at: http://bmm.cancerresearchuk.org/~populus. The average running time for a protein model of 150 residues is 6–7 hours for the standard GA and 15–25 hours if two intermediate non-scored rounds are used (AEP2). For details of this server [see Additional file 1].
Results and discussion
For this study we modelled 75 diverse protein sequences from the CASP7 dataset of the category template-based modelling. First, a novel backbone repair algorithm is introduced and compared to the performance of MODELLER. In the next step an optimal setup for pre-ranking is investigated. The resulting models are recombined using a GA and the improvement in model selection due to the introduction of AEP is shown. Finally, a summary is given showing the performance of several possible modelling pipelines.
Structural comparison of models before and after repair
We have developed a novel algorithm for completing and closing protein backbones. Coil and helical regions are completed and the length of incomplete helical secondary structure elements is adjusted to agree with the predicted secondary structure. In contrast to other loop modelling methods, only a single conformation is created for each added structural element. These conformations are further sampled once the model undergoes recombination using the GA.
The distribution of improvement in SC score due to this repair process is shown in Figure 2. In 72% of all cases, completed structures show improvement in comparison to their initial score. The models improved by the repair algorithm show an average improvement of 0.015 SC score and the best 25% of the population (3rd quantile) shows an improvement greater than 0.02 SC score.
Analysing the SC score in terms of the assigned secondary structure, we found that approximately 80% of the improvement made for all models lies within helical secondary structure elements (see Figure 2 inset). In contrast to this, only approximately 20% improvement is gained completing coil regions. Approximately 60% of the improvement is situated in the core region, defined here as the region between the N and C terminal secondary structure elements. The rest of the improvement is located within the termini.
Comparing the models derived from the top ten alignments of each target, with the equivalent models constructed with the automodel function in MODELLER , the repairing algorithm scores on average 0.428 SC score compared to 0.427 SC score for MODELLER. Although the score for the closing algorithm is not significantly better, this method allows repair of models without the need for the alignment and/or template.
For modelling pipelines with extensive conformational search algorithms it is not obvious when to rank models. Ranking can be applied at several stages, such as before insertions and deletions are dealt with, after backbone completion, after minimization or after refinement. Intuitively, one might think that backbone completion is a minimum criterion to be fulfilled before further consideration on model quality can be made. To address this question, we analysed the effectiveness of the FSF and DFIRE scoring schemes before and after repair.
Figure 3 presents the Pearson correlation coefficient, i.e. the correlation between SC score and energy score, for different setups, and scoring schemes (the Spearman correlation-coefficient shows similar results). Surprisingly, the ranking of repaired models produces a lower correlation-coefficient than the ranking of un-repaired models. The same trend is seen whether the FSF or the DFIRE energy function is applied, showing this effect to be independent of the actual energy-based/statistical scoring function used. This observation can be explained by the following. Models derived from alignments with fewer insertions and deletions tend to be closer to the template and are generally of better quality. Due to the use of pair potential energy functions, models with more residues tend to have better energies. Hence before repair, the better models with fewer insertions/deletions have lower energy scores. Once repaired, this effect disappears and the advantage gained from the better initial template quality is not picked up by the energy functions anymore.
On the other hand, ranking according to the alignments scores given by the alignments algorithms is normally not sufficient for model selection either. Ranking purely based on the coverage dependant sequence identities of the alignments, produced a correlation score of 0.772, 12% smaller than the best FSF ranking.
In Figure 3 it can be seen that the best ranking is obtained using the FSF on unrepaired, minimised structures, improving the correlation by 6% compared to the best DFIRE configuration and by 13% compared to ranking using the weighted SID. Non-repaired models are generally easier to rank, this is valid both for DFIRE and the FSF. Ranking unrepaired models results in an improved correlation coefficient of 6.2% for DFIRE, 4.7% for the FSF. Minimization further facilitates ranking ability for unrepaired models by a further 1% using the FSF.
To further sample the conformational space and select a good final model, all repaired models were recombined and optimised using the GA. Figure 4 shows the median, first and third quantile of the different modelling populations pre/post GA. All GA runs only have ten fine energy scored rounds. Applying more than ten fine energy rounds increases the probability of convergence of the model ensemble into a local minimum [see Additional file 1, Figure S3].
Running the GA optimization using the SC score to the native protein structure as the fitness function shows how much improvement can potentially be [see Additional file 1, Figure S2]. After the application of the GA using the native structure as guidance, the model ensemble is very narrow and on average the final population is improved by 51% to an SC score of 0.643 for non-repaired and 0.658 for repaired models. Interestingly even in this ideal scenario the move set is unable to produce better structure due to a lack of good quality templates, absence of secondary structure elements in the model population or insufficient sampling. It can also be seen that repaired models clearly improve the overall population, due to the models not missing secondary structure elements. Longer sampling further increases the improvement, but for the purpose of comparison we limited the sampling to 10 rounds.
In practice we do not have access to the reference structure and have to rely on energy scores to drive the GA. Figure 4 also shows the results using different energy scores for final model selection. It can be seen that repairing structures does improve the top model, for all energy functions used. The best results can be obtained using the DFIRE energy function producing a 2.4% improvement compared to the fine score, a 1.4% improvement compared to the FSF score and a 0.5% improvement compared to the combined energy function.
The difference between the average SC score for the best models created using the energy and SC score to drive the GA is only 7%. However, a further 4% improvement is lost when selecting the model with the lowest energy out of the final energy driven ensemble.
Alternating evolutionary pressure (AEP)
GAs and other similar conformational search algorithms suffer from the problem that they tend to stay within local minima instead of exploring further afield and potentially finding a deeper minimum. We investigated whether alternating evolutionary pressure (AEP) could facilitate energy based model selection, by gently pushing models over small energy barriers. This idea is illustrated in Figure 5, showing that small changes in protein structure, although insignificant in terms of the SC score, produce significantly better energies, hence facilitating protein model selection.
Two main elements dictate the success of GA approaches. One is the set of operators (move-set) and the other is the fitness function (energy scoring scheme). Classically, GAs operate for several generations, iteratively applying the conformational search engine and the fitness function  until convergence is obtained. However, interesting results can be observed, once a series of conformational changes are applied, without intermediate population scoring and reduction. Within these non-scored intermediate rounds the population grows linearly. The finer energy evaluation and the reduction of the model ensemble to the best members are not applied; however, during these rounds the basic protein health checks of the operators are still applied (see Methods).
A similar approach to AEP was introduced by Qian et al., where the refinement of protein structures was achieved using an iterative alternation of diversification and intensification steps . This approach combines ideas from tabu search and conformational space annealing, however, it is only applied if the lowest energy refined structures have not converged and show several variable and less reliable regions. In general, this methodology is different from classical GAs where the optimization process is more variable, less directed and, therefore, convergence is only achieved after intensive sampling. For these classical GAs, the principle of AEP has only been used before to provide theoretical predictions of algorithm performance . Here, we take this idea one step further by removing the ranking step for a number of intermediate rounds. We used four different setups: the standard GA and the GA with one, two and three non-scored intermediate rounds. The results presented in Figure 4 show that our ability to identify the better conformations varies strongly depending on the number of AEP rounds used. For each GA setup, we calculated the percentage of targets for which the best model based on SC score also had the lowest energy score. Using the standard GA protocol, the best model was identified in 25% of all targets using the DFIRE energy scores. Similar results are produced with a single (AEP1) intermediate round (31%). For the runs with two intermediate rounds (AEP2) the best model was identified in 40% of all targets. However, for three intermediate rounds (AEP3) the selectivity dropped to 30%.
Allowing two intermediate, un-scored GA rounds yield the best results for model selection in this analysis. In this setup, small energy barriers can be overcome, producing some very good individual models with low energies. However, once the evolutionary pressure is too low, as seen for the AEP with three intermediate rounds, the whole population drifts away and the quality of the lowest energy model decreases.
In Figure 6 we present the coarse energy distributions for two representative remote homology targets, T0300 and T0353, and fine energy distribution for two high homology targets, T0313 and T0329. Energies for the normal GA and AEP1-3 are shown for each distribution. Generally it can be seen that the energy funnel is less well defined for lower homology modelling, T0300 and T0353, a known observation for energy-based model ranking. The advantage of the optimization process with AEP2 (green) is illustrated in T0300. In this energy plot it can be seen that more sampling of higher quality models is found for AEP2 (green). Indeed, for 80% of all cases, including T0300, T0329 and T0353, AEP2 produced the best results in the final selection. T0313 and T0329 are examples where sampling with the standard GA is sufficient. In the cases of T0300, T0329 and T0353 the problem of local minima for AEP3 (blue) can be seen, where some individual models of poorer quality have very low energies compared to all other sampled models. Sampling of the normal GA is often not as thorough as for AEP, which can be seen in T0300 and especially well in T0353, two harder modelling targets. AEP1 is sampling more space than the standard GA, but still less than AEP2 and produces inferior results.
In order to further understand the effects of AEP we applied several "normal GA" runs with adapted parameters. First standard GAs were run with the same number of rounds as given for AEP1-3; these runs produced significantly inferior results compared to the standard GA. This effect can be explained by the oversampling of local minima, which could not be prevented even using statistically derived constraints (constraining the less variable structural regions). Additionally, we increased the population size for normal GAs to 500, 1000 and 1500 models per round, which increased the computational costs but did not show any improvement in model quality for the lowest energy ranked models. In general, it seems that the AEP2 protocol gives the better balance between sampling the variable regions without drifting too far away in regions that are more structurally conserved; models that undergo consecutive multiple mutations in the structurally conserved regions are less likely to survive.
Below, we present two cases; the first where there is no improvement using AEP; the second where significant improvement could be achieved having two intermediate, un-scored, sampling rounds.
Case I: T0380
For T0380 a β-strand mainly protein with 145 residues had to be modelled. With the template search 17 different templates were identified and, after backbone completion, the SC scores ranged from 0.292 – 0.761. Only one high quality template was identified for the starting population. The best member in all final ensembles had a SC score of approximately 0.77. For none of the four recombination-setups were we able to select a model close to the best input. The worst results were created using the AEP3. All selected models ranged between a SC score of 0.609 to 0.632. Interestingly, recombination of non-repaired models enabled the selection of a final model with a score close to 0.772. After backbone completion, the energy function was not able to distinguish between a good model derived from the best template, and an inferior model, produced by the closing algorithm.
Case II: T0311
In this case an all-helical protein with the length of 88 residues has been modelled. The sequence search produced a list of 165 potential templates. The top ranking alignment produced a model with a SC score of 0.600. The best, repaired, input model for the recombination had a SC score of 0.617. Applying the standard GA selected a model of relatively poor quality with a SC score of 0.575. However, allowing two non-scored intermediate rounds improved the model beyond all initial input models and aided selecting the best member of the final ensemble, based on energy. For this final model, which has a SC score of 0.637, two helical secondary structure elements show improved positioning relative to the native structure. The native structure and the best models for the standard GA and the two intermediate rounds GA are shown superimposed in Figure 7. Clearly, the overall topology of the non-standard GA (backbone RMSD 3.13 Å) is improved relative to the standard GA model (backbone RMSD 7.81 Å), having several helical elements in the correct orientation.
General improvement along the modelling pipeline
The results for the different pre- and post-GA conditions are compared in Figure 8. Here we compare several possible modelling pipelines. The first pipeline considered, consisted of model selection without application of the GA. In this context the best final models were obtained using the FSF on minimised and repaired models. This result seems to contradict our observations on ranking correlation; however, here the emphasis is on final model selection without further refinement of the population. For this setup, an improvement of 4.1% can be seen compared to the models derived by the initial alignment.
For all pipelines using the GA with or without AEP, the unrepaired models were pre-ranked and the resulting model population was repaired before optimization. Since it was shown that un-repaired models are easier to rank, one might think that backbone completion should be the final step after recombination. To test this we compared both possibilities and it can be seen that performing backbone completion before the GA rather than after provides an improvement of 1.4% SC score; this can be explained by the additional sampling of the added secondary structure elements in the GA optimization.
The performances of the fine, the FSF, the combined and DFIRE energy score are compared for final structure selection. The best results for the energy-driven GA were obtained using DFIRE which improves the average SC score by 2.4% compared to the fine, 1.4% compared to the FSF and 0.5% compared to the combined energy score. Using DFIRE for the final model selection improved the average SC score for the normal GA from 0.590 to 0.593.
Use of AEP2 during recombination further improved the final models' quality from 0.593 to 0.598 in SC score using DFIRE for final selection. DFIRE also performs better than the combined energy score for the final selection in AEP2. This shows the importance of using a final model selection scoring function that is not used for the optimization procedure.
Overall, an improvement of 6.8% is achieved for the optimum modelling pipeline (AEP2 + DFIRE) compared to the models derived from the first alignments. Comparing these results to the SC scores of all models produced by automatic servers during CASP7 for our 75 targets, the normal GA would rank 12th and the AEP2 5th.
Consistent selection of the best SC score model would enable a further improvement of up to 4%. Clustering, as described above, was used as an alternative selection protocol to identify the most representative models, but produced inferior results; on average 4.3% lower than our optimal setup. Visual inspection of the final model ensembles indicated that the better structures are often isolated from the largest clusters.
Overall, in 77.3% of all targets, the best model of the final ensemble had a greater or equal SC score compared to the best model of the initial input population. A similar trend was observed for the lowest energy models, where 69.3% of the lowest energy model of the final population had greater or equal SC score compared to the lowest energy model of the initial population. For 18.7% of all cases the final lowest energy model was improved in SC score than the best model of the initial input population, thereby selecting or improving the best model.
Finally, possible structural errors in the selected models were investigated, using the ProSA web server  which compares them to X-ray crystal and NMR structures. As can be seen in Figure 9, all 75 models produced energy-related z-scores comparable to the scores of X-ray crystal structures. Furthermore, it has been shown for our high accuracy CASP7 submissions that we ranked 8th of all submitting groups for the accurate prediction of the χ1/χ2 angles . The move set of the GA has not been changed for this work, and therefore these findings remain valid, indicating that the conformational sampling performed in internal coordinate space does not adversely affect the side chain quality. However, as a further check of model quality, a subset of randomly selected models plus the final models selected with the best overall pipeline were also tested for stereo-chemical properties using the PROCHECK  software package. These models showed a quality comparable to the other top-ranking models submitted to CASP7.
In the present work we have performed a detailed analysis of the different steps that form the pipeline of our template based GA protein modelling approach. The results shown here, clearly demonstrate that pre-ranking should be applied to unrepaired models before backbone completion. Ranking of these incomplete models using the novel FSF scoring scheme combined with minimization was shown to provide the best ranking (13% improvement compared to weighted sequence identity). This method could be used to pre-select a final model in protocols where rapid modelling from single templates is necessary.
DFIRE was found to be the most efficient way to select the top model (0.593 SC score). The selectivity of DFIRE can be further improved by introducing Alternative Evolutionary Pressure (AEP) to the GA protocol (0.598 SC score). Creating subtle movements in the protein models using AEP, helps to select the better models by nudging them to lower energy states.
When using GAs for protein modelling two different effects can be achieved. First, GAs can be applied to improve models beyond the best input structure. Second, GAs can be used to ease the selection of the better protein models by lowering their energies. However, lowering the model's energy does not necessarily improve the structural score. In the approach used here, the GA was used to improve selection of good models. Improvement beyond or maintaining the best input model was seen for 77.3% of all targets. However, these models could only be identified in 25% for the normal GA, 31%, 40%, 30% for AEP with 1–3 intermediate rounds respectively.
The application of all the above strategies improves the final average structural score by 7.4% compared to the purely alignment-based pipeline. This was achieved by a carefully balance between the number of sampled intermediate structures (AEP2), the scoring functions used in the GA and the final selection of models with DFIRE. Overall, this pipeline would rank 5th, comparing these results to the scores of all models produced by automatic servers during CASP7 for our 75 targets
Further investigations and development need to be undertaken in order to make full use of the GA/AEP conformational search engine. Other techniques will undoubtedly be required to further assist conformational search engines, such as GAs, to recover from local minima. In general more work is required to refine scoring schemes so that the best models can be consistently selected from the ensembles of structures.
Kryshtafovych A, Fidelis K, Moult J: Progress from CASP6 to CASP7. Proteins 2007, 69 Suppl 8: 194–207.
Kryshtafovych A, Venclovas C, Fidelis K, Moult J: Progress over the first decade of CASP experiments. Proteins 2005/09/28 edition. 2005, 61 Suppl 7: 225–236.
Giorgetti A, Raimondo D, Miele AE, Tramontano A: Evaluating the usefulness of protein structure models for molecular replacement. Bioinformatics 2005/10/06 edition. 2005, 21 Suppl 2: ii72–6.
Schwarzenbacher R, Godzik A, Grzechnik SK, Jaroszewski L: The importance of alignment accuracy for molecular replacement. Acta Crystallogr D Biol Crystallogr 2004/06/24 edition. 2004, 60(Pt 7):1229–1236.
Delarue M: Molecular Replacement techniques in the context of structural genomics. In Practical Approaches Series. Edited by: Sanderson MR, Skelly J. Eds. Oxford University Press.; 2005.
Skolnick J, Fetrow JS, Kolinski A: Structural genomics and its importance for gene function analysis. Nat Biotechnol 2000/03/04 edition. 2000, 18(3):283–287.
Lengauer T, Lemmen C, Rarey M, Zimmermann M: Novel technologies for virtual screening. Drug Discov Today 2004/02/06 edition. 2004, 9(1):27–34.
Battey JN, Kopp J, Bordoli L, Read RJ, Clarke ND, Schwede T: Automated server predictions in CASP7. Proteins 2007/09/27 edition. 2007, 69 (Suppl 8):68–82.
Kopp J, Bordoli L, Battey JN, Kiefer F, Schwede T: Assessment of CASP7 predictions for template-based modeling targets. Proteins 2007/09/27 edition. 2007, 69 (Suppl 8):38–56.
Pearson WR: Searching protein sequence libraries: comparison of the sensitivity and selectivity of the Smith-Waterman and FASTA algorithms. Genomics 1991/11/01 edition. 1991, 11(3):635–650.
Altschul SF, Gish W, Miller W, Myers EW, Lipman DJ: Basic local alignment search tool. J Mol Biol 1990/10/05 edition. 1990, 215(3):403–410.
Altschul SF, Madden TL, Schaffer AA, Zhang J, Zhang Z, Miller W, Lipman DJ: Gapped BLAST and PSI-BLAST: a new generation of protein database search programs. Nucleic Acids Res 1997/09/01 edition. 1997, 25(17):3389–3402.
Eddy SR: Profile hidden Markov models. Bioinformatics 1999/01/27 edition. 1998, 14(9):755–763.
Sadreyev R, Grishin N: COMPASS: a tool for comparison of multiple protein alignments with assessment of statistical significance. J Mol Biol 2003/01/28 edition. 2003, 326(1):317–336.
Soding J, Biegert A, Lupas AN: The HHpred interactive server for protein homology detection and structure prediction. Nucleic Acids Res 2005/06/28 edition. 2005, 33(Web Server issue):W244–8.
Yona G, Levitt M: Within the twilight zone: a sensitive profile-profile comparison tool based on information theory. J Mol Biol 2002/02/06 edition. 2002, 315(5):1257–1275.
Jaroszewski L, Rychlewski L, Li Z, Li W, Godzik A: FFAS03: a server for profile--profile sequence alignments. Nucleic Acids Res 2005/06/28 edition. 2005, 33(Web Server issue):W284–8.
Moult J, Fidelis K, Kryshtafovych A, Rost B, Hubbard T, Tramontano A: Critical assessment of methods of protein structure prediction-Round VII. Proteins 2007/10/09 edition. 2007, 69 Suppl 8: 3–9.
Read RJ, Chavali G: Assessment of CASP7 predictions in the high accuracy template-based modeling category. Proteins 2007/09/27 edition. 2007, 69 Suppl 8: 27–37.
Sali A, Blundell TL: Comparative protein modelling by satisfaction of spatial restraints. J Mol Biol 1993/12/05 edition. 1993, 234(3):779–815.
Contreras-Moreira B, Fitzjohn PW, Bates PA: In silico protein recombination: enhancing template and sequence alignment selection for comparative protein modelling. J Mol Biol 2003/04/23 edition. 2003, 328(3):593–608.
Offman MN, Fitzjohn PW, Bates PA: Developing a move-set for protein model refinement. Bioinformatics 2006/05/18 edition. 2006, 22(15):1838–1845.
Petersen K, Taylor WR: Modelling zinc-binding proteins with GADGET: genetic algorithm and distance geometry for exploring topology. J Mol Biol 2003/01/16 edition. 2003, 325(5):1039–1059.
Rabow AA, Scheraga HA: Improved genetic algorithm for the protein folding problem by use of a Cartesian combination operator. Protein Sci 1996/09/01 edition. 1996, 5(9):1800–1815.
Fang Q, Shortle D: Protein refolding in silico with atom-based statistical potentials and conformational search using a simple genetic algorithm. J Mol Biol 2006/05/09 edition. 2006, 359(5):1456–1467.
Das R, Qian B, Raman S, Vernon R, Thompson J, Bradley P, Khare S, Tyka MD, Bhat D, Chivian D, Kim DE, Sheffler WH, Malmstrom L, Wollacott AM, Wang C, Andre I, Baker D: Structure prediction for CASP7 targets using extensive all-atom refinement with Rosetta@home. Proteins 2007/09/27 edition. 2007, 69 Suppl 8: 118–128.
Lee MR, Tsai J, Baker D, Kollman PA: Molecular dynamics in the endgame of protein structure prediction. J Mol Biol 2002/01/22 edition. 2001, 313(2):417–430.
Qian B, Ortiz AR, Baker D: Improvement of comparative model accuracy by free-energy optimization along principal components of natural structural variation. Proc Natl Acad Sci U S A 2004/10/20 edition. 2004, 101(43):15346–15351.
Zhang Y: I-TASSER server for protein 3D structure prediction. BMC Bioinformatics 2008/01/25 edition. 2008, 9(1):40.
Zhou H, Pandit SB, Lee SY, Borreguero J, Chen H, Wroblewska L, Skolnick J: Analysis of TASSER-based CASP7 protein structure prediction results. Proteins 2007/08/21 edition. 2007, 69 Suppl 8: 90–97.
Terashi G, Takeda-Shitaka M, Kanou K, Iwadate M, Takaya D, Hosoi A, Ohta K, Umeyama H: Fams-ace: a combined method to select the best model after remodeling all server models. Proteins 2007/09/27 edition. 2007, 69 Suppl 8: 98–107.
Joo K, Lee J, Lee S, Seo JH, Lee SJ: High accuracy template based modeling by global optimization. Proteins 2007/09/27 edition. 2007, 69 Suppl 8: 83–89.
Kolinski A, Bujnicki JM: Generalized protein structure prediction based on combination of fold-recognition with de novo folding and evaluation of models. Proteins 2005/09/28 edition. 2005, 61 Suppl 7: 84–90.
Fischer D: Servers for protein structure prediction. Curr Opin Struct Biol 2006/03/21 edition. 2006, 16(2):178–182.
Cozzetto D, Kryshtafovych A, Ceriani M, Tramontano A: Assessment of predictions in the model quality assessment category. Proteins 2007/08/08 edition. 2007, 69 Suppl 8: 175–183.
Eisenberg D, McLachlan AD: Solvation energy in protein folding and binding. Nature 1986/01/16 edition. 1986, 319(6050):199–203.
Eisenberg D, Luthy R, Bowie JU: VERIFY3D: assessment of protein models with three-dimensional profiles. Methods Enzymol 1997/01/01 edition. 1997, 277: 396–404.
McGuffin LJ: Benchmarking consensus model quality assessment for protein fold recognition. BMC Bioinformatics 2007/09/20 edition. 2007, 8: 345.
Pettitt CS, McGuffin LJ, Jones DT: Improving sequence-based fold recognition by using 3D model quality assessment. Bioinformatics 2005/06/16 edition. 2005, 21(17):3509–3515.
Sippl MJ: Recognition of errors in three-dimensional structures of proteins. Proteins 1993/12/01 edition. 1993, 17(4):355–362.
Tosatto SC: The victor/FRST function for model quality estimation. J Comput Biol 2005/12/29 edition. 2005, 12(10):1316–1327.
Wallner B, Elofsson A: Can correct protein models be identified? Protein Sci 2003/04/30 edition. 2003, 12(5):1073–1086.
Trapane TL, Lattman EE: Seventh Meeting on the Critical Assessment of Techniques for Protein Structure Prediction. Proteins 2007, 69 Suppl 8: 1–2.
CASP: Protein Structure Prediction Center webpage.[http://www.predictioncenter.org/]
Jones DT: Protein secondary structure prediction based on position-specific scoring matrices. J Mol Biol 1999/09/24 edition. 1999, 292(2):195–202.
Ramachandran GN, Sasisekharan V: Conformation of polypeptides and proteins. Adv Protein Chem 1968/01/01 edition. 1968, 23: 283–438.
Canutescu AA, Dunbrack RL Jr.: Cyclic coordinate descent: A robotics algorithm for protein loop closure. Protein Sci 2003/04/30 edition. 2003, 12(5):963–972.
Zhang Y, Skolnick J: Scoring function for automated assessment of protein structure template quality. Proteins 2004/10/12 edition. 2004, 57(4):702–710.
Zemla A, Venclovas C, Moult J, Fidelis K: Processing and analysis of CASP3 protein structure predictions. Proteins 1999/10/20 edition. 1999, Suppl 3: 22–29.
Siew N, Elofsson A, Rychlewski L, Fischer D: MaxSub: an automated measure for the assessment of protein structure prediction quality. Bioinformatics 2000/12/08 edition. 2000, 16(9):776–785.
Mead R, Nelder JA: A simplex method for function minimization. Comp J 1965, 7: 308–313.
Frishman D, Argos P: Knowledge-based protein secondary structure assignment. Proteins 1995/12/01 edition. 1995, 23(4):566–579.
Canutescu AA, Shelenkov AA, Dunbrack RL Jr.: A graph-theory algorithm for rapid protein side-chain prediction. Protein Sci 2003/08/22 edition. 2003, 12(9):2001–2014.
Lazaridis T, Karplus M: Effective energy functions for protein structure prediction. Curr Opin Struct Biol 2000/04/08 edition. 2000, 10(2):139–145.
Cavallo L, Kleinjung J, Fraternali F: POPS: A fast algorithm for solvent accessible surface areas at atomic and residue level. Nucleic Acids Res 2003/06/26 edition. 2003, 31(13):3364–3366.
Zhang C, Liu S, Zhou Y: Accurate and efficient loop selections by the DFIRE-based all-atom statistical potential. Protein Sci 2004/01/24 edition. 2004, 13(2):391–399.
Brooks B, Bruccoleri R, Olafson B, States D, Swaminathan S, Karplus M: CHARMM: A program for macromolecular energy, minimization, and dynamics calculation. J Comp Chem 1983, 4: 187–217.
Offman MN: 3D Jigsaw 3.0 Modelling Server powered by POPULUS.[http://bmm.cancerresearchuk.org/~populus]
Eswar N, Webb B, Marti-Renom MA, Madhusudhan MS, Eramian D, Shen MY, Pieper U, Sali A: Comparative protein structure modeling using MODELLER. Curr Protoc Protein Sci 2008/04/23 edition. 2007, Chapter 2: Unit 2 9.
Forrest S: Genetic algorithms: principles of natural selection applied to computation. Science 1993/08/13 edition. 1993, 261(5123):872–878.
Qian B, Raman S, Das R, Bradley P, McCoy AJ, Read RJ, Baker D: High-resolution structure prediction and the crystallographic phase problem. Nature 2007/10/16 edition. 2007, 450(7167):259–264.
Liekens AML, ten Eikelder HMM, Hilbers PAJ: Finite population models of dynamic optimization with stochastically alternating fitness functions. In IEEE Conf Evolutionary Computation 2003, 2: 838–845.
Wiederstein M, Sippl MJ: ProSA-web: interactive web service for the recognition of errors in three-dimensional structures of proteins. Nucleic Acids Res 2007/05/23 edition. 2007, 35(Web Server issue):W407–10.
Laskowski LR: PROCHECK: a program to check the stereochemical quality of protein structures. J Appl Cryst 1993, 26: 283–291.
We thank all the members of the Biomolecular Modelling Laboratory for many useful discussions and insights. This work was funded by Cancer Research UK, the Barbara Mary Hill Memorial Fund and a B'nai B'rith Leo Back Lodge Scholarship awarded to MNO.
The authors declare that they have no competing interests.
MNO and PAB devised the work. MNO carried out all computational work and wrote the initial manuscript draft. ALT and PAB edited the paper. All authors read and approved the final manuscript.
Electronic supplementary material
Authors’ original submitted files for images
Below are the links to the authors’ original submitted files for images.
About this article
Cite this article
Offman, M.N., Tournier, A.L. & Bates, P.A. Alternating evolutionary pressure in a genetic algorithm facilitates protein model selection. BMC Struct Biol 8, 34 (2008). https://doi.org/10.1186/1472-6807-8-34
- Genetic Algorithm
- Model Ensemble
- Genetic Algorithm Optimization
- Energy Score
- Standard Genetic Algorithm