Computational protocol: EVAcon: a protein contact prediction evaluation service

Similar protocols

Protocol publication

[…] The current interface provides an easy navigation for target structures and prediction methods (): EVAcon provides a fast graphical view with directly accessible results that give an overview of the performance of the servers. This initial view is composed of static pages that are regenerated every week. These pages include results (with graphs, tables and raw files for Acc, Imp, Cov and Xd) for each individual server, a summary of the mean accuracy over the weeks of the current year and also a comparison of the accuracies of the servers for the set of all the proteins during the current week.The dynamic section provides an in-depth view of the results, suitable for method developers. A query-based system allows a flexible display of results. For example, it makes it possible to define subsets of proteins of interest common to all methods, and to visualize the comparative performances of the servers on these proteins. The results are presented as graphs, tables and raw files.EVAcon includes an additional facility for the direct evaluation of contact predictions that is intended for developers in order to evaluate the predictions of their methods according to the EVA criteria. These can be submitted in contact prediction format or as PDB files together with the corresponding experimental structure.EVAcon provides a fast graphical view with directly accessible results that give an overview of the performance of the servers. This initial view is composed of static pages that are regenerated every week. These pages include results (with graphs, tables and raw files for Acc, Imp, Cov and Xd) for each individual server, a summary of the mean accuracy over the weeks of the current year and also a comparison of the accuracies of the servers for the set of all the proteins during the current week.The dynamic section provides an in-depth view of the results, suitable for method developers. A query-based system allows a flexible display of results. For example, it makes it possible to define subsets of proteins of interest common to all methods, and to visualize the comparative performances of the servers on these proteins. The results are presented as graphs, tables and raw files.EVAcon includes an additional facility for the direct evaluation of contact predictions that is intended for developers in order to evaluate the predictions of their methods according to the EVA criteria. These can be submitted in contact prediction format or as PDB files together with the corresponding experimental structure.Another possibility of the system is to perform evaluations with a variable distance cutoff for the definition of contact which can be chosen by the user (apart from the default—8 Å between C-beta atoms). This option can be accessed also from the main page of EVAcon, through the link ‘Choose a different contact distance and see results for the evaluated targets’. This option is also available for the evaluation of predictions submited by the user [point (iii) above].It is still too soon to establish a clear ranking of the performance of the servers, due to the small number of weeks that the evaluation has been active. Despite this, the assessment yields very similar results for the contact specialist methods and the 3D structure prediction servers. For some protein targets, contact specialists are performing much better than the 3D prediction methods evaluated by EVAcon. For instance, protein 1pxe chain A (zinc binding domain of the neural zinc finger transcription factor 1), where GPCpred () has an accuracy of 50% with a coverage of contacts predicted of 27.3%, twice the accuracy and coverage of FUGUE (), the best performing threading server for this protein. Another interesting example is the case of the target 1xjh chain A (redox switch domain of the Escherichia coli Hsp33), where the best result is produced by CMAPpro_band (), with a prediction accuracy of 66.7% and a coverage of 20%, and where the best threading prediction, in this case done by WURST (), is only able to achieve an accuracy of 58.3% and a coverage of 17.5%. These results seem to confirm our initial observations (), and if confirmed in the following weeks with additional proteins and methods, it will demonstrate the utility of adding contact prediction methods to the set of tools used in fold recognition approaches. […]

Pipeline specifications

Software tools FUGUE, CMAPpro
Application Protein structure analysis