General Workflow

Part of the weekly release procedure by the PDB is to publish the sequences of the entries to be released the following Wednesday four days earlier. This pre-release is scheduled every Saturday at 3:00 UTC. CAMEO collects the pre-release and, after some pre-processing of the sequences and filtering steps described below, submits a selected set of sequences (targets) to the registered servers. Participants have until the following Wednesday at 02:00 UTC to return their predictions. Once the reference structures have been released by the PDB the following Wednesday, the evaluation is performed.

The categories currently supported by CAMEO are the protein structure modeling (3D), protein model quality assessment (QE) and protein contact prediction (CP). The upcoming complete modeling (CM) is an extension of the 3D category and supports evaluation of heteromers. Other categories might follow as requested by the community and subject to available funding. The ligand binding site prediction (LB) category has been discontinued in April 2016.

CAMEO servers can be registered as public server with its full name or an anonymous server, where all scoring is performed and visible to other method developers, but not to the public. Here, only the name is anonymized ('serverx'). See our complete list of registered servers.

A CAMEO target is a pre-released PDB entry, which is submitted to all registered servers. In CAMEO CM, one or more protein sequence(s) belonging to the same pre-released PDB entry resemble a target. A target can, thus, be a monomer, a homo-oligomer or a hetero-oligomer.

Pre-processing

After downloading the pre-released sequences from the PDB on Saturday, in order to submit a limited number of high-quality targets for modeling, CAMEO CM performs the following actions before submitting the sequences to the participants:

1. Canonicalization of the sequences
2. Filtering of the sequences
3. Clustering of similar targets
4. Filtering of targets that are too easy

1. Canonicalization of the sequences

CAMEO Complete Modeling only submits cleaned and filtered amino-acid sequences to the participants. Non standard and modified amino acid residues are replaced by the one letter code of the parent amino acid according to the PDB Chemical Component Dictionary.

2. Filtering of the sequences

A filtering step is performed on the canonical sequences, which removes targets if any of their sequences:

• contain nucleic acids (DNA or RNA)
• contain unknown residues after the cleanup
• contain non-canonical or modified residues that don't have a parent amino acid in the PDB Chemical Component Dictionary
• contain non-cannonical amino acids (selenocystein, etc.) after the cleanup
• contain sequence caps (such as N-terminal acetylation, C-terminal amidation) or contain residues annotated as non-linking by the PDB Chemical Component Dictionary
• are shorter than 30 amino acids
• or otherwise couldn't be fully converted to sequences of canonical amino acids.

3. Clustering of similar targets

In order to avoid "duplicate" submissions of very similar targets, CAMEO CM clusters the remaining targets according to the following method in a third step.

1. All sequences are clustered separately with CD-HIT at 99% sequence identity.
2. Targets with several sequences belonging to the same CD-HIT cluster are removed.
3. Targets are labeled by the list of CD-HIT clusters their sequences belong to.
4. Targets that have the same list of clusters constitute a "hetero cluster". Note that this is also the case for monomers and homo-oligomers, where the behaviour reverts to the separate CD-HIT cluster.
5. One "representative" target is selected per "hetero cluster" (the first PDB ID appearing in the pre-release).

4. Filtering too easy targets

Targets that are believed to be too trivial to model are not submitted to the participating servers. All representative targets (after clustering) are assessed for difficulty. All sequences are searched separately for templates with BLAST against the full list of protein sequences currently in the PDB. A target too "trivial" to model would feature a template with 85% sequence identity or more, additionally:

• at least 70% of the sequence is covered by the template
• and, for sequences longer than 250 residues, less than 40 amino acids are uncovered.

A target is referred to be "too easy" if a trivial template covers all the sequences of the target and there is a exhaustive mapping between every sequence of the target and every sequence of the template.

This implies:

• The template does not contain any sequence that would be rejected by the filtering (step 2 above), e.g. no nucleotides, short peptides or unknown residues
• The template has no additional sequence to the target

Only targets that passed filtering, constitute representatives in the clustering, and are not "too easy" are then submitted to the participating servers.

Scores

We are working hard to implement as many of the scores from the 3D category into CAMEO CM, please note that we primarily consider superposition-free scores as CAMEO targets might consist of multiple domains. So far the following scores are available:

The lDDT score (Local Distance Difference Test on All Atoms) evaluates the quality of the local atomic environment of a model. lDDT rewards the fraction of correctly predicted inter-atomic distances in a model at different threshold levels. lDDT does not depend on a global superposition of the prediction and target structure.
Specifically, interaction distances (cutoff 15 Å) between atoms in the reference protein structure are compared with distances between corresponding atoms in the predictions. If the difference between the two distances is within a defined threshold, the interaction is considered to be preserved in the prediction. The final lDDT-all score is computed by averaging the fraction of correctly modeled interactions for the following four distance difference thresholds: 0.5, 1, 2, and 4 Å (the same thresholds as GDT_HA). A filter based on the Engh and Huber bond lengths and angles removes stereochemical violations and steric clashes. CAMEO additionally offers a Cα - based lDDT score.[ref.: CASP9 TBM Assessment]