CAMEO-3D: Server Pairwise Comparison 1-week Performance (2022-04-16)

The pairwise comparison is showing the average differences of the reference with all the other servers, each on a common data subset. It is misleading to compare further servers among each other, without selecting a new reference as for each server pair the common subset may differ. Numbers in green denote scores where the reference server is better, numbers in red where another server was outperforming the reference server.

NaiveBLAST - Pairwise Comparison on common targets - dataset: 1-week [2022-04-16]

Server Name	Targets modeled		Oligo-Targets modeled		Avg. Response Time (hh:mm:ss)		Avg. lDDT		Avg. CAD-score		Avg. lDDT-BS		Avg. QS-score		Avg. MM-align
Server Name	Dif.	Com.	Dif.	Com.	Dif.	Ref	Dif.	Ref.	Dif.	Ref.	Dif.	Ref.	Dif.	Ref.	Dif.	Ref.
NaiveBLAST - Reference

Server Name	Targets modeled		Oligo-Targets modeled		Avg. Response Time (hh:mm:ss)		Avg. lDDT		Avg. CAD-score		Avg. lDDT-BS		Avg. QS-score		Avg. MM-align
Server Name	Dif.	Com.	Dif.	Com.	Dif.	Ref	Dif.	Ref.	Dif.	Ref.	Dif.	Ref.	Dif.	Ref.	Dif.	Ref.
NaiveBLAST - Reference
Naive AlphaFoldDB 100	10	1	0	0	--1:59:57	00:00:22	27.44	80.93	17.32	72.78	-	-	-	-	-	-
Naive AlphaFoldDB 90	5	5	0	0	--1:59:49	00:00:28	10.52	83.46	9.47	77.34	-	-	-	-	-	-
PRIMO	-1	10	0	0	-03:15:17	00:00:20	2.24	50.48	1.13	51.17	0.32	30.87	-	-	-	-
PRIMO_BST_3D	1	8	0	0	-03:02:54	00:00:17	1.60	42.19	-1.04	44.40	-10.72	30.87	-	-	-	-
PRIMO_BST_CL	-2	11	0	0	-03:14:04	00:00:19	0.40	47.19	-1.21	48.08	-3.21	26.45	-	-	-	-
PRIMO_HHS_CL	10	1	0	0	-04:13:37	00:00:10	-0.55	37.23	-0.14	50.14	-	-	-	-	-	-
Phyre2	-2	11	0	0	-01:08:41	00:00:21	-0.59	53.57	-4.27	53.27	-16.77	26.45	-	-	-	-
PRIMO_HHS_3D	10	2	0	0	-04:24:31	00:00:20	-1.48	60.55	-0.99	64.31	-	-	-	-	-	-
RocketX	-3	12	0	0	-41:28:03	00:00:20	-5.05	50.24	-8.56	50.62	0.35	26.45	-	-	-	-
SWISS-MODEL	-2	11	0	0	-00:05:10	00:00:20	-11.82	52.40	-12.49	52.49	-4.76	26.45	-	-	-	-
IntFOLD3-TS	-3	12	0	0	-42:20:04	00:00:20	-12.18	50.24	-13.25	50.62	-21.30	26.45	-	-	-	-
IntFOLD4-TS	-3	12	0	0	-62:17:04	00:00:20	-16.33	50.24	-16.00	50.62	-26.25	26.45	-	-	-	-
IntFOLD5-TS	-3	12	0	0	-29:40:50	00:00:20	-17.15	50.24	-16.07	50.62	-29.28	26.45	-	-	-	-
MultiDFold	-3	12	0	0	-79:49:27	00:00:20	-17.71	50.24	-17.84	50.62	-26.81	26.45	-	-	-	-
IntFOLD7	1	9	0	0	-39:57:36	00:00:18	-19.49	53.01	-17.55	53.12	-77.19	17.61	-	-	-	-
IntFOLD6-TS	-1	10	0	0	-26:06:26	00:00:18	-21.48	43.44	-19.01	45.05	-37.51	26.45	-	-	-	-
BestSingleStructuralTemplate	-2	11	0	0	-04:59:29	00:00:19	-24.46	47.19	-21.86	48.08	-27.40	26.45	-	-	-	-
RoseTTAFold	1	9	0	0	-50:59:05	00:00:16	-27.91	45.25	-26.10	46.08	-25.17	37.81	-	-	-	-
pureAF2_notemp	-3	12	0	0	-16:05:42	00:00:20	-29.56	50.24	-27.78	50.62	-26.36	26.45	-	-	-	-
ManiFold	-3	12	0	0	-58:46:43	00:00:20	-30.08	50.24	-27.51	50.62	-28.41	26.45	-	-	-	-
AIRFold	-3	12	0	0	-27:03:46	00:00:20	-30.49	50.24	-27.99	50.62	-26.16	26.45	-	-	-	-
SADA	-3	12	0	0	-78:25:28	00:00:20	-30.90	50.24	-29.04	50.62	-41.42	26.45	-	-	-	-
OpenComplex	-3	12	0	0	-48:56:03	00:00:20	-31.35	50.24	-28.69	50.62	-32.87	26.45	-	-	-	-
ZJUT-DeepAssembly	-3	12	0	0	-73:07:11	00:00:20	-31.45	50.24	-28.87	50.62	-41.07	26.45	-	-	-	-
PAthreader	-3	12	0	0	-25:01:19	00:00:20	-31.70	50.24	-28.49	50.62	-38.05	26.45	-	-	-	-
pureAF2_orig	-3	12	0	0	-14:24:28	00:00:20	-32.31	50.24	-29.35	50.62	-43.65	26.45	-	-	-	-
HeliXonAI	-3	12	0	0	-76:39:05	00:00:20	-32.91	50.24	-29.26	50.62	-48.17	26.45	-	-	-	-
MEGA-EvoGen	-3	12	0	0	-47:59:44	00:00:20	-33.32	50.24	-29.96	50.62	-25.55	26.45	-	-	-	-
PaFold	12	0	0	0	-	-	-	-	-	-	-	-	-	-	-	-
Robetta	12	0	0	0	-	-	-	-	-	-	-	-	-	-	-	-
RaptorX	12	0	0	0	-	-	-	-	-	-	-	-	-	-	-	-
SPARKS-X	12	0	0	0	-	-	-	-	-	-	-	-	-	-	-	-
ZlxFold	12	0	0	0	-	-	-	-	-	-	-	-	-	-	-	-



Naive AlphaFoldDB 100
Naive AlphaFoldDB 90
PRIMO
PRIMO_BST_3D
PRIMO_BST_CL
PRIMO_HHS_CL
Phyre2
PRIMO_HHS_3D
RocketX
SWISS-MODEL
IntFOLD3-TS
IntFOLD4-TS
IntFOLD5-TS
MultiDFold
IntFOLD7
IntFOLD6-TS
BestSingleStructuralTemplate
RoseTTAFold
pureAF2_notemp
ManiFold
AIRFold
SADA
OpenComplex
ZJUT-DeepAssembly
PAthreader
pureAF2_orig
HeliXonAI
MEGA-EvoGen
PaFold
Robetta
RaptorX
SPARKS-X
ZlxFold

NaiveBLAST - model_1 per target comparison with the best: 1-week - [2022-04-16]

Target	ref. PDB	lDDT			CAD-score			lDDT-BS			QS-score			MM-align			Difficulty
Target	ref. PDB	Diff	Ref	Best	Diff	Ref	Best	Diff	Ref	Best	Diff	Ref	Best	Diff	Ref	Best	Difficulty
2022-04-16_00000283_1	7x4e[A]	-	84.85	OpenComplex	-	79.62	ZJUT-DeepAssembly	-	-	-	-	-	-	-	-	-	Medium
2022-04-16_00000256_1	7v1k[A]	-11.36	92.29	BestSingleStructuralTemplate	-12.92	85.70	BestSingleStructuralTemplate	-	-	-	-	0.87	SWISS-MODEL	-	0.96	SWISS-MODEL	Easy
2022-04-16_00000246_1	7u2r[A]	-78.14	92.39	pureAF2_orig	-68.06	85.27	ZJUT-DeepAssembly	-79.24	96.85	HeliXonAI	-	0.45	SWISS-MODEL	-	0.83	SWISS-MODEL	Medium
2022-04-16_00000197_1	7t7y[A]	-	90.29	AIRFold	-	84.12	AIRFold	-	-	-	-	-	-	-	-	-	Medium
2022-04-16_00000167_1	7ri3[C]	-40.00	67.77	HeliXonAI	-33.98	69.56	MEGA-EvoGen	-62.69	66.42	HeliXonAI	-	-	-	-	-	-	Hard
2022-04-16_00000127_2	7pno[D]	-65.95	92.43	BestSingleStructuralTemplate	-55.97	85.92	RocketX	-	-	-	-	-	-	-	-	-	Easy
2022-04-16_00000123_1	7pc9[A]	-8.56	92.42	ZJUT-DeepAssembly	-8.98	87.45	MEGA-EvoGen	-	-	-	-	-	-	-	-	-	Easy
2022-04-16_00000121_1	7pc7[B]	-7.71	91.73	pureAF2_orig	-8.63	87.23	pureAF2_orig	-	-	-	-	-	-	-	-	-	Easy
2022-04-16_00000119_1	7pc4[A]	-8.99	94.26	pureAF2_orig	-9.36	88.31	ZJUT-DeepAssembly	-	-	-	-	-	-	-	-	-	Easy
2022-04-16_00000118_1	7pc3[A]	-10.61	93.83	MultiDFold	-10.82	88.72	MEGA-EvoGen	-	-	-	-	-	-	-	-	-	Easy
2022-04-16_00000112_1	7osw[A]	-15.47	52.70	BestSingleStructuralTemplate	-6.29	56.43	MultiDFold	-	-	-	-	-	-	-	-	-	Hard
2022-04-16_00000104_1	7ocn[A]	-73.41	87.62	HeliXonAI	-67.61	82.84	HeliXonAI	-4.28	62.28	AIRFold	-	0.00	SWISS-MODEL	-	0.25	SWISS-MODEL	Medium
2022-04-16_00000082_2	7n3t[C]	-73.85	87.52	OpenComplex	-60.78	82.22	OpenComplex	-	-	-	-	-	-	-	-	-	Medium
2022-04-16_00000079_1	7mhw[A]	-	91.70	MEGA-EvoGen	-	86.51	MEGA-EvoGen	-	-	-	-	-	-	-	-	-	Easy
2022-04-16_00000050_1	7f0o[B]	-32.47	84.46	HeliXonAI	-29.85	80.98	HeliXonAI	-	-	-	-	0.80	SWISS-MODEL	-	0.79	SWISS-MODEL	Medium



2022-04-16_00000283_1
2022-04-16_00000256_1
2022-04-16_00000246_1
2022-04-16_00000197_1
2022-04-16_00000167_1
2022-04-16_00000127_2
2022-04-16_00000123_1
2022-04-16_00000121_1
2022-04-16_00000119_1
2022-04-16_00000118_1
2022-04-16_00000112_1
2022-04-16_00000104_1
2022-04-16_00000082_2
2022-04-16_00000079_1
2022-04-16_00000050_1