Results for Linux

system	cpu	gpu	memory	os_version	ollama_version	model_name	throughput
Linux	12th Gen Intel(R) Core(TM) i5-12400	Intel Corporation	31.08	Ubuntu 22.04.4 LTS	0.1.30	mistral:7b	9.25
Linux	12th Gen Intel(R) Core(TM) i5-12400	Intel Corporation	31.08	Ubuntu 22.04.4 LTS	0.1.30	gemma:2b	20.25
Linux	12th Gen Intel(R) Core(TM) i5-12400	Intel Corporation	31.08	Ubuntu 22.04.4 LTS	0.1.30	gemma:7b	7.2
Linux	12th Gen Intel(R) Core(TM) i5-12400	Intel Corporation	31.08	Ubuntu 22.04.4 LTS	0.1.30	llama2:7b	9.59
Linux	12th Gen Intel(R) Core(TM) i5-12400	Intel Corporation	31.08	Ubuntu 22.04.4 LTS	0.1.30	llama2:13b	5.14
Linux	12th Gen Intel(R) Core(TM) i5-12400	Intel Corporation	31.08	Ubuntu 22.04.4 LTS	0.1.30	llava:7b	9.61
Linux	12th Gen Intel(R) Core(TM) i5-12400	Intel Corporation	31.08	Ubuntu 22.04.4 LTS	0.1.30	llava:13b	5.26
Linux	Intel(R) Xeon(R) CPU E5-2680 v2 @ 2.80GHz	NVIDIA GeForce RTX 3080 vram:10GB (2 cards)	230.60	Ubuntu 22.04.3 LTS	0.1.30	mistral:7b	92.85
Linux	Intel(R) Xeon(R) CPU E5-2680 v2 @ 2.80GHz	NVIDIA GeForce RTX 3080 vram:10GB (2 cards)	230.60	Ubuntu 22.04.3 LTS	0.1.30	gemma:2b	118.62
Linux	Intel(R) Xeon(R) CPU E5-2680 v2 @ 2.80GHz	NVIDIA GeForce RTX 3080 vram:10GB (2 cards)	230.60	Ubuntu 22.04.3 LTS	0.1.30	gemma:7b	73.84
Linux	Intel(R) Xeon(R) CPU E5-2680 v2 @ 2.80GHz	NVIDIA GeForce RTX 3080 vram:10GB (2 cards)	230.60	Ubuntu 22.04.3 LTS	0.1.30	llama2:7b	96.08
Linux	Intel(R) Xeon(R) CPU E5-2680 v2 @ 2.80GHz	NVIDIA GeForce RTX 3080 vram:10GB (2 cards)	230.60	Ubuntu 22.04.3 LTS	0.1.30	llama2:13b	62.01
Linux	Intel(R) Xeon(R) CPU E5-2680 v2 @ 2.80GHz	NVIDIA GeForce RTX 3080 vram:10GB (2 cards)	230.60	Ubuntu 22.04.3 LTS	0.1.30	llava:7b	93.11
Linux	Intel(R) Xeon(R) CPU E5-2680 v2 @ 2.80GHz	NVIDIA GeForce RTX 3080 vram:10GB (2 cards)	230.60	Ubuntu 22.04.3 LTS	0.1.30	llava:13b	63.61
Linux	Intel(R) Xeon(R) Platinum 8352V CPU @ 2.10GHz	NVIDIA GeForce RTX 4090 vram:24GB	503.53	Ubuntu 22.04.3 LTS	0.1.30	mistral:7b	127.26
Linux	Intel(R) Xeon(R) Platinum 8352V CPU @ 2.10GHz	NVIDIA GeForce RTX 4090 vram:24GB	503.53	Ubuntu 22.04.3 LTS	0.1.30	gemma:2b	159.48
Linux	Intel(R) Xeon(R) Platinum 8352V CPU @ 2.10GHz	NVIDIA GeForce RTX 4090 vram:24GB	503.53	Ubuntu 22.04.3 LTS	0.1.30	gemma:7b	93.57
Linux	Intel(R) Xeon(R) Platinum 8352V CPU @ 2.10GHz	NVIDIA GeForce RTX 4090 vram:24GB	503.53	Ubuntu 22.04.3 LTS	0.1.30	llama2:7b	130.82
Linux	Intel(R) Xeon(R) Platinum 8352V CPU @ 2.10GHz	NVIDIA GeForce RTX 4090 vram:24GB	503.53	Ubuntu 22.04.3 LTS	0.1.30	llama2:13b	84.14
Linux	Intel(R) Xeon(R) Platinum 8352V CPU @ 2.10GHz	NVIDIA GeForce RTX 4090 vram:24GB	503.53	Ubuntu 22.04.3 LTS	0.1.30	llava:7b	126.22
Linux	Intel(R) Xeon(R) Platinum 8352V CPU @ 2.10GHz	NVIDIA GeForce RTX 4090 vram:24GB	503.53	Ubuntu 22.04.3 LTS	0.1.30	llava:13b	84.44
Linux	AMD Eng Sample: 100-000000163-01_43/29_Y	Advanced Micro Devices, Inc. [AMD/ATI]	125.65	Linux Mint 21.3	0.1.30	mistral:7b	99.25
Linux	AMD Eng Sample: 100-000000163-01_43/29_Y	Advanced Micro Devices, Inc. [AMD/ATI]	125.65	Linux Mint 21.3	0.1.30	gemma:2b	188.77
Linux	AMD Eng Sample: 100-000000163-01_43/29_Y	Advanced Micro Devices, Inc. [AMD/ATI]	125.65	Linux Mint 21.3	0.1.30	gemma:7b	88.31
Linux	AMD Eng Sample: 100-000000163-01_43/29_Y	Advanced Micro Devices, Inc. [AMD/ATI]	125.65	Linux Mint 21.3	0.1.30	llama2:7b	100.11
Linux	AMD Eng Sample: 100-000000163-01_43/29_Y	Advanced Micro Devices, Inc. [AMD/ATI]	125.65	Linux Mint 21.3	0.1.30	llama2:13b	65.68
Linux	AMD Eng Sample: 100-000000163-01_43/29_Y	Advanced Micro Devices, Inc. [AMD/ATI]	125.65	Linux Mint 21.3	0.1.30	llava:7b	101.21
Linux	AMD Eng Sample: 100-000000163-01_43/29_Y	Advanced Micro Devices, Inc. [AMD/ATI]	125.65	Linux Mint 21.3	0.1.30	llava:13b	66.64
Linux	Intel(R) Core(TM) i7-14700K	NVIDIA GeForce RTX 4070 SUPER	31.10	Ubuntu 22.04.4 LTS	0.1.30	mistral:7b	86.3
Linux	Intel(R) Core(TM) i7-14700K	NVIDIA GeForce RTX 4070 SUPER	31.10	Ubuntu 22.04.4 LTS	0.1.30	gemma:2b	166.17
Linux	Intel(R) Core(TM) i7-14700K	NVIDIA GeForce RTX 4070 SUPER	31.10	Ubuntu 22.04.4 LTS	0.1.30	gemma:7b	70.23
Linux	Intel(R) Core(TM) i7-14700K	NVIDIA GeForce RTX 4070 SUPER	31.10	Ubuntu 22.04.4 LTS	0.1.30	llama2:7b	89.99
Linux	Intel(R) Core(TM) i7-14700K	NVIDIA GeForce RTX 4070 SUPER	31.10	Ubuntu 22.04.4 LTS	0.1.30	llama2:13b	51.45
Linux	Intel(R) Core(TM) i7-14700K	NVIDIA GeForce RTX 4070 SUPER	31.10	Ubuntu 22.04.4 LTS	0.1.30	llava:7b	91.53
Linux	Intel(R) Core(TM) i7-14700K	NVIDIA GeForce RTX 4070 SUPER	31.10	Ubuntu 22.04.4 LTS	0.1.30	llava:13b	52.85
Linux	Intel(R) Core(TM) i7-14700K	NVIDIA GeForce RTX 4070 SUPER	31.10	Ubuntu 22.04.4 LTS	0.1.30	mistral:7b	88.87
Linux	Intel(R) Core(TM) i7-14700K	NVIDIA GeForce RTX 4070 SUPER	31.10	Ubuntu 22.04.4 LTS	0.1.30	gemma:2b	171.67
Linux	Intel(R) Core(TM) i7-14700K	NVIDIA GeForce RTX 4070 SUPER	31.10	Ubuntu 22.04.4 LTS	0.1.30	gemma:7b	72.38
Linux	Intel(R) Core(TM) i7-14700K	NVIDIA GeForce RTX 4070 SUPER	31.10	Ubuntu 22.04.4 LTS	0.1.30	llama2:7b	91.68
Linux	Intel(R) Core(TM) i7-14700K	NVIDIA GeForce RTX 4070 SUPER	31.10	Ubuntu 22.04.4 LTS	0.1.30	llama2:13b	53.35
Linux	Intel(R) Core(TM) i7-14700K	NVIDIA GeForce RTX 4070 SUPER	31.10	Ubuntu 22.04.4 LTS	0.1.30	llava:7b	91.33
Linux	Intel(R) Core(TM) i7-14700K	NVIDIA GeForce RTX 4070 SUPER	31.10	Ubuntu 22.04.4 LTS	0.1.30	llava:13b	54.41
Linux	Intel(R) Core(TM) i7-14700K	NVIDIA GeForce RTX 4070 SUPER	31.10	Ubuntu 22.04.4 LTS	0.1.31	mistral:7b	88.88
Linux	Intel(R) Core(TM) i7-14700K	NVIDIA GeForce RTX 4070 SUPER	31.10	Ubuntu 22.04.4 LTS	0.1.31	gemma:2b	172.87
Linux	Intel(R) Core(TM) i7-14700K	NVIDIA GeForce RTX 4070 SUPER	31.10	Ubuntu 22.04.4 LTS	0.1.31	gemma:7b	72.22
Linux	Intel(R) Core(TM) i7-14700K	NVIDIA GeForce RTX 4070 SUPER	31.10	Ubuntu 22.04.4 LTS	0.1.31	llama2:7b	92.08
Linux	Intel(R) Core(TM) i7-14700K	NVIDIA GeForce RTX 4070 SUPER	31.10	Ubuntu 22.04.4 LTS	0.1.31	llama2:13b	53.16
Linux	Intel(R) Core(TM) i7-14700K	NVIDIA GeForce RTX 4070 SUPER	31.10	Ubuntu 22.04.4 LTS	0.1.31	llava:7b	91.88
Linux	Intel(R) Core(TM) i7-14700K	NVIDIA GeForce RTX 4070 SUPER	31.10	Ubuntu 22.04.4 LTS	0.1.31	llava:13b	54.27
Linux	Intel(R) Xeon(R) Gold 6152 CPU @ 2.10GHz	Tesla V100-PCIE-16GB	1509.66	openEuler release 22.03 (LTS-SP2)	0.1.30	mistral:7b	85.16
Linux	Intel(R) Xeon(R) Gold 6152 CPU @ 2.10GHz	Tesla V100-PCIE-16GB	1509.66	openEuler release 22.03 (LTS-SP2)	0.1.30	gemma:2b	122.62
Linux	Intel(R) Xeon(R) Gold 6152 CPU @ 2.10GHz	Tesla V100-PCIE-16GB	1509.66	openEuler release 22.03 (LTS-SP2)	0.1.30	gemma:7b	73.76
Linux	Intel(R) Xeon(R) Gold 6152 CPU @ 2.10GHz	Tesla V100-PCIE-16GB	1509.66	openEuler release 22.03 (LTS-SP2)	0.1.30	llama2:7b	88.94
Linux	Intel(R) Xeon(R) Gold 6152 CPU @ 2.10GHz	Tesla V100-PCIE-16GB	1509.66	openEuler release 22.03 (LTS-SP2)	0.1.30	llama2:13b	60.21
Linux	Intel(R) Xeon(R) Gold 6152 CPU @ 2.10GHz	Tesla V100-PCIE-16GB	1509.66	openEuler release 22.03 (LTS-SP2)	0.1.30	llava:7b	91.51
Linux	Intel(R) Xeon(R) Gold 6152 CPU @ 2.10GHz	Tesla V100-PCIE-16GB	1509.66	openEuler release 22.03 (LTS-SP2)	0.1.30	llava:13b	61.91
Linux	Intel(R) Xeon(R) Gold 6152 CPU @ 2.10GHz	Tesla V100-PCIE-16GB	1509.66	openEuler release 22.03 (LTS-SP2)	0.1.30	mistral:7b	3.61
Linux	Intel(R) Xeon(R) Gold 6152 CPU @ 2.10GHz	Tesla V100-PCIE-16GB	1509.66	openEuler release 22.03 (LTS-SP2)	0.1.30	gemma:2b	10.62
Linux	Intel(R) Xeon(R) Gold 6152 CPU @ 2.10GHz	Tesla V100-PCIE-16GB	1509.66	openEuler release 22.03 (LTS-SP2)	0.1.30	gemma:7b	3.01
Linux	Intel(R) Xeon(R) Gold 6152 CPU @ 2.10GHz	Tesla V100-PCIE-16GB	1509.66	openEuler release 22.03 (LTS-SP2)	0.1.30	llama2:7b	3.75
Linux	Intel(R) Xeon(R) Gold 6152 CPU @ 2.10GHz	Tesla V100-PCIE-16GB	1509.66	openEuler release 22.03 (LTS-SP2)	0.1.30	llama2:13b	2.06
Linux	Intel(R) Xeon(R) Gold 6152 CPU @ 2.10GHz	Tesla V100-PCIE-16GB	1509.66	openEuler release 22.03 (LTS-SP2)	0.1.30	llava:7b	3.62
Linux	Intel(R) Xeon(R) Gold 6152 CPU @ 2.10GHz	Tesla V100-PCIE-16GB	1509.66	openEuler release 22.03 (LTS-SP2)	0.1.30	llava:13b	2.1
Linux	AMD Ryzen 9 5950X 16-Core Processor	NVIDIA GeForce RTX 3090	31.26	Pop!_OS 22.04 LTS	0.1.31	mistral:7b	119.91
Linux	AMD Ryzen 9 5950X 16-Core Processor	NVIDIA GeForce RTX 3090	31.26	Pop!_OS 22.04 LTS	0.1.31	gemma:2b	190.72
Linux	AMD Ryzen 9 5950X 16-Core Processor	NVIDIA GeForce RTX 3090	31.26	Pop!_OS 22.04 LTS	0.1.31	gemma:7b	99.49
Linux	AMD Ryzen 9 5950X 16-Core Processor	NVIDIA GeForce RTX 3090	31.26	Pop!_OS 22.04 LTS	0.1.31	llama2:7b	125.53
Linux	AMD Ryzen 9 5950X 16-Core Processor	NVIDIA GeForce RTX 3090	31.26	Pop!_OS 22.04 LTS	0.1.31	llama2:13b	78.05
Linux	AMD Ryzen 9 5950X 16-Core Processor	NVIDIA GeForce RTX 3090	31.26	Pop!_OS 22.04 LTS	0.1.31	llava:7b	124.58
Linux	AMD Ryzen 9 5950X 16-Core Processor	NVIDIA GeForce RTX 3090	31.26	Pop!_OS 22.04 LTS	0.1.31	llava:13b	79.61
Linux	Intel Corporation	Integrated Matrox G200eW3 Graphics Controller	1007.39	Ubuntu 22.04.4 LTS	0.1.30	mistral:7b	18.89
Linux	Intel Corporation	Integrated Matrox G200eW3 Graphics Controller	1007.39	Ubuntu 22.04.4 LTS	0.1.30	gemma:2b	42.47
Linux	Intel Corporation	Integrated Matrox G200eW3 Graphics Controller	1007.39	Ubuntu 22.04.4 LTS	0.1.30	gemma:7b	19.39
Linux	Intel Corporation	Integrated Matrox G200eW3 Graphics Controller	1007.39	Ubuntu 22.04.4 LTS	0.1.30	llama2:7b	24.36
Linux	Intel Corporation	Integrated Matrox G200eW3 Graphics Controller	1007.39	Ubuntu 22.04.4 LTS	0.1.30	llama2:13b	11.24
Linux	Intel Corporation	Integrated Matrox G200eW3 Graphics Controller	1007.39	Ubuntu 22.04.4 LTS	0.1.30	llava:7b	16.45
Linux	Intel Corporation	Integrated Matrox G200eW3 Graphics Controller	1007.39	Ubuntu 22.04.4 LTS	0.1.30	llava:13b	13.47
Linux	Intel Corporation	Integrated Matrox G200eW3 Graphics Controller	1007.38	Ubuntu 22.04.4 LTS	0.1.30	mistral:7b	7.18
Linux	Intel Corporation	Integrated Matrox G200eW3 Graphics Controller	1007.38	Ubuntu 22.04.4 LTS	0.1.30	gemma:2b	11.06
Linux	Intel Corporation	Integrated Matrox G200eW3 Graphics Controller	1007.38	Ubuntu 22.04.4 LTS	0.1.30	gemma:7b	9.75
Linux	Intel Corporation	Integrated Matrox G200eW3 Graphics Controller	1007.38	Ubuntu 22.04.4 LTS	0.1.30	llama2:7b	9.94
Linux	Intel Corporation	Integrated Matrox G200eW3 Graphics Controller	1007.38	Ubuntu 22.04.4 LTS	0.1.30	llama2:13b	7.69
Linux	Intel Corporation	Integrated Matrox G200eW3 Graphics Controller	1007.38	Ubuntu 22.04.4 LTS	0.1.30	llava:7b	12.88
Linux	Intel Corporation	Integrated Matrox G200eW3 Graphics Controller	1007.38	Ubuntu 22.04.4 LTS	0.1.30	llava:13b	7.41
Linux	Intel Corporation	Integrated Matrox G200eW3 Graphics Controller	1007.39	Ubuntu 22.04.4 LTS	0.1.31	mistral:7b	17.85
Linux	Intel Corporation	Integrated Matrox G200eW3 Graphics Controller	1007.39	Ubuntu 22.04.4 LTS	0.1.31	gemma:2b	49.3
Linux	Intel Corporation	Integrated Matrox G200eW3 Graphics Controller	1007.39	Ubuntu 22.04.4 LTS	0.1.31	gemma:7b	18.89
Linux	Intel Corporation	Integrated Matrox G200eW3 Graphics Controller	1007.39	Ubuntu 22.04.4 LTS	0.1.31	llama2:7b	22.06
Linux	Intel Corporation	Integrated Matrox G200eW3 Graphics Controller	1007.39	Ubuntu 22.04.4 LTS	0.1.31	llama2:13b	11.57
Linux	Intel Corporation	Integrated Matrox G200eW3 Graphics Controller	1007.39	Ubuntu 22.04.4 LTS	0.1.31	llava:7b	17.62
Linux	Intel Corporation	Integrated Matrox G200eW3 Graphics Controller	1007.39	Ubuntu 22.04.4 LTS	0.1.31	llava:13b	9.67
Linux	Intel Corporation	Integrated Matrox G200eW3 Graphics Controller	1007.38	Ubuntu 22.04.4 LTS	0.1.31	mistral:7b	5.18
Linux	Intel Corporation	Integrated Matrox G200eW3 Graphics Controller	1007.38	Ubuntu 22.04.4 LTS	0.1.31	gemma:2b	6.55
Linux	Intel Corporation	Integrated Matrox G200eW3 Graphics Controller	1007.38	Ubuntu 22.04.4 LTS	0.1.31	gemma:7b	5.43
Linux	Intel Corporation	Integrated Matrox G200eW3 Graphics Controller	1007.38	Ubuntu 22.04.4 LTS	0.1.31	llama2:7b	8.42
Linux	Intel Corporation	Integrated Matrox G200eW3 Graphics Controller	1007.38	Ubuntu 22.04.4 LTS	0.1.31	llama2:13b	6.13
Linux	Intel Corporation	Integrated Matrox G200eW3 Graphics Controller	1007.38	Ubuntu 22.04.4 LTS	0.1.31	llava:7b	7.63
Linux	Intel Corporation	Integrated Matrox G200eW3 Graphics Controller	1007.38	Ubuntu 22.04.4 LTS	0.1.31	llava:13b	6.9
Linux	Intel Corporation	Integrated Matrox G200eW3 Graphics Controller	1007.39	Ubuntu 22.04.4 LTS	0.1.31	mistral:7b	9.13
Linux	Intel Corporation	Integrated Matrox G200eW3 Graphics Controller	1007.39	Ubuntu 22.04.4 LTS	0.1.31	gemma:2b	48.72
Linux	Intel Corporation	Integrated Matrox G200eW3 Graphics Controller	1007.39	Ubuntu 22.04.4 LTS	0.1.31	gemma:7b	13.28
Linux	Intel Corporation	Integrated Matrox G200eW3 Graphics Controller	1007.39	Ubuntu 22.04.4 LTS	0.1.31	llama2:7b	22.71
Linux	Intel Corporation	Integrated Matrox G200eW3 Graphics Controller	1007.39	Ubuntu 22.04.4 LTS	0.1.31	llama2:13b	5.4
Linux	Intel Corporation	Integrated Matrox G200eW3 Graphics Controller	1007.39	Ubuntu 22.04.4 LTS	0.1.31	llava:7b	11.97
Linux	Intel Corporation	Integrated Matrox G200eW3 Graphics Controller	1007.39	Ubuntu 22.04.4 LTS	0.1.31	llava:13b	7.75
Linux	Intel Corporation	Integrated Matrox G200eW3 Graphics Controller	1007.38	Ubuntu 22.04.4 LTS	0.1.31	mistral:7b	6.59
Linux	Intel Corporation	Integrated Matrox G200eW3 Graphics Controller	1007.38	Ubuntu 22.04.4 LTS	0.1.31	gemma:2b	13.75
Linux	Intel Corporation	Integrated Matrox G200eW3 Graphics Controller	1007.38	Ubuntu 22.04.4 LTS	0.1.31	gemma:7b	6.28
Linux	Intel Corporation	Integrated Matrox G200eW3 Graphics Controller	1007.38	Ubuntu 22.04.4 LTS	0.1.31	llama2:7b	10.02
Linux	Intel Corporation	Integrated Matrox G200eW3 Graphics Controller	1007.38	Ubuntu 22.04.4 LTS	0.1.31	llama2:13b	6.4
Linux	Intel Corporation	Integrated Matrox G200eW3 Graphics Controller	1007.38	Ubuntu 22.04.4 LTS	0.1.31	llava:7b	13.77
Linux	Intel Corporation	Integrated Matrox G200eW3 Graphics Controller	1007.38	Ubuntu 22.04.4 LTS	0.1.31	llava:13b	7.89
Linux	Intel Corporation	Integrated Matrox G200eW3 Graphics Controller	1007.38	Ubuntu 22.04.4 LTS	0.1.31	mistral:7b	6.07
Linux	Intel Corporation	Integrated Matrox G200eW3 Graphics Controller	1007.38	Ubuntu 22.04.4 LTS	0.1.31	gemma:2b	10.76
Linux	Intel Corporation	Integrated Matrox G200eW3 Graphics Controller	1007.38	Ubuntu 22.04.4 LTS	0.1.31	gemma:7b	10.41
Linux	Intel Corporation	Integrated Matrox G200eW3 Graphics Controller	1007.38	Ubuntu 22.04.4 LTS	0.1.31	llama2:7b	10.12
Linux	Intel Corporation	Integrated Matrox G200eW3 Graphics Controller	1007.38	Ubuntu 22.04.4 LTS	0.1.31	llama2:13b	6.52
Linux	Intel Corporation	Integrated Matrox G200eW3 Graphics Controller	1007.38	Ubuntu 22.04.4 LTS	0.1.31	llava:7b	11.96
Linux	Intel Corporation	Integrated Matrox G200eW3 Graphics Controller	1007.38	Ubuntu 22.04.4 LTS	0.1.31	llava:13b	9.31
Linux	AMD Ryzen Threadripper PRO 7975WX 32-Cores	unknown	160.98	Ubuntu 22.04.4 LTS	0.1.30	mistral:7b	14.3
Linux	AMD Ryzen Threadripper PRO 7975WX 32-Cores	unknown	160.98	Ubuntu 22.04.4 LTS	0.1.30	gemma:2b	31.14
Linux	AMD Ryzen Threadripper PRO 7975WX 32-Cores	unknown	160.98	Ubuntu 22.04.4 LTS	0.1.30	gemma:7b	11.29
Linux	AMD Ryzen Threadripper PRO 7975WX 32-Cores	unknown	160.98	Ubuntu 22.04.4 LTS	0.1.30	llama2:7b	14.34
Linux	AMD Ryzen Threadripper PRO 7975WX 32-Cores	unknown	160.98	Ubuntu 22.04.4 LTS	0.1.30	llama2:13b	7.98
Linux	AMD Ryzen Threadripper PRO 7975WX 32-Cores	unknown	160.98	Ubuntu 22.04.4 LTS	0.1.30	llava:7b	14.22
Linux	AMD Ryzen Threadripper PRO 7975WX 32-Cores	unknown	160.98	Ubuntu 22.04.4 LTS	0.1.30	llava:13b	8.27
Linux	Intel(R) Xeon(R) Gold 6238R CPU @ 2.20GHz	NVIDIA A100-PCIE-40GB	440.48	Ubuntu 22.04.4 LTS	0.1.31	mistral:7b	117.69
Linux	Intel(R) Xeon(R) Gold 6238R CPU @ 2.20GHz	NVIDIA A100-PCIE-40GB	440.48	Ubuntu 22.04.4 LTS	0.1.31	gemma:2b	169.03
Linux	Intel(R) Xeon(R) Gold 6238R CPU @ 2.20GHz	NVIDIA A100-PCIE-40GB	440.48	Ubuntu 22.04.4 LTS	0.1.31	gemma:7b	100.91
Linux	Intel(R) Xeon(R) Gold 6238R CPU @ 2.20GHz	NVIDIA A100-PCIE-40GB	440.48	Ubuntu 22.04.4 LTS	0.1.31	llama2:7b	125.34
Linux	Intel(R) Xeon(R) Gold 6238R CPU @ 2.20GHz	NVIDIA A100-PCIE-40GB	440.48	Ubuntu 22.04.4 LTS	0.1.31	llama2:13b	83.81
Linux	Intel(R) Xeon(R) Gold 6238R CPU @ 2.20GHz	NVIDIA A100-PCIE-40GB	440.48	Ubuntu 22.04.4 LTS	0.1.31	llava:7b	122.16
Linux	Intel(R) Xeon(R) Gold 6238R CPU @ 2.20GHz	NVIDIA A100-PCIE-40GB	440.48	Ubuntu 22.04.4 LTS	0.1.31	llava:13b	83.82
Linux	AMD EPYC 7F72 24-Core Processor	bochs-drmdrmfb	251.67	Ubuntu 22.04.4 LTS	0.1.30	mistral:7b	15.24
Linux	AMD EPYC 7F72 24-Core Processor	bochs-drmdrmfb	251.67	Ubuntu 22.04.4 LTS	0.1.30	gemma:2b	26.15
Linux	AMD EPYC 7F72 24-Core Processor	bochs-drmdrmfb	251.67	Ubuntu 22.04.4 LTS	0.1.30	gemma:7b	12.52
Linux	AMD EPYC 7F72 24-Core Processor	bochs-drmdrmfb	251.67	Ubuntu 22.04.4 LTS	0.1.30	llama2:7b	15
Linux	AMD EPYC 7F72 24-Core Processor	bochs-drmdrmfb	251.67	Ubuntu 22.04.4 LTS	0.1.30	llama2:13b	8.18
Linux	AMD EPYC 7F72 24-Core Processor	bochs-drmdrmfb	251.67	Ubuntu 22.04.4 LTS	0.1.30	llava:7b	15.47
Linux	AMD EPYC 7F72 24-Core Processor	bochs-drmdrmfb	251.67	Ubuntu 22.04.4 LTS	0.1.30	llava:13b	9.95
Linux	AMD EPYC 7F72 24-Core Processor	bochs-drmdrmfb	251.67	Ubuntu 22.04.4 LTS	0.1.30	mistral:7b	15.44
Linux	AMD EPYC 7F72 24-Core Processor	bochs-drmdrmfb	251.67	Ubuntu 22.04.4 LTS	0.1.30	gemma:2b	25.46
Linux	AMD EPYC 7F72 24-Core Processor	bochs-drmdrmfb	251.67	Ubuntu 22.04.4 LTS	0.1.30	gemma:7b	12.52
Linux	AMD EPYC 7F72 24-Core Processor	bochs-drmdrmfb	251.67	Ubuntu 22.04.4 LTS	0.1.30	llama2:7b	15.06
Linux	AMD EPYC 7F72 24-Core Processor	bochs-drmdrmfb	251.67	Ubuntu 22.04.4 LTS	0.1.30	llama2:13b	8.16
Linux	AMD EPYC 7F72 24-Core Processor	bochs-drmdrmfb	251.67	Ubuntu 22.04.4 LTS	0.1.30	llava:7b	15.54
Linux	AMD EPYC 7F72 24-Core Processor	bochs-drmdrmfb	251.67	Ubuntu 22.04.4 LTS	0.1.30	llava:13b	9.73
Linux	AMD EPYC 7F72 24-Core Processor	unknown	47.05	Ubuntu 20.04.3 LTS	0.1.30	mistral:7b	16.6
Linux	AMD EPYC 7F72 24-Core Processor	unknown	47.05	Ubuntu 20.04.3 LTS	0.1.30	gemma:2b	32.17
Linux	AMD EPYC 7F72 24-Core Processor	unknown	47.05	Ubuntu 20.04.3 LTS	0.1.30	gemma:7b	12.74
Linux	AMD EPYC 7F72 24-Core Processor	unknown	47.05	Ubuntu 20.04.3 LTS	0.1.30	llama2:7b	15.2
Linux	AMD EPYC 7F72 24-Core Processor	unknown	47.05	Ubuntu 20.04.3 LTS	0.1.30	llama2:13b	9.41
Linux	AMD EPYC 7F72 24-Core Processor	unknown	47.05	Ubuntu 20.04.3 LTS	0.1.30	llava:7b	15.87
Linux	AMD EPYC 7F72 24-Core Processor	unknown	47.05	Ubuntu 20.04.3 LTS	0.1.30	llava:13b	9.69
Linux	Intel(R) Xeon(R) Gold 6238R CPU @ 2.20GHz	NVIDIA A100-PCIE-40GB	440.48	Ubuntu 22.04.4 LTS	0.1.31	mistral:7b	118.87
Linux	Intel(R) Xeon(R) Gold 6238R CPU @ 2.20GHz	NVIDIA A100-PCIE-40GB	440.48	Ubuntu 22.04.4 LTS	0.1.31	gemma:2b	168.07
Linux	Intel(R) Xeon(R) Gold 6238R CPU @ 2.20GHz	NVIDIA A100-PCIE-40GB	440.48	Ubuntu 22.04.4 LTS	0.1.31	gemma:7b	100.97
Linux	Intel(R) Xeon(R) Gold 6238R CPU @ 2.20GHz	NVIDIA A100-PCIE-40GB	440.48	Ubuntu 22.04.4 LTS	0.1.31	llama2:7b	122.45
Linux	Intel(R) Xeon(R) Gold 6238R CPU @ 2.20GHz	NVIDIA A100-PCIE-40GB	440.48	Ubuntu 22.04.4 LTS	0.1.31	llama2:13b	83.6
Linux	Intel(R) Xeon(R) Gold 6238R CPU @ 2.20GHz	NVIDIA A100-PCIE-40GB	440.48	Ubuntu 22.04.4 LTS	0.1.31	llava:7b	121.32
Linux	Intel(R) Xeon(R) Gold 6238R CPU @ 2.20GHz	NVIDIA A100-PCIE-40GB	440.48	Ubuntu 22.04.4 LTS	0.1.31	llava:13b	83.96
Linux	AMD EPYC 7F72 24-Core Processor	bochs-drmdrmfb	251.67	Ubuntu 22.04.4 LTS	0.1.30	mistral:7b	15.17
Linux	AMD EPYC 7F72 24-Core Processor	bochs-drmdrmfb	251.67	Ubuntu 22.04.4 LTS	0.1.30	gemma:2b	25.33
Linux	AMD EPYC 7F72 24-Core Processor	bochs-drmdrmfb	251.67	Ubuntu 22.04.4 LTS	0.1.30	gemma:7b	12.58
Linux	AMD EPYC 7F72 24-Core Processor	bochs-drmdrmfb	251.67	Ubuntu 22.04.4 LTS	0.1.30	llama2:7b	15.04
Linux	AMD EPYC 7F72 24-Core Processor	bochs-drmdrmfb	251.67	Ubuntu 22.04.4 LTS	0.1.30	llama2:13b	8.32
Linux	AMD EPYC 7F72 24-Core Processor	bochs-drmdrmfb	251.67	Ubuntu 22.04.4 LTS	0.1.30	llava:7b	15.32
Linux	AMD EPYC 7F72 24-Core Processor	bochs-drmdrmfb	251.67	Ubuntu 22.04.4 LTS	0.1.30	llava:13b	10.01
Linux	AMD Ryzen 9 3900X 12-Core Processor	Navi 31 [Radeon RX 7900 XT/7900 XTX/7900M]	125.70	Arch Linux	0.1.31	mistral:7b	87.68
Linux	AMD Ryzen 9 3900X 12-Core Processor	Navi 31 [Radeon RX 7900 XT/7900 XTX/7900M]	125.70	Arch Linux	0.1.31	gemma:2b	162.07
Linux	AMD Ryzen 9 3900X 12-Core Processor	Navi 31 [Radeon RX 7900 XT/7900 XTX/7900M]	125.70	Arch Linux	0.1.31	gemma:7b	76.6
Linux	AMD Ryzen 9 3900X 12-Core Processor	Navi 31 [Radeon RX 7900 XT/7900 XTX/7900M]	125.70	Arch Linux	0.1.31	llama2:7b	89.62
Linux	AMD Ryzen 9 3900X 12-Core Processor	Navi 31 [Radeon RX 7900 XT/7900 XTX/7900M]	125.70	Arch Linux	0.1.31	llama2:13b	59.94
Linux	AMD Ryzen 9 3900X 12-Core Processor	Navi 31 [Radeon RX 7900 XT/7900 XTX/7900M]	125.70	Arch Linux	0.1.31	llava:7b	91.3
Linux	AMD Ryzen 9 3900X 12-Core Processor	Navi 31 [Radeon RX 7900 XT/7900 XTX/7900M]	125.70	Arch Linux	0.1.31	llava:13b	60.63
Linux	AMD Ryzen 9 3900X 12-Core Processor	Navi 31 [Radeon RX 7900 XT/7900 XTX/7900M]	125.70	Arch Linux	0.1.31	mistral:7b	91.09
Linux	AMD Ryzen 9 3900X 12-Core Processor	Navi 31 [Radeon RX 7900 XT/7900 XTX/7900M]	125.70	Arch Linux	0.1.31	gemma:2b	163.46
Linux	AMD Ryzen 9 3900X 12-Core Processor	Navi 31 [Radeon RX 7900 XT/7900 XTX/7900M]	125.70	Arch Linux	0.1.31	gemma:7b	79.16
Linux	AMD Ryzen 9 3900X 12-Core Processor	Navi 31 [Radeon RX 7900 XT/7900 XTX/7900M]	125.70	Arch Linux	0.1.31	llama2:7b	92.71
Linux	AMD Ryzen 9 3900X 12-Core Processor	Navi 31 [Radeon RX 7900 XT/7900 XTX/7900M]	125.70	Arch Linux	0.1.31	llama2:13b	61.11
Linux	AMD Ryzen 9 3900X 12-Core Processor	Navi 31 [Radeon RX 7900 XT/7900 XTX/7900M]	125.70	Arch Linux	0.1.31	llava:7b	93.05
Linux	AMD Ryzen 9 3900X 12-Core Processor	Navi 31 [Radeon RX 7900 XT/7900 XTX/7900M]	125.70	Arch Linux	0.1.31	llava:13b	62.79
Linux	AMD Ryzen 9 3900X 12-Core Processor	Navi 31 [Radeon RX 7900 XT/7900 XTX/7900M]	125.70	Arch Linux	0.1.31	mistral:7b	79.79
Linux	AMD Ryzen 9 3900X 12-Core Processor	Navi 31 [Radeon RX 7900 XT/7900 XTX/7900M]	125.70	Arch Linux	0.1.31	gemma:2b	154.08
Linux	AMD Ryzen 9 3900X 12-Core Processor	Navi 31 [Radeon RX 7900 XT/7900 XTX/7900M]	125.70	Arch Linux	0.1.31	gemma:7b	65.7
Linux	AMD Ryzen 9 3900X 12-Core Processor	Navi 31 [Radeon RX 7900 XT/7900 XTX/7900M]	125.70	Arch Linux	0.1.31	llama2:7b	82.25
Linux	AMD Ryzen 9 3900X 12-Core Processor	Navi 31 [Radeon RX 7900 XT/7900 XTX/7900M]	125.70	Arch Linux	0.1.31	llama2:13b	48.71
Linux	AMD Ryzen 9 3900X 12-Core Processor	Navi 31 [Radeon RX 7900 XT/7900 XTX/7900M]	125.70	Arch Linux	0.1.31	llava:7b	81.63
Linux	AMD Ryzen 9 3900X 12-Core Processor	Navi 31 [Radeon RX 7900 XT/7900 XTX/7900M]	125.70	Arch Linux	0.1.31	llava:13b	50.25
Linux	AMD Ryzen 7 3700X 8-Core Processor	unknown	46.97	Arch Linux	0.0.0	mistral:7b	65.92
Linux	AMD Ryzen 7 3700X 8-Core Processor	unknown	46.97	Arch Linux	0.0.0	gemma:2b	124.37
Linux	AMD Ryzen 7 3700X 8-Core Processor	unknown	46.97	Arch Linux	0.0.0	gemma:7b	56.86
Linux	AMD Ryzen 7 3700X 8-Core Processor	unknown	46.97	Arch Linux	0.0.0	llama2:7b	64.36
Linux	AMD Ryzen 7 3700X 8-Core Processor	unknown	46.97	Arch Linux	0.0.0	llama2:13b	41.16
Linux	AMD Ryzen 7 3700X 8-Core Processor	unknown	46.97	Arch Linux	0.0.0	llava:7b	70.2
Linux	AMD Ryzen 7 3700X 8-Core Processor	unknown	46.97	Arch Linux	0.0.0	llava:13b	42.44
Linux	AMD Ryzen 9 3900X 12-Core Processor	Navi 31 [Radeon RX 7900 XT/7900 XTX/7900M]	125.70	Arch Linux	0.1.31	mistral:7b	41
Linux	AMD Ryzen 9 3900X 12-Core Processor	Navi 31 [Radeon RX 7900 XT/7900 XTX/7900M]	125.70	Arch Linux	0.1.31	gemma:2b	80.91
Linux	AMD Ryzen 9 3900X 12-Core Processor	Navi 31 [Radeon RX 7900 XT/7900 XTX/7900M]	125.70	Arch Linux	0.1.31	gemma:7b	34.67
Linux	AMD Ryzen 9 3900X 12-Core Processor	Navi 31 [Radeon RX 7900 XT/7900 XTX/7900M]	125.70	Arch Linux	0.1.31	llama2:7b	41.69
Linux	AMD Ryzen 9 3900X 12-Core Processor	Navi 31 [Radeon RX 7900 XT/7900 XTX/7900M]	125.70	Arch Linux	0.1.31	llama2:13b	25.04
Linux	AMD Ryzen 9 3900X 12-Core Processor	Navi 31 [Radeon RX 7900 XT/7900 XTX/7900M]	125.70	Arch Linux	0.1.31	llava:7b	43.1
Linux	AMD Ryzen 9 3900X 12-Core Processor	Navi 31 [Radeon RX 7900 XT/7900 XTX/7900M]	125.70	Arch Linux	0.1.31	llava:13b	25.64
Linux	Intel(R) Xeon(R) CPU Max 9460	NVIDIA A30	1007.54	Ubuntu 22.04.4 LTS	0.1.31	mistral:7b	79.42
Linux	Intel(R) Xeon(R) CPU Max 9460	NVIDIA A30	1007.54	Ubuntu 22.04.4 LTS	0.1.31	gemma:2b	126.84
Linux	Intel(R) Xeon(R) CPU Max 9460	NVIDIA A30	1007.54	Ubuntu 22.04.4 LTS	0.1.31	gemma:7b	71.66
Linux	Intel(R) Xeon(R) CPU Max 9460	NVIDIA A30	1007.54	Ubuntu 22.04.4 LTS	0.1.31	llama2:7b	81.85
Linux	Intel(R) Xeon(R) CPU Max 9460	NVIDIA A30	1007.54	Ubuntu 22.04.4 LTS	0.1.31	llama2:13b	46.33
Linux	Intel(R) Xeon(R) CPU Max 9460	NVIDIA A30	1007.54	Ubuntu 22.04.4 LTS	0.1.31	llava:7b	71.26
Linux	Intel(R) Xeon(R) CPU Max 9460	NVIDIA A30	1007.54	Ubuntu 22.04.4 LTS	0.1.31	llava:13b	60.34
Linux	Intel(R) Core(TM) i9-10980XE CPU @ 3.00GHz	NVIDIA GeForce RTX 3050	251.37	Ubuntu 22.04.4 LTS	0.1.30	mistral:7b	42.36
Linux	Intel(R) Core(TM) i9-10980XE CPU @ 3.00GHz	NVIDIA GeForce RTX 3050	251.37	Ubuntu 22.04.4 LTS	0.1.30	gemma:2b	86.21
Linux	Intel(R) Core(TM) i9-10980XE CPU @ 3.00GHz	NVIDIA GeForce RTX 3050	251.37	Ubuntu 22.04.4 LTS	0.1.30	gemma:7b	34.96
Linux	Intel(R) Core(TM) i9-10980XE CPU @ 3.00GHz	NVIDIA GeForce RTX 3050	251.37	Ubuntu 22.04.4 LTS	0.1.30	llama2:7b	43.97
Linux	Intel(R) Core(TM) i9-10980XE CPU @ 3.00GHz	NVIDIA GeForce RTX 3050	251.37	Ubuntu 22.04.4 LTS	0.1.30	llama2:13b	11.43
Linux	Intel(R) Core(TM) i9-10980XE CPU @ 3.00GHz	NVIDIA GeForce RTX 3050	251.37	Ubuntu 22.04.4 LTS	0.1.30	llava:7b	43.58
Linux	Intel(R) Core(TM) i9-10980XE CPU @ 3.00GHz	NVIDIA GeForce RTX 3050	251.37	Ubuntu 22.04.4 LTS	0.1.30	llava:13b	12
Linux	11th Gen Intel(R) Core(TM) i7-1165G7 @ 2.80GHz	TigerLake-LP GT2 [Iris Xe Graphics]	31.06	Ubuntu 22.04.4 LTS	0.1.30	mistral:7b	2.23
Linux	11th Gen Intel(R) Core(TM) i7-1165G7 @ 2.80GHz	TigerLake-LP GT2 [Iris Xe Graphics]	31.06	Ubuntu 22.04.4 LTS	0.1.30	gemma:2b	6.32
Linux	11th Gen Intel(R) Core(TM) i7-1165G7 @ 2.80GHz	TigerLake-LP GT2 [Iris Xe Graphics]	31.06	Ubuntu 22.04.4 LTS	0.1.30	gemma:7b	1.42
Linux	11th Gen Intel(R) Core(TM) i7-1165G7 @ 2.80GHz	TigerLake-LP GT2 [Iris Xe Graphics]	31.06	Ubuntu 22.04.4 LTS	0.1.30	llama2:7b	1.71
Linux	11th Gen Intel(R) Core(TM) i7-1165G7 @ 2.80GHz	TigerLake-LP GT2 [Iris Xe Graphics]	31.06	Ubuntu 22.04.4 LTS	0.1.30	llama2:13b	0.93
Linux	11th Gen Intel(R) Core(TM) i7-1165G7 @ 2.80GHz	TigerLake-LP GT2 [Iris Xe Graphics]	31.06	Ubuntu 22.04.4 LTS	0.1.30	llava:7b	1.65
Linux	11th Gen Intel(R) Core(TM) i7-1165G7 @ 2.80GHz	TigerLake-LP GT2 [Iris Xe Graphics]	31.06	Ubuntu 22.04.4 LTS	0.1.30	llava:13b	0.94
Linux	11th Gen Intel(R) Core(TM) i7-11850H @ 2.50GHz	NVIDIA RTX A3000 Laptop GPU	31.12	Ubuntu 22.04.4 LTS	0.1.31	mistral:7b	49.63
Linux	11th Gen Intel(R) Core(TM) i7-11850H @ 2.50GHz	NVIDIA RTX A3000 Laptop GPU	31.12	Ubuntu 22.04.4 LTS	0.1.31	gemma:2b	99.41
Linux	11th Gen Intel(R) Core(TM) i7-11850H @ 2.50GHz	NVIDIA RTX A3000 Laptop GPU	31.12	Ubuntu 22.04.4 LTS	0.1.31	gemma:7b	17.11
Linux	11th Gen Intel(R) Core(TM) i7-11850H @ 2.50GHz	NVIDIA RTX A3000 Laptop GPU	31.12	Ubuntu 22.04.4 LTS	0.1.31	llama2:7b	35.64
Linux	11th Gen Intel(R) Core(TM) i7-11850H @ 2.50GHz	NVIDIA RTX A3000 Laptop GPU	31.12	Ubuntu 22.04.4 LTS	0.1.31	llama2:13b	7.72
Linux	11th Gen Intel(R) Core(TM) i7-11850H @ 2.50GHz	NVIDIA RTX A3000 Laptop GPU	31.12	Ubuntu 22.04.4 LTS	0.1.31	llava:7b	50.4
Linux	11th Gen Intel(R) Core(TM) i7-11850H @ 2.50GHz	NVIDIA RTX A3000 Laptop GPU	31.12	Ubuntu 22.04.4 LTS	0.1.31	llava:13b	7.89
Linux	AMD Ryzen 7 PRO 7840U w/ Radeon 780M Graphics	Phoenix1	30.05	Ubuntu 23.10	0.1.32	mistral:7b	7.75
Linux	AMD Ryzen 7 PRO 7840U w/ Radeon 780M Graphics	Phoenix1	30.05	Ubuntu 23.10	0.1.32	gemma:2b	15.67
Linux	AMD Ryzen 7 PRO 7840U w/ Radeon 780M Graphics	Phoenix1	30.05	Ubuntu 23.10	0.1.32	gemma:7b	5.89
Linux	AMD Ryzen 7 PRO 7840U w/ Radeon 780M Graphics	Phoenix1	30.05	Ubuntu 23.10	0.1.32	llama2:7b	7.79
Linux	AMD Ryzen 7 PRO 7840U w/ Radeon 780M Graphics	Phoenix1	30.05	Ubuntu 23.10	0.1.32	llama2:13b	4.28
Linux	AMD Ryzen 7 PRO 7840U w/ Radeon 780M Graphics	Phoenix1	30.05	Ubuntu 23.10	0.1.32	llava:7b	7.84
Linux	AMD Ryzen 7 PRO 7840U w/ Radeon 780M Graphics	Phoenix1	30.05	Ubuntu 23.10	0.1.32	llava:13b	4.36
Linux	AMD Ryzen 5 5600G with Radeon Graphics	NVIDIA RTX A4000	23.33	Ubuntu 23.10	0.1.32	gemma:2b	120.27
Linux	AMD Ryzen 5 5600G with Radeon Graphics	NVIDIA RTX A4000	23.33	Ubuntu 23.10	0.1.32	gemma:2b	133.06
Linux	AMD Ryzen 5 5600G with Radeon Graphics	NVIDIA RTX A4000	23.33	Ubuntu 23.10	0.1.32	gemma:2b	132.52
Linux	Intel(R) Xeon(R) CPU E5-2690 v4 @ 2.60GHz	Tesla V100-PCIE-16GB	58.84	Ubuntu 22.04.4 LTS	0.1.32	mistral:7b	99.97
Linux	Intel(R) Xeon(R) CPU E5-2690 v4 @ 2.60GHz	Tesla V100-PCIE-16GB	58.84	Ubuntu 22.04.4 LTS	0.1.32	gemma:2b	139.48
Linux	Intel(R) Xeon(R) CPU E5-2690 v4 @ 2.60GHz	Tesla V100-PCIE-16GB	58.84	Ubuntu 22.04.4 LTS	0.1.32	gemma:7b	81.88
Linux	Intel(R) Xeon(R) CPU E5-2690 v4 @ 2.60GHz	Tesla V100-PCIE-16GB	58.84	Ubuntu 22.04.4 LTS	0.1.32	llama2:7b	102.03
Linux	Intel(R) Xeon(R) CPU E5-2690 v4 @ 2.60GHz	Tesla V100-PCIE-16GB	58.84	Ubuntu 22.04.4 LTS	0.1.32	llama2:13b	67.36
Linux	Intel(R) Xeon(R) CPU E5-2690 v4 @ 2.60GHz	Tesla V100-PCIE-16GB	58.84	Ubuntu 22.04.4 LTS	0.1.32	llava:7b	104.19
Linux	Intel(R) Xeon(R) CPU E5-2690 v4 @ 2.60GHz	Tesla V100-PCIE-16GB	58.84	Ubuntu 22.04.4 LTS	0.1.32	llava:13b	68.88
Linux	AMD Ryzen 7 7840U w/ Radeon 780M Graphics	Advanced Micro Devices, Inc. [AMD/ATI]	30.65	KDE neon 6.0	0.1.32	mistral:7b	12.12
Linux	AMD Ryzen 7 7840U w/ Radeon 780M Graphics	Advanced Micro Devices, Inc. [AMD/ATI]	30.65	KDE neon 6.0	0.1.32	gemma:2b	26.39
Linux	AMD Ryzen 7 7840U w/ Radeon 780M Graphics	Advanced Micro Devices, Inc. [AMD/ATI]	30.65	KDE neon 6.0	0.1.32	gemma:7b	9.75
Linux	AMD Ryzen 7 7840U w/ Radeon 780M Graphics	Advanced Micro Devices, Inc. [AMD/ATI]	30.65	KDE neon 6.0	0.1.32	llama2:7b	12.01
Linux	AMD Ryzen 7 7840U w/ Radeon 780M Graphics	Advanced Micro Devices, Inc. [AMD/ATI]	30.65	KDE neon 6.0	0.1.32	llama2:13b	6.82
Linux	AMD Ryzen 7 7840U w/ Radeon 780M Graphics	Advanced Micro Devices, Inc. [AMD/ATI]	30.65	KDE neon 6.0	0.1.32	llava:7b	12.67
Linux	AMD Ryzen 7 7840U w/ Radeon 780M Graphics	Advanced Micro Devices, Inc. [AMD/ATI]	30.65	KDE neon 6.0	0.1.32	llava:13b	7.02
Linux	AMD Ryzen 7 7840U w/ Radeon 780M Graphics	no_gpu	58.66	Fedora Linux 39 (Sway)	0.1.32	mistral:7b	12.41
Linux	AMD Ryzen 7 7840U w/ Radeon 780M Graphics	no_gpu	58.66	Fedora Linux 39 (Sway)	0.1.32	gemma:2b	28.17
Linux	AMD Ryzen 7 7840U w/ Radeon 780M Graphics	no_gpu	58.66	Fedora Linux 39 (Sway)	0.1.32	gemma:7b	9.97
Linux	AMD Ryzen 7 7840U w/ Radeon 780M Graphics	no_gpu	58.66	Fedora Linux 39 (Sway)	0.1.32	llama2:7b	12.54
Linux	AMD Ryzen 7 7840U w/ Radeon 780M Graphics	no_gpu	58.66	Fedora Linux 39 (Sway)	0.1.32	llama2:13b	6.87
Linux	AMD Ryzen 7 7840U w/ Radeon 780M Graphics	no_gpu	58.66	Fedora Linux 39 (Sway)	0.1.32	llava:7b	12.85
Linux	AMD Ryzen 7 7840U w/ Radeon 780M Graphics	no_gpu	58.66	Fedora Linux 39 (Sway)	0.1.32	llava:13b	7.03
Linux	12th Gen Intel(R) Core(TM) i7-12700KF	NVIDIA GeForce RTX 4070	27.42	Ubuntu 20.04.5 LTS	0.1.39	mistral:7b	88.4
Linux	QEMU Virtual CPU version 2.5+	bochs-drmdrmfb	62.88	Ubuntu 22.04.4 LTS	0.1.32	mistral:7b	2.03
Linux	QEMU Virtual CPU version 2.5+	bochs-drmdrmfb	62.88	Ubuntu 22.04.4 LTS	0.1.32	gemma:2b	5.24
Linux	QEMU Virtual CPU version 2.5+	bochs-drmdrmfb	62.88	Ubuntu 22.04.4 LTS	0.1.32	gemma:7b	1.64
Linux	QEMU Virtual CPU version 2.5+	bochs-drmdrmfb	62.88	Ubuntu 22.04.4 LTS	0.1.32	llama2:7b	2.06
Linux	QEMU Virtual CPU version 2.5+	bochs-drmdrmfb	62.88	Ubuntu 22.04.4 LTS	0.1.32	llama2:13b	1.15
Linux	QEMU Virtual CPU version 2.5+	bochs-drmdrmfb	62.88	Ubuntu 22.04.4 LTS	0.1.32	llava:7b	2.05
Linux	QEMU Virtual CPU version 2.5+	bochs-drmdrmfb	62.88	Ubuntu 22.04.4 LTS	0.1.32	llava:13b	1.14
Linux	QEMU Virtual CPU version 2.5+	bochs-drmdrmfb	62.86	Ubuntu 22.04.4 LTS	0.1.32	mistral:7b	1.22
Linux	QEMU Virtual CPU version 2.5+	bochs-drmdrmfb	62.86	Ubuntu 22.04.4 LTS	0.1.32	gemma:2b	3.08
Linux	QEMU Virtual CPU version 2.5+	bochs-drmdrmfb	62.86	Ubuntu 22.04.4 LTS	0.1.32	gemma:7b	1.16
Linux	QEMU Virtual CPU version 2.5+	bochs-drmdrmfb	62.86	Ubuntu 22.04.4 LTS	0.1.32	llama2:7b	1.32
Linux	QEMU Virtual CPU version 2.5+	bochs-drmdrmfb	62.86	Ubuntu 22.04.4 LTS	0.1.32	llama2:13b	0.72
Linux	QEMU Virtual CPU version 2.5+	bochs-drmdrmfb	62.86	Ubuntu 22.04.4 LTS	0.1.32	llava:7b	1.27
Linux	QEMU Virtual CPU version 2.5+	bochs-drmdrmfb	62.86	Ubuntu 22.04.4 LTS	0.1.32	llava:13b	0.82
Linux	13th Gen Intel(R) Core(TM) i5-13600K	Raptor Lake-S GT1 [UHD Graphics 770]	31.06	KDE neon 6.0	0.1.28	mistral:7b	16.98
Linux	13th Gen Intel(R) Core(TM) i5-13600K	Raptor Lake-S GT1 [UHD Graphics 770]	31.06	KDE neon 6.0	0.1.28	gemma:2b	28.47
Linux	13th Gen Intel(R) Core(TM) i5-13600K	Raptor Lake-S GT1 [UHD Graphics 770]	31.06	KDE neon 6.0	0.1.28	gemma:7b	21.19
Linux	13th Gen Intel(R) Core(TM) i5-13600K	Raptor Lake-S GT1 [UHD Graphics 770]	31.06	KDE neon 6.0	0.1.28	llama2:7b	21.05
Linux	13th Gen Intel(R) Core(TM) i5-13600K	Raptor Lake-S GT1 [UHD Graphics 770]	31.06	KDE neon 6.0	0.1.28	llama2:13b	13.15
Linux	13th Gen Intel(R) Core(TM) i5-13600K	Raptor Lake-S GT1 [UHD Graphics 770]	31.06	KDE neon 6.0	0.1.28	llava:7b	17.41
Linux	13th Gen Intel(R) Core(TM) i5-13600K	Raptor Lake-S GT1 [UHD Graphics 770]	31.06	KDE neon 6.0	0.1.28	llava:13b	14.16
Linux	Intel(R) Core(TM) i9-14900K	NVIDIA GeForce RTX 4090	125.50	Ubuntu 22.04.4 LTS	0.1.32	mistral:7b	142.67
Linux	Intel(R) Core(TM) i9-14900K	NVIDIA GeForce RTX 4090	125.50	Ubuntu 22.04.4 LTS	0.1.32	gemma:2b	222.98
Linux	Intel(R) Core(TM) i9-14900K	NVIDIA GeForce RTX 4090	125.50	Ubuntu 22.04.4 LTS	0.1.32	gemma:7b	116.27
Linux	Intel(R) Core(TM) i9-14900K	NVIDIA GeForce RTX 4090	125.50	Ubuntu 22.04.4 LTS	0.1.32	llama2:7b	148.75
Linux	Intel(R) Core(TM) i9-14900K	NVIDIA GeForce RTX 4090	125.50	Ubuntu 22.04.4 LTS	0.1.32	llama2:13b	90.82
Linux	Intel(R) Core(TM) i9-14900K	NVIDIA GeForce RTX 4090	125.50	Ubuntu 22.04.4 LTS	0.1.32	llava:7b	144.99
Linux	Intel(R) Core(TM) i9-14900K	NVIDIA GeForce RTX 4090	125.50	Ubuntu 22.04.4 LTS	0.1.32	llava:13b	93.76
Linux	AMD Ryzen 9 5950X 16-Core Processor	NVIDIA GeForce RTX 4060 Ti	31.30	Ubuntu 22.04.3 LTS	0.1.32	mistral:7b	41.9
Linux	AMD Ryzen 9 5950X 16-Core Processor	NVIDIA GeForce RTX 4060 Ti	31.30	Ubuntu 22.04.3 LTS	0.1.32	gemma:2b	57.02
Linux	AMD Ryzen 9 5950X 16-Core Processor	NVIDIA GeForce RTX 4060 Ti	31.30	Ubuntu 22.04.3 LTS	0.1.32	gemma:7b	38.02
Linux	AMD Ryzen 9 5950X 16-Core Processor	NVIDIA GeForce RTX 4060 Ti	31.30	Ubuntu 22.04.3 LTS	0.1.32	llama2:7b	45.31
Linux	AMD Ryzen 9 5950X 16-Core Processor	NVIDIA GeForce RTX 4060 Ti	31.30	Ubuntu 22.04.3 LTS	0.1.32	llama2:13b	28.96
Linux	AMD Ryzen 9 5950X 16-Core Processor	NVIDIA GeForce RTX 4060 Ti	31.30	Ubuntu 22.04.3 LTS	0.1.32	llava:7b	43.38
Linux	AMD Ryzen 9 5950X 16-Core Processor	NVIDIA GeForce RTX 4060 Ti	31.30	Ubuntu 22.04.3 LTS	0.1.32	llava:13b	28.74
Linux	AMD Ryzen 9 5950X 16-Core Processor	NVIDIA GeForce RTX 4060 Ti	31.30	Ubuntu 22.04.3 LTS	0.1.32	mistral:7b	42.39
Linux	AMD Ryzen 9 5950X 16-Core Processor	NVIDIA GeForce RTX 4060 Ti	31.30	Ubuntu 22.04.3 LTS	0.1.32	gemma:2b	58.16
Linux	AMD Ryzen 9 5950X 16-Core Processor	NVIDIA GeForce RTX 4060 Ti	31.30	Ubuntu 22.04.3 LTS	0.1.32	gemma:7b	38.64
Linux	AMD Ryzen 9 5950X 16-Core Processor	NVIDIA GeForce RTX 4060 Ti	31.30	Ubuntu 22.04.3 LTS	0.1.32	llama2:7b	44.55
Linux	AMD Ryzen 9 5950X 16-Core Processor	NVIDIA GeForce RTX 4060 Ti	31.30	Ubuntu 22.04.3 LTS	0.1.32	llama2:13b	29.29
Linux	AMD Ryzen 9 5950X 16-Core Processor	NVIDIA GeForce RTX 4060 Ti	31.30	Ubuntu 22.04.3 LTS	0.1.32	llava:7b	41.33
Linux	AMD Ryzen 9 5950X 16-Core Processor	NVIDIA GeForce RTX 4060 Ti	31.30	Ubuntu 22.04.3 LTS	0.1.32	llava:13b	28.54
Linux	Intel(R) Xeon(R) CPU E5-2470 v2 @ 2.40GHz	NVIDIA GeForce GTX 1650 SUPER	31.33	Ubuntu 22.04.4 LTS	0.1.32	mistral:7b	5.82
Linux	Intel(R) Xeon(R) CPU E5-2470 v2 @ 2.40GHz	NVIDIA GeForce GTX 1650 SUPER	31.33	Ubuntu 22.04.4 LTS	0.1.32	gemma:2b	47
Linux	Intel(R) Xeon(R) CPU E5-2470 v2 @ 2.40GHz	NVIDIA GeForce GTX 1650 SUPER	31.33	Ubuntu 22.04.4 LTS	0.1.32	gemma:7b	2.73
Linux	Intel(R) Xeon(R) CPU E5-2470 v2 @ 2.40GHz	NVIDIA GeForce GTX 1650 SUPER	31.33	Ubuntu 22.04.4 LTS	0.1.32	llama2:7b	4.96
Linux	Intel(R) Xeon(R) CPU E5-2470 v2 @ 2.40GHz	NVIDIA GeForce GTX 1650 SUPER	31.33	Ubuntu 22.04.4 LTS	0.1.32	llama2:13b	1.76
Linux	Intel(R) Xeon(R) CPU E5-2470 v2 @ 2.40GHz	NVIDIA GeForce GTX 1650 SUPER	31.33	Ubuntu 22.04.4 LTS	0.1.32	llava:7b	0.7
Linux	Intel(R) Xeon(R) CPU E5-2470 v2 @ 2.40GHz	NVIDIA GeForce GTX 1650 SUPER	31.33	Ubuntu 22.04.4 LTS	0.1.32	llava:13b	1.69
Linux	Intel(R) Xeon(R) CPU E5-2697A v4 @ 2.60GHz	NVIDIA GeForce GTX 1080 Ti	125.69	Ubuntu 22.04.4 LTS	0.1.32	mistral:7b	57.16
Linux	Intel(R) Xeon(R) CPU E5-2697A v4 @ 2.60GHz	NVIDIA GeForce GTX 1080 Ti	125.69	Ubuntu 22.04.4 LTS	0.1.32	gemma:2b	106.37
Linux	Intel(R) Xeon(R) CPU E5-2697A v4 @ 2.60GHz	NVIDIA GeForce GTX 1080 Ti	125.69	Ubuntu 22.04.4 LTS	0.1.32	gemma:7b	45.4
Linux	Intel(R) Xeon(R) CPU E5-2697A v4 @ 2.60GHz	NVIDIA GeForce GTX 1080 Ti	125.69	Ubuntu 22.04.4 LTS	0.1.32	llama2:7b	58.2
Linux	Intel(R) Xeon(R) CPU E5-2697A v4 @ 2.60GHz	NVIDIA GeForce GTX 1080 Ti	125.69	Ubuntu 22.04.4 LTS	0.1.32	llama2:13b	33.84
Linux	Intel(R) Xeon(R) CPU E5-2697A v4 @ 2.60GHz	NVIDIA GeForce GTX 1080 Ti	125.69	Ubuntu 22.04.4 LTS	0.1.32	llava:7b	59.57
Linux	Intel(R) Xeon(R) CPU E5-2697A v4 @ 2.60GHz	NVIDIA GeForce GTX 1080 Ti	125.69	Ubuntu 22.04.4 LTS	0.1.32	llava:13b	35.3
Linux	Intel(R) Xeon(R) CPU E5-2697A v4 @ 2.60GHz	NVIDIA GeForce GTX 1080 Ti	125.69	Ubuntu 22.04.4 LTS	0.1.32	mistral:7b	56.85
Linux	Intel(R) Xeon(R) CPU E5-2697A v4 @ 2.60GHz	NVIDIA GeForce GTX 1080 Ti	125.69	Ubuntu 22.04.4 LTS	0.1.32	gemma:2b	106.33
Linux	Intel(R) Xeon(R) CPU E5-2697A v4 @ 2.60GHz	NVIDIA GeForce GTX 1080 Ti	125.69	Ubuntu 22.04.4 LTS	0.1.32	gemma:7b	45.37
Linux	Intel(R) Xeon(R) CPU E5-2697A v4 @ 2.60GHz	NVIDIA GeForce GTX 1080 Ti	125.69	Ubuntu 22.04.4 LTS	0.1.32	llama2:7b	58.46
Linux	Intel(R) Xeon(R) CPU E5-2697A v4 @ 2.60GHz	NVIDIA GeForce GTX 1080 Ti	125.69	Ubuntu 22.04.4 LTS	0.1.32	llama2:13b	33.88
Linux	Intel(R) Xeon(R) CPU E5-2697A v4 @ 2.60GHz	NVIDIA GeForce GTX 1080 Ti	125.69	Ubuntu 22.04.4 LTS	0.1.32	llava:7b	59.64
Linux	Intel(R) Xeon(R) CPU E5-2697A v4 @ 2.60GHz	NVIDIA GeForce GTX 1080 Ti	125.69	Ubuntu 22.04.4 LTS	0.1.32	llava:13b	35.06
Linux	AMD Ryzen 9 7950X3D 16-Core Processor	NVIDIA GeForce RTX 4090	30.95	Fedora Linux 40.20240424.0 (Bazzite)	0.1.32	mistral:7b	134.56
Linux	AMD Ryzen 9 7950X3D 16-Core Processor	NVIDIA GeForce RTX 4090	30.95	Fedora Linux 40.20240424.0 (Bazzite)	0.1.32	gemma:2b	216.31
Linux	AMD Ryzen 9 7950X3D 16-Core Processor	NVIDIA GeForce RTX 4090	30.95	Fedora Linux 40.20240424.0 (Bazzite)	0.1.32	gemma:7b	108.95
Linux	AMD Ryzen 9 7950X3D 16-Core Processor	NVIDIA GeForce RTX 4090	30.95	Fedora Linux 40.20240424.0 (Bazzite)	0.1.32	llama2:7b	138.4
Linux	AMD Ryzen 9 7950X3D 16-Core Processor	NVIDIA GeForce RTX 4090	30.95	Fedora Linux 40.20240424.0 (Bazzite)	0.1.32	llama2:13b	85.5
Linux	AMD Ryzen 9 7950X3D 16-Core Processor	NVIDIA GeForce RTX 4090	30.95	Fedora Linux 40.20240424.0 (Bazzite)	0.1.32	llava:7b	139.04
Linux	AMD Ryzen 9 7950X3D 16-Core Processor	NVIDIA GeForce RTX 4090	30.95	Fedora Linux 40.20240424.0 (Bazzite)	0.1.32	llava:13b	88.82
Linux	Intel(R) Xeon(R) CPU E5-2680 v4 @ 2.40GHz	NVIDIA RTX A4000	70.74	Ubuntu 22.04.4 LTS	0.1.32	mistral:7b	66.38
Linux	Intel(R) Xeon(R) CPU E5-2680 v4 @ 2.40GHz	NVIDIA RTX A4000	70.74	Ubuntu 22.04.4 LTS	0.1.32	gemma:2b	104.12
Linux	Intel(R) Xeon(R) CPU E5-2680 v4 @ 2.40GHz	NVIDIA RTX A4000	70.74	Ubuntu 22.04.4 LTS	0.1.32	gemma:7b	51.36
Linux	Intel(R) Xeon(R) CPU E5-2680 v4 @ 2.40GHz	NVIDIA RTX A4000	70.74	Ubuntu 22.04.4 LTS	0.1.32	llama2:7b	67.65
Linux	Intel(R) Xeon(R) CPU E5-2680 v4 @ 2.40GHz	NVIDIA RTX A4000	70.74	Ubuntu 22.04.4 LTS	0.1.32	llama2:13b	40.69
Linux	Intel(R) Xeon(R) CPU E5-2680 v4 @ 2.40GHz	NVIDIA RTX A4000	70.74	Ubuntu 22.04.4 LTS	0.1.32	llava:7b	66.89
Linux	Intel(R) Xeon(R) CPU E5-2680 v4 @ 2.40GHz	NVIDIA RTX A4000	70.74	Ubuntu 22.04.4 LTS	0.1.32	llava:13b	41.31
Linux	AMD Ryzen Threadripper 7960X 24-Cores	NVIDIA GeForce RTX 4090	62.28	Pop!_OS 22.04 LTS	0.1.32	mistral:7b	136.03
Linux	AMD Ryzen Threadripper 7960X 24-Cores	NVIDIA GeForce RTX 4090	62.28	Pop!_OS 22.04 LTS	0.1.32	gemma:2b	220.16
Linux	AMD Ryzen Threadripper 7960X 24-Cores	NVIDIA GeForce RTX 4090	62.28	Pop!_OS 22.04 LTS	0.1.32	gemma:7b	109.76
Linux	AMD Ryzen Threadripper 7960X 24-Cores	NVIDIA GeForce RTX 4090	62.28	Pop!_OS 22.04 LTS	0.1.32	llama2:7b	145.02
Linux	AMD Ryzen Threadripper 7960X 24-Cores	NVIDIA GeForce RTX 4090	62.28	Pop!_OS 22.04 LTS	0.1.32	llama2:13b	88.21
Linux	AMD Ryzen Threadripper 7960X 24-Cores	NVIDIA GeForce RTX 4090	62.28	Pop!_OS 22.04 LTS	0.1.32	llava:7b	141.69
Linux	AMD Ryzen Threadripper 7960X 24-Cores	NVIDIA GeForce RTX 4090	62.28	Pop!_OS 22.04 LTS	0.1.32	llava:13b	90.73
Linux	AMD Ryzen Threadripper 7960X 24-Cores	NVIDIA GeForce RTX 4090	62.28	Pop!_OS 22.04 LTS	0.1.32	mistral:7b	140.26
Linux	AMD Ryzen Threadripper 7960X 24-Cores	NVIDIA GeForce RTX 4090	62.28	Pop!_OS 22.04 LTS	0.1.32	gemma:2b	222.92
Linux	AMD Ryzen Threadripper 7960X 24-Cores	NVIDIA GeForce RTX 4090	62.28	Pop!_OS 22.04 LTS	0.1.32	gemma:7b	114.4
Linux	AMD Ryzen Threadripper 7960X 24-Cores	NVIDIA GeForce RTX 4090	62.28	Pop!_OS 22.04 LTS	0.1.32	llama2:7b	145.36
Linux	AMD Ryzen Threadripper 7960X 24-Cores	NVIDIA GeForce RTX 4090	62.28	Pop!_OS 22.04 LTS	0.1.32	llama2:13b	89.54
Linux	AMD Ryzen Threadripper 7960X 24-Cores	NVIDIA GeForce RTX 4090	62.28	Pop!_OS 22.04 LTS	0.1.32	llava:7b	142.53
Linux	AMD Ryzen Threadripper 7960X 24-Cores	NVIDIA GeForce RTX 4090	62.28	Pop!_OS 22.04 LTS	0.1.32	llava:13b	90.29
Linux	12th Gen Intel(R) Core(TM) i7-12700KF	NVIDIA GeForce RTX 4070	27.42	Ubuntu 20.04.5 LTS	0.1.39	llama3:8b	77.82
Linux	12th Gen Intel(R) Core(TM) i7-12700KF	NVIDIA GeForce RTX 4070	27.42	Ubuntu 20.04.5 LTS	0.1.39	phi3:3.8b	127.27
Linux	12th Gen Intel(R) Core(TM) i7-12700KF	NVIDIA GeForce RTX 4070	27.42	Ubuntu 20.04.5 LTS	0.1.39	gemma:2b	158.96
Linux	12th Gen Intel(R) Core(TM) i7-12700KF	NVIDIA GeForce RTX 4070	27.42	Ubuntu 20.04.5 LTS	0.1.39	gemma:7b	70.93
Linux	12th Gen Intel(R) Core(TM) i7-12700KF	NVIDIA GeForce RTX 4070	27.42	Ubuntu 20.04.5 LTS	0.1.39	llava:7b	91.02
Linux	12th Gen Intel(R) Core(TM) i7-12700KF	NVIDIA GeForce RTX 4070	27.42	Ubuntu 20.04.5 LTS	0.1.39	llava:13b	53.82
Linux	12th Gen Intel(R) Core(TM) i7-12700KF	NVIDIA GeForce RTX 4070	27.42	Ubuntu 20.04.5 LTS	0.1.39	mistral:7b	87.9
Linux	12th Gen Intel(R) Core(TM) i7-12700KF	NVIDIA GeForce RTX 4070	27.42	Ubuntu 20.04.5 LTS	0.1.39	llama3:8b	77.96
Linux	12th Gen Intel(R) Core(TM) i7-12700KF	NVIDIA GeForce RTX 4070	27.42	Ubuntu 20.04.5 LTS	0.1.39	phi3:3.8b	126.87
Linux	12th Gen Intel(R) Core(TM) i7-12700KF	NVIDIA GeForce RTX 4070	27.42	Ubuntu 20.04.5 LTS	0.1.39	gemma:2b	159.62
Linux	12th Gen Intel(R) Core(TM) i7-12700KF	NVIDIA GeForce RTX 4070	27.42	Ubuntu 20.04.5 LTS	0.1.39	gemma:7b	71.03
Linux	12th Gen Intel(R) Core(TM) i7-12700KF	NVIDIA GeForce RTX 4070	27.42	Ubuntu 20.04.5 LTS	0.1.39	llava:7b	91.06
Linux	12th Gen Intel(R) Core(TM) i7-12700KF	NVIDIA GeForce RTX 4070	27.42	Ubuntu 20.04.5 LTS	0.1.39	llava:13b	53.87
Linux	12th Gen Intel(R) Core(TM) i7-12700KF	NVIDIA GeForce RTX 4070	27.42	Ubuntu 20.04.5 LTS	0.1.39	mistral:7b	79.13
Linux	12th Gen Intel(R) Core(TM) i7-12700KF	NVIDIA GeForce RTX 4070	27.42	Ubuntu 20.04.5 LTS	0.1.39	llama3:8b	67.67
Linux	12th Gen Intel(R) Core(TM) i7-12700KF	NVIDIA GeForce RTX 4070	27.42	Ubuntu 20.04.5 LTS	0.1.39	phi3:3.8b	113.29
Linux	12th Gen Intel(R) Core(TM) i7-12700KF	NVIDIA GeForce RTX 4070	27.42	Ubuntu 20.04.5 LTS	0.1.39	gemma:2b	126.41
Linux	12th Gen Intel(R) Core(TM) i7-12700KF	NVIDIA GeForce RTX 4070	27.42	Ubuntu 20.04.5 LTS	0.1.39	gemma:7b	61.52
Linux	12th Gen Intel(R) Core(TM) i7-12700KF	NVIDIA GeForce RTX 4070	27.42	Ubuntu 20.04.5 LTS	0.1.39	llava:7b	80.63
Linux	12th Gen Intel(R) Core(TM) i7-12700KF	NVIDIA GeForce RTX 4070	27.42	Ubuntu 20.04.5 LTS	0.1.39	llava:13b	50.43
Linux	Intel(R) Core(TM) i5-2500K CPU @ 3.30GHz	NVIDIA GeForce GTX 1070	15.57	Linux Mint 21.3	0.1.39	mistral:7b	39.26
Linux	Intel(R) Core(TM) i5-2500K CPU @ 3.30GHz	NVIDIA GeForce GTX 1070	15.57	Linux Mint 21.3	0.1.39	llama3:8b	34.15
Linux	Intel(R) Core(TM) i5-2500K CPU @ 3.30GHz	NVIDIA GeForce GTX 1070	15.57	Linux Mint 21.3	0.1.39	phi3:3.8b	51.23
Linux	Intel(R) Core(TM) i5-2500K CPU @ 3.30GHz	NVIDIA GeForce GTX 1070	15.57	Linux Mint 21.3	0.1.39	gemma:2b	85.63
Linux	Intel(R) Core(TM) i5-2500K CPU @ 3.30GHz	NVIDIA GeForce GTX 1070	15.57	Linux Mint 21.3	0.1.39	gemma:7b	31.16
Linux	Intel(R) Core(TM) i5-2500K CPU @ 3.30GHz	NVIDIA GeForce GTX 1070	15.57	Linux Mint 21.3	0.1.39	llava:7b	42.13
Linux	Intel(R) Core(TM) i5-2500K CPU @ 3.30GHz	NVIDIA GeForce GTX 1070	15.57	Linux Mint 21.3	0.1.39	llava:13b	7.31
Linux	Intel(R) Core(TM) i7-7700HQ CPU @ 2.80GHz	NVIDIA GeForce GTX 1050 Ti	15.49	Ubuntu 22.04.4 LTS	0.1.39	mistral:7b	8.35
Linux	Intel(R) Xeon(R) CPU E5-2690 v4 @ 2.60GHz	unknown	125.78	Ubuntu 22.04.4 LTS	0.1.32	mistral:7b	12.39
Linux	Intel(R) Xeon(R) CPU E5-2690 v4 @ 2.60GHz	unknown	125.78	Ubuntu 22.04.4 LTS	0.1.32	gemma:2b	26.36
Linux	Intel(R) Xeon(R) CPU E5-2690 v4 @ 2.60GHz	unknown	125.78	Ubuntu 22.04.4 LTS	0.1.32	gemma:7b	10.48
Linux	Intel(R) Xeon(R) CPU E5-2690 v4 @ 2.60GHz	unknown	125.78	Ubuntu 22.04.4 LTS	0.1.32	llama2:7b	13.09
Linux	Intel(R) Xeon(R) CPU E5-2690 v4 @ 2.60GHz	unknown	125.78	Ubuntu 22.04.4 LTS	0.1.32	llama2:13b	6.25
Linux	Intel(R) Xeon(R) CPU E5-2690 v4 @ 2.60GHz	unknown	125.78	Ubuntu 22.04.4 LTS	0.1.32	llava:7b	6.58
Linux	Intel(R) Xeon(R) CPU E5-2690 v4 @ 2.60GHz	unknown	125.78	Ubuntu 22.04.4 LTS	0.1.32	llava:13b	7.11
Linux	AMD Ryzen 7 7700X 8-Core Processor	Advanced Micro Devices, Inc. [AMD/ATI]	30.50	Ubuntu 22.04.4 LTS	0.0.0	mistral:7b	72.94
Linux	AMD Ryzen 7 7700X 8-Core Processor	Advanced Micro Devices, Inc. [AMD/ATI]	30.50	Ubuntu 22.04.4 LTS	0.0.0	gemma:2b	148.92
Linux	AMD Ryzen 7 7700X 8-Core Processor	Advanced Micro Devices, Inc. [AMD/ATI]	30.50	Ubuntu 22.04.4 LTS	0.0.0	gemma:7b	58.58
Linux	AMD Ryzen 7 7700X 8-Core Processor	Advanced Micro Devices, Inc. [AMD/ATI]	30.50	Ubuntu 22.04.4 LTS	0.0.0	llama2:7b	66.13
Linux	AMD Ryzen 7 7700X 8-Core Processor	Advanced Micro Devices, Inc. [AMD/ATI]	30.50	Ubuntu 22.04.4 LTS	0.0.0	llama2:13b	39.96
Linux	AMD Ryzen 7 7700X 8-Core Processor	Advanced Micro Devices, Inc. [AMD/ATI]	30.50	Ubuntu 22.04.4 LTS	0.0.0	llava:7b	76.53
Linux	AMD Ryzen 7 7700X 8-Core Processor	Advanced Micro Devices, Inc. [AMD/ATI]	30.50	Ubuntu 22.04.4 LTS	0.0.0	llava:13b	44
Linux	AMD Ryzen 5 5600X 6-Core Processor	Navi 31 [Radeon RX 7900 XT/7900 XTX/7900M]	62.71	Ubuntu 22.04.4 LTS	0.1.32	mistral:7b	75.18
Linux	AMD Ryzen 5 5600X 6-Core Processor	Navi 31 [Radeon RX 7900 XT/7900 XTX/7900M]	62.71	Ubuntu 22.04.4 LTS	0.1.32	gemma:2b	149.1
Linux	AMD Ryzen 5 5600X 6-Core Processor	Navi 31 [Radeon RX 7900 XT/7900 XTX/7900M]	62.71	Ubuntu 22.04.4 LTS	0.1.32	gemma:7b	64.56
Linux	AMD Ryzen 5 5600X 6-Core Processor	Navi 31 [Radeon RX 7900 XT/7900 XTX/7900M]	62.71	Ubuntu 22.04.4 LTS	0.1.32	llama2:7b	72.83
Linux	AMD Ryzen 5 5600X 6-Core Processor	Navi 31 [Radeon RX 7900 XT/7900 XTX/7900M]	62.71	Ubuntu 22.04.4 LTS	0.1.32	llama2:13b	43.25
Linux	AMD Ryzen 5 5600X 6-Core Processor	Navi 31 [Radeon RX 7900 XT/7900 XTX/7900M]	62.71	Ubuntu 22.04.4 LTS	0.1.32	llava:7b	77.14
Linux	AMD Ryzen 5 5600X 6-Core Processor	Navi 31 [Radeon RX 7900 XT/7900 XTX/7900M]	62.71	Ubuntu 22.04.4 LTS	0.1.32	llava:13b	44.32
Linux	AMD EPYC 7543P 32-Core Processor	NVIDIA RTX A5000	251.70	Ubuntu 20.04.6 LTS	0.1.32	gemma:2b	168.36
Linux	Intel(R) Core(TM) i5-14490F	NVIDIA GeForce RTX 4060 Ti	31.15	Ubuntu 22.04.4 LTS	0.1.32	llama3:8b-instruct-fp16	17.08
Linux	Intel(R) Core(TM) i5-14490F	NVIDIA GeForce RTX 4060 Ti	31.15	Ubuntu 22.04.4 LTS	0.1.32	llama3:8b	50.74
Linux	Intel(R) Core(TM) i5-14490F	NVIDIA GeForce RTX 4060 Ti	31.15	Ubuntu 22.04.4 LTS	0.1.32	llama3:8b-instruct-fp16	17.21
Linux	Intel(R) Core(TM) i5-14490F	NVIDIA GeForce RTX 4060 Ti	31.15	Ubuntu 22.04.4 LTS	0.1.32	llama3:8b	50.74
Linux	Intel(R) Core(TM) i5-14490F	NVIDIA GeForce RTX 4060 Ti	31.15	Ubuntu 22.04.4 LTS	0.1.32	llama3:8b-instruct-q6_K	38.03
Linux	Intel(R) Xeon(R) CPU D-1577 @ 1.30GHz	Quadro RTX 5000	94.22	Ubuntu 22.04.4 LTS	0.1.32	mistral:7b	63.66
Linux	Intel(R) Xeon(R) CPU D-1577 @ 1.30GHz	Quadro RTX 5000	94.22	Ubuntu 22.04.4 LTS	0.1.32	gemma:2b	98.85
Linux	Intel(R) Xeon(R) CPU D-1577 @ 1.30GHz	Quadro RTX 5000	94.22	Ubuntu 22.04.4 LTS	0.1.32	gemma:7b	49.73
Linux	Intel(R) Xeon(R) CPU D-1577 @ 1.30GHz	Quadro RTX 5000	94.22	Ubuntu 22.04.4 LTS	0.1.32	llama2:7b	65
Linux	Intel(R) Xeon(R) CPU D-1577 @ 1.30GHz	Quadro RTX 5000	94.22	Ubuntu 22.04.4 LTS	0.1.32	llama2:13b	39.21
Linux	Intel(R) Xeon(R) CPU D-1577 @ 1.30GHz	Quadro RTX 5000	94.22	Ubuntu 22.04.4 LTS	0.1.32	llava:7b	65.33
Linux	Intel(R) Xeon(R) CPU D-1577 @ 1.30GHz	Quadro RTX 5000	94.22	Ubuntu 22.04.4 LTS	0.1.32	llava:13b	39.86
Linux	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	Tesla T4	15.41	Ubuntu 22.04.4 LTS	0.1.31	mistral:7b	45.48
Linux	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	Tesla T4	15.41	Ubuntu 22.04.4 LTS	0.1.31	gemma:2b	86.17
Linux	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	Tesla T4	15.41	Ubuntu 22.04.4 LTS	0.1.31	gemma:7b	35.27
Linux	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	Tesla T4	15.41	Ubuntu 22.04.4 LTS	0.1.31	llama2:7b	46.95
Linux	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	Tesla T4	15.41	Ubuntu 22.04.4 LTS	0.1.31	llama2:13b	26.06
Linux	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	Tesla T4	15.41	Ubuntu 22.04.4 LTS	0.1.31	llava:7b	47.48
Linux	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	Tesla T4	15.41	Ubuntu 22.04.4 LTS	0.1.31	llava:13b	27.25
Linux	Intel(R) Core(TM) i5-14490F	NVIDIA GeForce RTX 4060 Ti	31.15	Ubuntu 22.04.4 LTS	0.1.32	llama3:8b-instruct-fp16	17.29
Linux	Intel(R) Core(TM) i5-14490F	NVIDIA GeForce RTX 4060 Ti	31.15	Ubuntu 22.04.4 LTS	0.1.32	llama3:8b	50.66
Linux	Intel(R) Core(TM) i5-14490F	NVIDIA GeForce RTX 4060 Ti	31.15	Ubuntu 22.04.4 LTS	0.1.32	llama3:8b-instruct-q6_K	37.92
Linux	Intel Corporation	Integrated Matrox G200eW3 Graphics Controller	1007.39	Ubuntu 22.04.4 LTS	0.1.32	mistral:7b	7.75
Linux	Intel Corporation	Integrated Matrox G200eW3 Graphics Controller	1007.39	Ubuntu 22.04.4 LTS	0.1.32	gemma:2b	13.83
Linux	Intel Corporation	Integrated Matrox G200eW3 Graphics Controller	1007.39	Ubuntu 22.04.4 LTS	0.1.32	gemma:7b	9.07
Linux	Intel Corporation	Integrated Matrox G200eW3 Graphics Controller	1007.39	Ubuntu 22.04.4 LTS	0.1.32	llama2:7b	8.75
Linux	Intel Corporation	Integrated Matrox G200eW3 Graphics Controller	1007.39	Ubuntu 22.04.4 LTS	0.1.32	llama2:13b	5.6
Linux	Intel Corporation	Integrated Matrox G200eW3 Graphics Controller	1007.39	Ubuntu 22.04.4 LTS	0.1.32	llava:7b	13.6
Linux	Intel Corporation	Integrated Matrox G200eW3 Graphics Controller	1007.39	Ubuntu 22.04.4 LTS	0.1.32	llava:13b	7.46
Linux	AMD EPYC 7543P 32-Core Processor	NVIDIA RTX A5000	251.70	Ubuntu 20.04.6 LTS	0.1.32	gemma:2b	166.67
Linux	AMD EPYC 7543P 32-Core Processor	NVIDIA RTX A5000	251.70	Ubuntu 20.04.6 LTS	0.1.32	gemma:2b	167.07
Linux	AMD EPYC 7543P 32-Core Processor	NVIDIA RTX A5000	251.70	Ubuntu 20.04.6 LTS	0.1.32	llama2:7b	107.7
Linux	Intel Corporation	Integrated Matrox G200eW3 Graphics Controller	1007.39	Ubuntu 22.04.4 LTS	0.1.32	mistral:7b	7.56
Linux	Intel Corporation	Integrated Matrox G200eW3 Graphics Controller	1007.39	Ubuntu 22.04.4 LTS	0.1.32	gemma:2b	24.4
Linux	Intel Corporation	Integrated Matrox G200eW3 Graphics Controller	1007.39	Ubuntu 22.04.4 LTS	0.1.32	gemma:7b	13.39
Linux	Intel Corporation	Integrated Matrox G200eW3 Graphics Controller	1007.39	Ubuntu 22.04.4 LTS	0.1.32	llama2:7b	9.74
Linux	Intel Corporation	Integrated Matrox G200eW3 Graphics Controller	1007.39	Ubuntu 22.04.4 LTS	0.1.32	llama2:13b	5.76
Linux	Intel Corporation	Integrated Matrox G200eW3 Graphics Controller	1007.39	Ubuntu 22.04.4 LTS	0.1.32	llava:7b	8.79
Linux	Intel Corporation	Integrated Matrox G200eW3 Graphics Controller	1007.39	Ubuntu 22.04.4 LTS	0.1.32	llava:13b	6.45
Linux	AMD EPYC 7543P 32-Core Processor	NVIDIA RTX A5000	251.70	Ubuntu 20.04.6 LTS	0.1.32	gemma:2b	165.57
Linux	AMD EPYC 7543P 32-Core Processor	NVIDIA RTX A5000	251.70	Ubuntu 20.04.6 LTS	0.1.32	llama2:7b	107.1
Linux	AMD EPYC 7543P 32-Core Processor	NVIDIA RTX A5000	251.70	Ubuntu 20.04.6 LTS	0.1.32	llama3:8b	90.89
Linux	13th Gen Intel(R) Core(TM) i7-13700K	NVIDIA GeForce RTX 4070 Ti	31.22	Ubuntu 22.04.4 LTS	0.1.32	mistral:7b	75.12
Linux	13th Gen Intel(R) Core(TM) i7-13700K	NVIDIA GeForce RTX 4070 Ti	31.22	Ubuntu 22.04.4 LTS	0.1.32	gemma:2b	126.27
Linux	13th Gen Intel(R) Core(TM) i7-13700K	NVIDIA GeForce RTX 4070 Ti	31.22	Ubuntu 22.04.4 LTS	0.1.32	gemma:7b	64.36
Linux	13th Gen Intel(R) Core(TM) i7-13700K	NVIDIA GeForce RTX 4070 Ti	31.22	Ubuntu 22.04.4 LTS	0.1.32	llama2:7b	82.06
Linux	13th Gen Intel(R) Core(TM) i7-13700K	NVIDIA GeForce RTX 4070 Ti	31.22	Ubuntu 22.04.4 LTS	0.1.32	llama2:13b	49.03
Linux	13th Gen Intel(R) Core(TM) i7-13700K	NVIDIA GeForce RTX 4070 Ti	31.22	Ubuntu 22.04.4 LTS	0.1.32	llava:7b	75.63
Linux	13th Gen Intel(R) Core(TM) i7-13700K	NVIDIA GeForce RTX 4070 Ti	31.22	Ubuntu 22.04.4 LTS	0.1.32	llava:13b	49.28
Linux	AMD EPYC 7543P 32-Core Processor	NVIDIA RTX A5000	251.70	Ubuntu 20.04.6 LTS	0.1.32	gemma:2b	159.93
Linux	AMD EPYC 7543P 32-Core Processor	NVIDIA RTX A5000	251.70	Ubuntu 20.04.6 LTS	0.1.32	llama2:7b	104.5
Linux	AMD EPYC 7543P 32-Core Processor	NVIDIA RTX A5000	251.70	Ubuntu 20.04.6 LTS	0.1.32	llama3:8b	90.31
Linux	12th Gen Intel(R) Core(TM) i5-12500	NVIDIA GeForce RTX 4060 Ti	62.78	Debian GNU/Linux 12 (bookworm)	0.1.32	mistral:7b	56.19
Linux	12th Gen Intel(R) Core(TM) i5-12500	NVIDIA GeForce RTX 4060 Ti	62.78	Debian GNU/Linux 12 (bookworm)	0.1.32	gemma:2b	109.9
Linux	12th Gen Intel(R) Core(TM) i5-12500	NVIDIA GeForce RTX 4060 Ti	62.78	Debian GNU/Linux 12 (bookworm)	0.1.32	gemma:7b	45
Linux	12th Gen Intel(R) Core(TM) i5-12500	NVIDIA GeForce RTX 4060 Ti	62.78	Debian GNU/Linux 12 (bookworm)	0.1.32	llama2:7b	58.19
Linux	12th Gen Intel(R) Core(TM) i5-12500	NVIDIA GeForce RTX 4060 Ti	62.78	Debian GNU/Linux 12 (bookworm)	0.1.32	llama2:13b	32.47
Linux	12th Gen Intel(R) Core(TM) i5-12500	NVIDIA GeForce RTX 4060 Ti	62.78	Debian GNU/Linux 12 (bookworm)	0.1.32	llava:7b	57.26
Linux	12th Gen Intel(R) Core(TM) i5-12500	NVIDIA GeForce RTX 4060 Ti	62.78	Debian GNU/Linux 12 (bookworm)	0.1.32	llava:13b	33.04
Linux	12th Gen Intel(R) Core(TM) i5-12500	NVIDIA GeForce RTX 4060 Ti	62.78	Debian GNU/Linux 12 (bookworm)	0.1.32	mistral:7b	64.29
Linux	12th Gen Intel(R) Core(TM) i5-12500	NVIDIA GeForce RTX 4060 Ti	62.78	Debian GNU/Linux 12 (bookworm)	0.1.32	gemma:2b	122.19
Linux	12th Gen Intel(R) Core(TM) i5-12500	NVIDIA GeForce RTX 4060 Ti	62.78	Debian GNU/Linux 12 (bookworm)	0.1.32	gemma:7b	51.41
Linux	12th Gen Intel(R) Core(TM) i5-12500	NVIDIA GeForce RTX 4060 Ti	62.78	Debian GNU/Linux 12 (bookworm)	0.1.32	llama2:7b	66.98
Linux	12th Gen Intel(R) Core(TM) i5-12500	NVIDIA GeForce RTX 4060 Ti	62.78	Debian GNU/Linux 12 (bookworm)	0.1.32	llama2:13b	37.58
Linux	12th Gen Intel(R) Core(TM) i5-12500	NVIDIA GeForce RTX 4060 Ti	62.78	Debian GNU/Linux 12 (bookworm)	0.1.32	llava:7b	65.51
Linux	12th Gen Intel(R) Core(TM) i5-12500	NVIDIA GeForce RTX 4060 Ti	62.78	Debian GNU/Linux 12 (bookworm)	0.1.32	llava:13b	38.16
Linux	AMD Ryzen 5 PRO 4650G with Radeon Graphics	no_gpu	30.77	Debian GNU/Linux 11 (bullseye)	0.1.39	mistral:7b	5.17
Linux	AMD Ryzen 5 PRO 4650G with Radeon Graphics	no_gpu	30.77	Debian GNU/Linux 11 (bullseye)	0.1.39	llama3:8b	4.78
Linux	Intel(R) Xeon(R) CPU D-1577 @ 1.30GHz	Quadro RTX 5000	94.22	Ubuntu 22.04.4 LTS	0.1.32	gemma:2b	100.21
Linux	Intel(R) Xeon(R) CPU D-1577 @ 1.30GHz	Quadro RTX 5000	94.22	Ubuntu 22.04.4 LTS	0.1.32	llama2:7b	65.73
Linux	Intel(R) Xeon(R) CPU D-1577 @ 1.30GHz	Quadro RTX 5000	94.22	Ubuntu 22.04.4 LTS	0.1.32	llama3:8b	51.12
Linux	Intel(R) Xeon(R) CPU D-1577 @ 1.30GHz	Quadro RTX 5000	94.22	Ubuntu 22.04.4 LTS	0.1.32	mistral:7b	63.95
Linux	Intel(R) Xeon(R) CPU D-1577 @ 1.30GHz	Quadro RTX 5000	94.22	Ubuntu 22.04.4 LTS	0.1.32	gemma:2b	98.65
Linux	Intel(R) Xeon(R) CPU D-1577 @ 1.30GHz	Quadro RTX 5000	94.22	Ubuntu 22.04.4 LTS	0.1.32	gemma:7b	49.89
Linux	Intel(R) Xeon(R) CPU D-1577 @ 1.30GHz	Quadro RTX 5000	94.22	Ubuntu 22.04.4 LTS	0.1.32	llama2:7b	65.36
Linux	Intel(R) Xeon(R) CPU D-1577 @ 1.30GHz	Quadro RTX 5000	94.22	Ubuntu 22.04.4 LTS	0.1.32	llama2:13b	39.4
Linux	Intel(R) Xeon(R) CPU D-1577 @ 1.30GHz	Quadro RTX 5000	94.22	Ubuntu 22.04.4 LTS	0.1.32	llama3:8b	51.01
Linux	Intel(R) Xeon(R) CPU D-1577 @ 1.30GHz	Quadro RTX 5000	94.22	Ubuntu 22.04.4 LTS	0.1.32	llava:7b	65.4
Linux	Intel(R) Xeon(R) CPU D-1577 @ 1.30GHz	Quadro RTX 5000	94.22	Ubuntu 22.04.4 LTS	0.1.32	llava:13b	40.64
Linux	AMD EPYC 7543P 32-Core Processor	NVIDIA RTX A5000	251.70	Ubuntu 20.04.6 LTS	0.1.32	mistral:7b	104.38
Linux	AMD EPYC 7543P 32-Core Processor	NVIDIA RTX A5000	251.70	Ubuntu 20.04.6 LTS	0.1.32	gemma:2b	167.79
Linux	AMD EPYC 7543P 32-Core Processor	NVIDIA RTX A5000	251.70	Ubuntu 20.04.6 LTS	0.1.32	gemma:7b	84.13
Linux	AMD EPYC 7543P 32-Core Processor	NVIDIA RTX A5000	251.70	Ubuntu 20.04.6 LTS	0.1.32	llama2:7b	106.51
Linux	AMD EPYC 7543P 32-Core Processor	NVIDIA RTX A5000	251.70	Ubuntu 20.04.6 LTS	0.1.32	llama2:13b	64.7
Linux	AMD EPYC 7543P 32-Core Processor	NVIDIA RTX A5000	251.70	Ubuntu 20.04.6 LTS	0.1.32	llama3:8b	90.98
Linux	AMD EPYC 7543P 32-Core Processor	NVIDIA RTX A5000	251.70	Ubuntu 20.04.6 LTS	0.1.32	llava:7b	107.58
Linux	AMD EPYC 7543P 32-Core Processor	NVIDIA RTX A5000	251.70	Ubuntu 20.04.6 LTS	0.1.32	llava:13b	67.34
Linux	Intel Corporation	Integrated Matrox G200eW3 Graphics Controller	1007.38	Ubuntu 22.04.4 LTS	0.1.32	mistral:7b	5.26
Linux	Intel Corporation	Integrated Matrox G200eW3 Graphics Controller	1007.38	Ubuntu 22.04.4 LTS	0.1.32	gemma:2b	3.71
Linux	Intel Corporation	Integrated Matrox G200eW3 Graphics Controller	1007.38	Ubuntu 22.04.4 LTS	0.1.32	gemma:7b	5.89
Linux	Intel Corporation	Integrated Matrox G200eW3 Graphics Controller	1007.38	Ubuntu 22.04.4 LTS	0.1.32	llama2:7b	3.06
Linux	Intel(R) Xeon(R) CPU D-1577 @ 1.30GHz	Quadro RTX 5000	94.22	Ubuntu 22.04.4 LTS	0.1.32	phi3:3.8b	85.72
Linux	AMD EPYC 7543P 32-Core Processor	NVIDIA RTX A5000	251.70	Ubuntu 20.04.6 LTS	0.1.32	phi3:3.8b	142.14
Linux	Intel Corporation	Integrated Matrox G200eW3 Graphics Controller	1007.38	Ubuntu 22.04.4 LTS	0.1.32	llama2:13b	5.05
Linux	Intel Corporation	Integrated Matrox G200eW3 Graphics Controller	1007.38	Ubuntu 22.04.4 LTS	0.1.32	llava:7b	6.37
Linux	Intel Corporation	Integrated Matrox G200eW3 Graphics Controller	1007.38	Ubuntu 22.04.4 LTS	0.1.32	llava:13b	5.34
Linux	AMD EPYC 7413 24-Core Processor	NVIDIA L4	47.04	Ubuntu 22.04.4 LTS	0.1.32	mistral:7b	51.44
Linux	AMD EPYC 7413 24-Core Processor	NVIDIA L4	47.04	Ubuntu 22.04.4 LTS	0.1.32	llama3:8b	46.3
Linux	AMD EPYC 7413 24-Core Processor	NVIDIA L4	47.04	Ubuntu 22.04.4 LTS	0.1.32	phi3:3.8b	75.24
Linux	AMD EPYC 7413 24-Core Processor	NVIDIA L4	47.04	Ubuntu 22.04.4 LTS	0.1.32	gemma:2b	100.51
Linux	AMD EPYC 7413 24-Core Processor	NVIDIA L4	47.04	Ubuntu 22.04.4 LTS	0.1.32	gemma:7b	41.72
Linux	AMD EPYC 7413 24-Core Processor	NVIDIA L4	47.04	Ubuntu 22.04.4 LTS	0.1.32	llava:7b	52.79
Linux	AMD EPYC 7413 24-Core Processor	NVIDIA L4	47.04	Ubuntu 22.04.4 LTS	0.1.32	llava:13b	30.22
Linux	Intel(R) Core(TM) i7-7700 CPU @ 3.60GHz	NVIDIA GeForce RTX 3070 Ti	31.23	Ubuntu 22.04.3 LTS	0.1.32	mistral:7b	89.58
Linux	Intel(R) Core(TM) i7-7700 CPU @ 3.60GHz	NVIDIA GeForce RTX 3070 Ti	31.23	Ubuntu 22.04.3 LTS	0.1.32	llama3:8b	74.43
Linux	Intel(R) Core(TM) i7-7700 CPU @ 3.60GHz	NVIDIA GeForce RTX 3070 Ti	31.23	Ubuntu 22.04.3 LTS	0.1.32	phi3:3.8b	120.59
Linux	AMD Ryzen 9 7950X 16-Core Processor	Advanced Micro Devices, Inc. [AMD/ATI]	61.98	Ubuntu 22.04.4 LTS	0.1.30	mistral:7b	97.4
Linux	AMD Ryzen 9 7950X 16-Core Processor	Advanced Micro Devices, Inc. [AMD/ATI]	61.98	Ubuntu 22.04.4 LTS	0.1.30	llama3:8b	90.05
Linux	AMD Ryzen 9 7950X 16-Core Processor	Advanced Micro Devices, Inc. [AMD/ATI]	61.98	Ubuntu 22.04.4 LTS	0.1.30	phi3:3.8b	106.06
Linux	AMD Ryzen 9 7950X 16-Core Processor	Advanced Micro Devices, Inc. [AMD/ATI]	61.98	Ubuntu 22.04.4 LTS	0.1.30	gemma:2b	185.57
Linux	AMD Ryzen 9 7950X 16-Core Processor	Advanced Micro Devices, Inc. [AMD/ATI]	61.98	Ubuntu 22.04.4 LTS	0.1.30	gemma:7b	86.6
Linux	AMD Ryzen 9 7950X 16-Core Processor	Advanced Micro Devices, Inc. [AMD/ATI]	61.98	Ubuntu 22.04.4 LTS	0.1.30	llava:7b	101.43
Linux	AMD Ryzen 9 7950X 16-Core Processor	Advanced Micro Devices, Inc. [AMD/ATI]	61.98	Ubuntu 22.04.4 LTS	0.1.30	llava:13b	63.12
Linux	Intel(R) Xeon(R) Platinum 8468	NVIDIA H100 80GB HBM3	2015.37	Ubuntu 22.04.4 LTS	0.1.33	mistral:7b	55.41
Linux	Intel(R) Xeon(R) Platinum 8468	NVIDIA H100 80GB HBM3	2015.37	Ubuntu 22.04.4 LTS	0.1.33	llama3:8b	48.45
Linux	Intel(R) Xeon(R) Platinum 8468	NVIDIA H100 80GB HBM3	2015.37	Ubuntu 22.04.4 LTS	0.1.33	phi3:3.8b	64.32
Linux	Intel(R) Xeon(R) Platinum 8468	NVIDIA H100 80GB HBM3	2015.37	Ubuntu 22.04.4 LTS	0.1.33	gemma:2b	93.12
Linux	Intel(R) Xeon(R) Platinum 8468	NVIDIA H100 80GB HBM3	2015.37	Ubuntu 22.04.4 LTS	0.1.33	gemma:7b	43.43
Linux	Intel(R) Xeon(R) Platinum 8468	NVIDIA H100 80GB HBM3	2015.37	Ubuntu 22.04.4 LTS	0.1.33	llava:7b	58.62
Linux	Intel(R) Xeon(R) Platinum 8468	NVIDIA H100 80GB HBM3	2015.37	Ubuntu 22.04.4 LTS	0.1.33	llava:13b	35.53
Linux	Intel(R) Xeon(R) Platinum 8468	NVIDIA H100 80GB HBM3	2015.37	Ubuntu 22.04.4 LTS	0.1.33	mistral:7b	79.02
Linux	Intel(R) Xeon(R) Platinum 8468	NVIDIA H100 80GB HBM3	2015.37	Ubuntu 22.04.4 LTS	0.1.33	llama3:8b	69.53
Linux	Intel(R) Xeon(R) Platinum 8468	NVIDIA H100 80GB HBM3	2015.37	Ubuntu 22.04.4 LTS	0.1.33	phi3:3.8b	87.89
Linux	Intel(R) Xeon(R) Platinum 8468	NVIDIA H100 80GB HBM3	2015.37	Ubuntu 22.04.4 LTS	0.1.33	gemma:2b	123.68
Linux	Intel(R) Xeon(R) Platinum 8468	NVIDIA H100 80GB HBM3	2015.37	Ubuntu 22.04.4 LTS	0.1.33	gemma:7b	62.55
Linux	Intel(R) Xeon(R) Platinum 8468	NVIDIA H100 80GB HBM3	2015.37	Ubuntu 22.04.4 LTS	0.1.33	llava:7b	81.74
Linux	Intel(R) Xeon(R) Platinum 8468	NVIDIA H100 80GB HBM3	2015.37	Ubuntu 22.04.4 LTS	0.1.33	llava:13b	53.8
Linux	13th Gen Intel(R) Core(TM) i5-13600K	Raptor Lake-S GT1 [UHD Graphics 770]	31.06	KDE neon 6.0	0.1.28	mistral:7b	19.93
Linux	13th Gen Intel(R) Core(TM) i5-13600K	Raptor Lake-S GT1 [UHD Graphics 770]	31.06	KDE neon 6.0	0.1.28	gemma:2b	28.98
Linux	13th Gen Intel(R) Core(TM) i5-13600K	Raptor Lake-S GT1 [UHD Graphics 770]	31.06	KDE neon 6.0	0.1.28	gemma:7b	19.5
Linux	13th Gen Intel(R) Core(TM) i5-13600K	Raptor Lake-S GT1 [UHD Graphics 770]	31.06	KDE neon 6.0	0.1.28	llama2:7b	20.07
Linux	13th Gen Intel(R) Core(TM) i5-13600K	Raptor Lake-S GT1 [UHD Graphics 770]	31.06	KDE neon 6.0	0.1.28	llama2:13b	13.1
Linux	13th Gen Intel(R) Core(TM) i5-13600K	Raptor Lake-S GT1 [UHD Graphics 770]	31.06	KDE neon 6.0	0.1.28	llava:7b	17.27
Linux	13th Gen Intel(R) Core(TM) i5-13600K	Raptor Lake-S GT1 [UHD Graphics 770]	31.06	KDE neon 6.0	0.1.28	llava:13b	14.91
Linux	11th Gen Intel(R) Core(TM) i9-11950H @ 2.60GHz	NVIDIA GeForce RTX 3080 Laptop GPU	15.48	Ubuntu 22.04.3 LTS	0.1.33	mistral:7b	43.33
Linux	11th Gen Intel(R) Core(TM) i9-11950H @ 2.60GHz	NVIDIA GeForce RTX 3080 Laptop GPU	15.48	Ubuntu 22.04.3 LTS	0.1.33	llama3:8b	37.69
Linux	11th Gen Intel(R) Core(TM) i9-11950H @ 2.60GHz	NVIDIA GeForce RTX 3080 Laptop GPU	15.48	Ubuntu 22.04.3 LTS	0.1.33	phi3:3.8b	51.21
Linux	11th Gen Intel(R) Core(TM) i9-11950H @ 2.60GHz	NVIDIA GeForce RTX 3080 Laptop GPU	15.48	Ubuntu 22.04.3 LTS	0.1.33	gemma:2b	69.89
Linux	11th Gen Intel(R) Core(TM) i9-11950H @ 2.60GHz	NVIDIA GeForce RTX 3080 Laptop GPU	15.48	Ubuntu 22.04.3 LTS	0.1.33	gemma:7b	33.78
Linux	11th Gen Intel(R) Core(TM) i9-11950H @ 2.60GHz	NVIDIA GeForce RTX 3080 Laptop GPU	15.48	Ubuntu 22.04.3 LTS	0.1.33	llava:7b	40.26
Linux	11th Gen Intel(R) Core(TM) i9-11950H @ 2.60GHz	NVIDIA GeForce RTX 3080 Laptop GPU	15.48	Ubuntu 22.04.3 LTS	0.1.33	llava:13b	27.36
Linux	AMD Ryzen 9 5950X 16-Core Processor	NVIDIA GeForce RTX 3090	62.75	Ubuntu 22.04.3 LTS	0.1.33	mistral:7b	68.11
Linux	AMD Ryzen 9 5950X 16-Core Processor	NVIDIA GeForce RTX 3090	62.75	Ubuntu 22.04.3 LTS	0.1.33	llama3:8b	63.42
Linux	AMD Ryzen 9 5950X 16-Core Processor	NVIDIA GeForce RTX 3090	62.75	Ubuntu 22.04.3 LTS	0.1.33	phi3:3.8b	72.59
Linux	AMD Ryzen 9 5950X 16-Core Processor	NVIDIA GeForce RTX 3090	62.75	Ubuntu 22.04.3 LTS	0.1.33	gemma:2b	112.42
Linux	AMD Ryzen 9 5950X 16-Core Processor	NVIDIA GeForce RTX 3090	62.75	Ubuntu 22.04.3 LTS	0.1.33	gemma:7b	68.67
Linux	AMD Ryzen 9 5950X 16-Core Processor	NVIDIA GeForce RTX 3090	62.75	Ubuntu 22.04.3 LTS	0.1.33	llava:7b	66.17
Linux	AMD Ryzen 9 5950X 16-Core Processor	NVIDIA GeForce RTX 3090	62.75	Ubuntu 22.04.3 LTS	0.1.33	llava:13b	56.23
Linux	AMD Ryzen 7 5800H with Radeon Graphics	NVIDIA GeForce RTX 3070 Laptop GPU	27.25	Ubuntu 22.04.4 LTS	0.1.32	mistral:7b	65.45
Linux	AMD Ryzen 7 5800H with Radeon Graphics	NVIDIA GeForce RTX 3070 Laptop GPU	27.25	Ubuntu 22.04.4 LTS	0.1.32	llama3:8b	59.84
Linux	AMD Ryzen 7 5800H with Radeon Graphics	NVIDIA GeForce RTX 3070 Laptop GPU	27.25	Ubuntu 22.04.4 LTS	0.1.32	phi3:3.8b	89
Linux	AMD Ryzen 7 5800H with Radeon Graphics	NVIDIA GeForce RTX 3070 Laptop GPU	27.25	Ubuntu 22.04.4 LTS	0.1.32	gemma:2b	121.72
Linux	AMD Ryzen 7 5800H with Radeon Graphics	NVIDIA GeForce RTX 3070 Laptop GPU	27.25	Ubuntu 22.04.4 LTS	0.1.32	gemma:7b	52.19
Linux	AMD Ryzen 7 5800H with Radeon Graphics	NVIDIA GeForce RTX 3070 Laptop GPU	27.25	Ubuntu 22.04.4 LTS	0.1.32	llava:7b	65.85
Linux	AMD Ryzen 7 5800H with Radeon Graphics	NVIDIA GeForce RTX 3070 Laptop GPU	27.25	Ubuntu 22.04.4 LTS	0.1.32	llava:13b	8.97
Linux	AMD Ryzen 9 3950X 16-Core Processor	NVIDIA GeForce RTX 3090 Ti	125.70	Ubuntu 22.04.4 LTS	0.1.33	mistral:7b	121.91
Linux	AMD Ryzen 9 3950X 16-Core Processor	NVIDIA GeForce RTX 3090 Ti	125.70	Ubuntu 22.04.4 LTS	0.1.33	llama3:8b	100.27
Linux	AMD Ryzen 9 3950X 16-Core Processor	NVIDIA GeForce RTX 3090 Ti	125.70	Ubuntu 22.04.4 LTS	0.1.33	phi3:3.8b	157.91
Linux	AMD Ryzen 9 3950X 16-Core Processor	NVIDIA GeForce RTX 3090 Ti	125.70	Ubuntu 22.04.4 LTS	0.1.33	gemma:2b	183.89
Linux	AMD Ryzen 9 3950X 16-Core Processor	NVIDIA GeForce RTX 3090 Ti	125.70	Ubuntu 22.04.4 LTS	0.1.33	gemma:7b	98.07
Linux	AMD Ryzen 9 3950X 16-Core Processor	NVIDIA GeForce RTX 3090 Ti	125.70	Ubuntu 22.04.4 LTS	0.1.33	llava:7b	124.79
Linux	AMD Ryzen 9 3950X 16-Core Processor	NVIDIA GeForce RTX 3090 Ti	125.70	Ubuntu 22.04.4 LTS	0.1.33	llava:13b	80.04
Linux	AMD Ryzen 9 3950X 16-Core Processor	NVIDIA GeForce RTX 3090 Ti	125.70	Ubuntu 22.04.4 LTS	0.1.33	llama3:8b	101.01
Linux	AMD Ryzen 7 PRO 4750U with Radeon Graphics	Basic Render Driver	10.56	Ubuntu 24.04 LTS	0.1.33	mistral:7b	4.27
Linux	AMD Ryzen 7 PRO 4750U with Radeon Graphics	Basic Render Driver	10.56	Ubuntu 24.04 LTS	0.1.33	llama3:8b	4.16
Linux	AMD Ryzen 7 PRO 4750U with Radeon Graphics	Basic Render Driver	10.56	Ubuntu 24.04 LTS	0.1.33	phi3:3.8b	7.41
Linux	AMD Ryzen 7 PRO 4750U with Radeon Graphics	Basic Render Driver	10.56	Ubuntu 24.04 LTS	0.1.33	gemma:2b	9.7
Linux	AMD Ryzen 7 PRO 4750U with Radeon Graphics	Basic Render Driver	10.56	Ubuntu 24.04 LTS	0.1.33	gemma:7b	3.5
Linux	AMD Ryzen 7 PRO 4750U with Radeon Graphics	Basic Render Driver	10.56	Ubuntu 24.04 LTS	0.1.33	llava:7b	4.56
Linux	Intel(R) Core(TM) i3-7100 CPU @ 3.90GHz	NVIDIA GeForce RTX 4060 Ti	11.66	Ubuntu 24.04 LTS	0.1.33	mistral:7b	48.29
Linux	Intel(R) Core(TM) i3-7100 CPU @ 3.90GHz	NVIDIA GeForce RTX 4060 Ti	11.66	Ubuntu 24.04 LTS	0.1.33	llama3:8b	41.67
Linux	Intel(R) Core(TM) i3-7100 CPU @ 3.90GHz	NVIDIA GeForce RTX 4060 Ti	11.66	Ubuntu 24.04 LTS	0.1.33	phi3:3.8b	70.94
Linux	Intel(R) Core(TM) i3-7100 CPU @ 3.90GHz	NVIDIA GeForce RTX 4060 Ti	11.66	Ubuntu 24.04 LTS	0.1.33	gemma:2b	89.01
Linux	Intel(R) Core(TM) i3-7100 CPU @ 3.90GHz	NVIDIA GeForce RTX 4060 Ti	11.66	Ubuntu 24.04 LTS	0.1.33	gemma:7b	39.35
Linux	Intel(R) Core(TM) i3-7100 CPU @ 3.90GHz	NVIDIA GeForce RTX 4060 Ti	11.66	Ubuntu 24.04 LTS	0.1.33	llava:7b	49.41
Linux	AMD Ryzen 7 5800X 8-Core Processor	Navi 31 [Radeon RX 7900 XT/7900 XTX/7900M]	31.24	Ubuntu 24.04 LTS	0.1.33	mistral:7b	29.58
Linux	AMD Ryzen 7 5800X 8-Core Processor	Navi 31 [Radeon RX 7900 XT/7900 XTX/7900M]	31.24	Ubuntu 24.04 LTS	0.1.33	llama3:8b	28.81
Linux	AMD Ryzen 7 5800X 8-Core Processor	Navi 31 [Radeon RX 7900 XT/7900 XTX/7900M]	31.24	Ubuntu 24.04 LTS	0.1.33	phi3:3.8b	31.63
Linux	AMD Ryzen 7 5800X 8-Core Processor	Navi 31 [Radeon RX 7900 XT/7900 XTX/7900M]	31.24	Ubuntu 24.04 LTS	0.1.33	gemma:2b	51.62
Linux	AMD Ryzen 7 5800X 8-Core Processor	Navi 31 [Radeon RX 7900 XT/7900 XTX/7900M]	31.24	Ubuntu 24.04 LTS	0.1.33	gemma:7b	30.45
Linux	AMD Ryzen 7 5800X 8-Core Processor	Navi 31 [Radeon RX 7900 XT/7900 XTX/7900M]	31.24	Ubuntu 24.04 LTS	0.1.33	llava:7b	30.9
Linux	AMD Ryzen 7 5800X 8-Core Processor	Navi 31 [Radeon RX 7900 XT/7900 XTX/7900M]	31.24	Ubuntu 24.04 LTS	0.1.33	llava:13b	22.24
Linux	AMD Ryzen 7 5800X 8-Core Processor	Navi 31 [Radeon RX 7900 XT/7900 XTX/7900M]	31.24	Ubuntu 24.04 LTS	0.1.33	mistral:7b	75.07
Linux	AMD Ryzen 7 5800X 8-Core Processor	Navi 31 [Radeon RX 7900 XT/7900 XTX/7900M]	31.24	Ubuntu 24.04 LTS	0.1.33	llama3:8b	69.98
Linux	AMD Ryzen 7 5800X 8-Core Processor	Navi 31 [Radeon RX 7900 XT/7900 XTX/7900M]	31.24	Ubuntu 24.04 LTS	0.1.33	phi3:3.8b	82.4
Linux	AMD Ryzen 7 5800X 8-Core Processor	Navi 31 [Radeon RX 7900 XT/7900 XTX/7900M]	31.24	Ubuntu 24.04 LTS	0.1.33	gemma:2b	146.84
Linux	AMD Ryzen 7 5800X 8-Core Processor	Navi 31 [Radeon RX 7900 XT/7900 XTX/7900M]	31.24	Ubuntu 24.04 LTS	0.1.33	gemma:7b	63.92
Linux	AMD Ryzen 7 5800X 8-Core Processor	Navi 31 [Radeon RX 7900 XT/7900 XTX/7900M]	31.24	Ubuntu 24.04 LTS	0.1.33	llava:7b	77.69
Linux	AMD Ryzen 7 5800X 8-Core Processor	Navi 31 [Radeon RX 7900 XT/7900 XTX/7900M]	31.24	Ubuntu 24.04 LTS	0.1.33	llava:13b	46.31
Linux	AMD Ryzen 9 3900X 12-Core Processor	NVIDIA GeForce GTX 1650	125.80	Ubuntu 20.04.6 LTS	0.1.32	mistral:7b	12.41
Linux	AMD Ryzen 9 3900X 12-Core Processor	NVIDIA GeForce GTX 1650	125.80	Ubuntu 20.04.6 LTS	0.1.32	llama3:8b	10
Linux	AMD Ryzen 9 3900X 12-Core Processor	NVIDIA GeForce GTX 1650	125.80	Ubuntu 20.04.6 LTS	0.1.32	phi3:3.8b	20.55
Linux	AMD Ryzen 9 3900X 12-Core Processor	NVIDIA GeForce GTX 1650	125.80	Ubuntu 20.04.6 LTS	0.1.32	gemma:2b	44.93
Linux	AMD Ryzen 9 3900X 12-Core Processor	NVIDIA GeForce GTX 1650	125.80	Ubuntu 20.04.6 LTS	0.1.32	gemma:7b	7.98
Linux	AMD Ryzen 9 3900X 12-Core Processor	NVIDIA GeForce GTX 1650	125.80	Ubuntu 20.04.6 LTS	0.1.32	llava:7b	2.22
Linux	AMD Ryzen 9 3900X 12-Core Processor	NVIDIA GeForce GTX 1650	125.80	Ubuntu 20.04.6 LTS	0.1.32	llava:13b	5.53
Linux	AMD Ryzen 9 3900X 12-Core Processor	NVIDIA GeForce GTX 1650	125.80	Ubuntu 20.04.6 LTS	0.1.33	mistral:7b	12.28
Linux	AMD Ryzen 9 3900X 12-Core Processor	NVIDIA GeForce GTX 1650	125.80	Ubuntu 20.04.6 LTS	0.1.33	llama3:8b	10
Linux	AMD Ryzen 9 3900X 12-Core Processor	NVIDIA GeForce GTX 1650	125.80	Ubuntu 20.04.6 LTS	0.1.33	phi3:3.8b	21.54
Linux	AMD Ryzen 9 3900X 12-Core Processor	NVIDIA GeForce GTX 1650	125.80	Ubuntu 20.04.6 LTS	0.1.33	gemma:2b	47.48
Linux	AMD Ryzen 9 3900X 12-Core Processor	NVIDIA GeForce GTX 1650	125.80	Ubuntu 20.04.6 LTS	0.1.33	gemma:7b	7.97
Linux	AMD Ryzen 9 3900X 12-Core Processor	NVIDIA GeForce GTX 1650	125.80	Ubuntu 20.04.6 LTS	0.1.33	llava:7b	12.97
Linux	AMD Ryzen 9 3900X 12-Core Processor	NVIDIA GeForce GTX 1650	125.80	Ubuntu 20.04.6 LTS	0.1.33	llava:13b	5.44
Linux	AMD Ryzen 5 5600X 6-Core Processor	NVIDIA GeForce RTX 2080 Ti	15.53	Ubuntu 22.04.4 LTS	0.1.31	mistral:7b	92.72
Linux	AMD Ryzen 5 5600X 6-Core Processor	NVIDIA GeForce RTX 2080 Ti	15.53	Ubuntu 22.04.4 LTS	0.1.31	llama3:8b	82
Linux	AMD Ryzen 5 5600X 6-Core Processor	NVIDIA GeForce RTX 2080 Ti	15.53	Ubuntu 22.04.4 LTS	0.1.31	phi3:3.8b	124.91
Linux	AMD Ryzen 5 5600X 6-Core Processor	NVIDIA GeForce RTX 2080 Ti	15.53	Ubuntu 22.04.4 LTS	0.1.31	gemma:2b	147.45
Linux	AMD Ryzen 5 5600X 6-Core Processor	NVIDIA GeForce RTX 2080 Ti	15.53	Ubuntu 22.04.4 LTS	0.1.31	gemma:7b	73.5
Linux	AMD Ryzen 5 5600X 6-Core Processor	NVIDIA GeForce RTX 2080 Ti	15.53	Ubuntu 22.04.4 LTS	0.1.31	llava:7b	95.73
Linux	AMD Ryzen 5 5600X 6-Core Processor	NVIDIA GeForce RTX 2080 Ti	15.53	Ubuntu 22.04.4 LTS	0.1.31	llava:13b	58.91
Linux	Intel Corporation	Integrated Matrox G200eW3 Graphics Controller	1007.35	Ubuntu 22.04.4 LTS	0.1.33	mistral:7b	18.14
Linux	Intel Corporation	Integrated Matrox G200eW3 Graphics Controller	1007.35	Ubuntu 22.04.4 LTS	0.1.33	llama3:8b	17.38
Linux	Intel Corporation	Integrated Matrox G200eW3 Graphics Controller	1007.35	Ubuntu 22.04.4 LTS	0.1.33	phi3:3.8b	20.97
Linux	Intel Corporation	Integrated Matrox G200eW3 Graphics Controller	1007.35	Ubuntu 22.04.4 LTS	0.1.33	gemma:2b	32.24
Linux	Intel Corporation	Integrated Matrox G200eW3 Graphics Controller	1007.35	Ubuntu 22.04.4 LTS	0.1.33	gemma:7b	17.01
Linux	Intel Corporation	Integrated Matrox G200eW3 Graphics Controller	1007.35	Ubuntu 22.04.4 LTS	0.1.33	llava:7b	18.29
Linux	Intel Corporation	Integrated Matrox G200eW3 Graphics Controller	1007.35	Ubuntu 22.04.4 LTS	0.1.33	llava:13b	12.49
Linux	Intel Corporation	Integrated Matrox G200eW3 Graphics Controller	1007.38	Ubuntu 22.04.4 LTS	0.1.32	mistral:7b	4.6
Linux	Intel Corporation	Integrated Matrox G200eW3 Graphics Controller	1007.38	Ubuntu 22.04.4 LTS	0.1.32	llama3:8b	6.05
Linux	Intel Corporation	Integrated Matrox G200eW3 Graphics Controller	1007.38	Ubuntu 22.04.4 LTS	0.1.32	phi3:3.8b	7.09
Linux	Intel Corporation	Integrated Matrox G200eW3 Graphics Controller	1007.38	Ubuntu 22.04.4 LTS	0.1.32	gemma:2b	11.41
Linux	Intel Corporation	Integrated Matrox G200eW3 Graphics Controller	1007.38	Ubuntu 22.04.4 LTS	0.1.32	gemma:7b	7.05
Linux	Intel Corporation	Integrated Matrox G200eW3 Graphics Controller	1007.38	Ubuntu 22.04.4 LTS	0.1.32	llava:7b	5.24
Linux	Intel Corporation	Integrated Matrox G200eW3 Graphics Controller	1007.38	Ubuntu 22.04.4 LTS	0.1.32	llava:13b	4.52
Linux	AMD Ryzen 5 PRO 4650G with Radeon Graphics	no_gpu	30.77	Debian GNU/Linux 11 (bullseye)	0.1.39	phi3:3.8b	8.12
Linux	AMD Ryzen 5 PRO 4650G with Radeon Graphics	no_gpu	30.77	Debian GNU/Linux 11 (bullseye)	0.1.39	gemma:2b	12.02
Linux	AMD Ryzen 5 PRO 4650G with Radeon Graphics	no_gpu	30.77	Debian GNU/Linux 11 (bullseye)	0.1.39	gemma:7b	4.1
Linux	AMD Ryzen 5 PRO 4650G with Radeon Graphics	no_gpu	30.77	Debian GNU/Linux 11 (bullseye)	0.1.39	llava:7b	5.31
Linux	AMD Ryzen 5 PRO 4650G with Radeon Graphics	no_gpu	30.77	Debian GNU/Linux 11 (bullseye)	0.1.39	llava:13b	2.9
Linux	12th Gen Intel(R) Core(TM) i7-12700KF	NVIDIA GeForce RTX 4070	27.42	Ubuntu 20.04.5 LTS	0.1.39	mistral:7b	88.22
Linux	12th Gen Intel(R) Core(TM) i7-12700KF	NVIDIA GeForce RTX 4070	27.42	Ubuntu 20.04.5 LTS	0.1.39	llama3:8b	77.96
Linux	Intel Corporation	Integrated Matrox G200eW3 Graphics Controller	1007.35	Ubuntu 22.04.4 LTS	0.1.32	mistral:7b	17.91
Linux	Intel Corporation	Integrated Matrox G200eW3 Graphics Controller	1007.35	Ubuntu 22.04.4 LTS	0.1.32	llama3:8b	16.52
Linux	Intel Corporation	Integrated Matrox G200eW3 Graphics Controller	1007.35	Ubuntu 22.04.4 LTS	0.1.32	phi3:3.8b	19.9
Linux	Intel Corporation	Integrated Matrox G200eW3 Graphics Controller	1007.35	Ubuntu 22.04.4 LTS	0.1.32	gemma:2b	31.13
Linux	Intel Corporation	Integrated Matrox G200eW3 Graphics Controller	1007.35	Ubuntu 22.04.4 LTS	0.1.32	gemma:7b	16.57
Linux	Intel Corporation	Integrated Matrox G200eW3 Graphics Controller	1007.35	Ubuntu 22.04.4 LTS	0.1.32	llava:7b	18.38
Linux	Intel Corporation	Integrated Matrox G200eW3 Graphics Controller	1007.35	Ubuntu 22.04.4 LTS	0.1.32	llava:13b	12.35
Linux	Intel Corporation	Integrated Matrox G200eW3 Graphics Controller	1007.35	Ubuntu 22.04.4 LTS	0.1.33	mistral:7b	18.1
Linux	Intel Corporation	Integrated Matrox G200eW3 Graphics Controller	1007.35	Ubuntu 22.04.4 LTS	0.1.33	llama3:8b	16.87
Linux	Intel Corporation	Integrated Matrox G200eW3 Graphics Controller	1007.35	Ubuntu 22.04.4 LTS	0.1.33	phi3:3.8b	20.84
Linux	Intel Corporation	Integrated Matrox G200eW3 Graphics Controller	1007.35	Ubuntu 22.04.4 LTS	0.1.33	gemma:2b	32.2
Linux	Intel Corporation	Integrated Matrox G200eW3 Graphics Controller	1007.35	Ubuntu 22.04.4 LTS	0.1.33	gemma:7b	17.16
Linux	Intel Corporation	Integrated Matrox G200eW3 Graphics Controller	1007.35	Ubuntu 22.04.4 LTS	0.1.33	llava:7b	18.6
Linux	Intel Corporation	Integrated Matrox G200eW3 Graphics Controller	1007.35	Ubuntu 22.04.4 LTS	0.1.33	llava:13b	12.64
Linux	Intel Corporation	Integrated Matrox G200eW3 Graphics Controller	1007.35	Ubuntu 22.04.4 LTS	0.1.33	mistral:7b	18.18
Linux	Intel Corporation	Integrated Matrox G200eW3 Graphics Controller	1007.35	Ubuntu 22.04.4 LTS	0.1.33	llama3:8b	17.36
Linux	Intel Corporation	Integrated Matrox G200eW3 Graphics Controller	1007.35	Ubuntu 22.04.4 LTS	0.1.33	phi3:3.8b	20.93
Linux	Intel Corporation	Integrated Matrox G200eW3 Graphics Controller	1007.35	Ubuntu 22.04.4 LTS	0.1.33	gemma:2b	32.03
Linux	Intel Corporation	Integrated Matrox G200eW3 Graphics Controller	1007.35	Ubuntu 22.04.4 LTS	0.1.33	gemma:7b	17.08
Linux	Intel Corporation	Integrated Matrox G200eW3 Graphics Controller	1007.35	Ubuntu 22.04.4 LTS	0.1.33	llava:7b	18.85
Linux	Intel Corporation	Integrated Matrox G200eW3 Graphics Controller	1007.35	Ubuntu 22.04.4 LTS	0.1.33	llava:13b	12.64
Linux	Intel Corporation	Integrated Matrox G200eW3 Graphics Controller	1007.35	Ubuntu 22.04.4 LTS	0.1.33	mistral:7b	12.5
Linux	Intel Corporation	Integrated Matrox G200eW3 Graphics Controller	1007.35	Ubuntu 22.04.4 LTS	0.1.33	llama3:8b	13.56
Linux	Intel Corporation	Integrated Matrox G200eW3 Graphics Controller	1007.35	Ubuntu 22.04.4 LTS	0.1.33	phi3:3.8b	15.9
Linux	Intel Corporation	Integrated Matrox G200eW3 Graphics Controller	1007.35	Ubuntu 22.04.4 LTS	0.1.33	gemma:2b	24.28
Linux	Intel Corporation	Integrated Matrox G200eW3 Graphics Controller	1007.35	Ubuntu 22.04.4 LTS	0.1.33	gemma:7b	10.98
Linux	Intel Corporation	Integrated Matrox G200eW3 Graphics Controller	1007.35	Ubuntu 22.04.4 LTS	0.1.33	llava:7b	12.52
Linux	Intel Corporation	Integrated Matrox G200eW3 Graphics Controller	1007.35	Ubuntu 22.04.4 LTS	0.1.33	llava:13b	7.98
Linux	Intel(R) Xeon(R) Gold 6238R CPU @ 2.20GHz	NVIDIA A100-PCIE-40GB	440.48	Ubuntu 22.04.4 LTS	0.1.33	mistral:7b	122.16
Linux	Intel(R) Xeon(R) Gold 6238R CPU @ 2.20GHz	NVIDIA A100-PCIE-40GB	440.48	Ubuntu 22.04.4 LTS	0.1.33	llama3:8b	97.08
Linux	Intel(R) Xeon(R) Gold 6238R CPU @ 2.20GHz	NVIDIA A100-PCIE-40GB	440.48	Ubuntu 22.04.4 LTS	0.1.33	phi3:3.8b	132.73
Linux	Intel(R) Xeon(R) Gold 6238R CPU @ 2.20GHz	NVIDIA A100-PCIE-40GB	440.48	Ubuntu 22.04.4 LTS	0.1.33	gemma:2b	169.05
Linux	Intel(R) Xeon(R) Gold 6238R CPU @ 2.20GHz	NVIDIA A100-PCIE-40GB	440.48	Ubuntu 22.04.4 LTS	0.1.33	gemma:7b	102.59
Linux	Intel(R) Xeon(R) Gold 6238R CPU @ 2.20GHz	NVIDIA A100-PCIE-40GB	440.48	Ubuntu 22.04.4 LTS	0.1.33	llava:7b	124.82
Linux	Intel(R) Xeon(R) Gold 6238R CPU @ 2.20GHz	NVIDIA A100-PCIE-40GB	440.48	Ubuntu 22.04.4 LTS	0.1.33	llava:13b	85.45
Linux	AMD EPYC 9754 128-Core Processor	ASPEED Graphics Family	1511.46	Ubuntu 22.04.4 LTS	0.1.34	mistral:7b	17.24
Linux	AMD EPYC 9754 128-Core Processor	ASPEED Graphics Family	1511.46	Ubuntu 22.04.4 LTS	0.1.34	llama3:8b	16.55
Linux	AMD EPYC 9754 128-Core Processor	ASPEED Graphics Family	1511.46	Ubuntu 22.04.4 LTS	0.1.34	phi3:3.8b	17.97
Linux	AMD EPYC 9754 128-Core Processor	ASPEED Graphics Family	1511.46	Ubuntu 22.04.4 LTS	0.1.34	gemma:2b	26.32
Linux	AMD EPYC 9754 128-Core Processor	ASPEED Graphics Family	1511.46	Ubuntu 22.04.4 LTS	0.1.34	gemma:7b	16.87
Linux	AMD EPYC 9754 128-Core Processor	ASPEED Graphics Family	1511.46	Ubuntu 22.04.4 LTS	0.1.34	llava:7b	17.05
Linux	AMD EPYC 9754 128-Core Processor	ASPEED Graphics Family	1511.46	Ubuntu 22.04.4 LTS	0.1.34	llava:13b	13.11
Linux	13th Gen Intel(R) Core(TM) i9-13900KS	NVIDIA GeForce RTX 3060	62.56	Ubuntu 22.04.4 LTS	0.0.0	mistral:7b	64.34
Linux	13th Gen Intel(R) Core(TM) i9-13900KS	NVIDIA GeForce RTX 3060	62.56	Ubuntu 22.04.4 LTS	0.0.0	llama3:8b	58.08
Linux	13th Gen Intel(R) Core(TM) i9-13900KS	NVIDIA GeForce RTX 3060	62.56	Ubuntu 22.04.4 LTS	0.0.0	phi3:3.8b	92.9
Linux	13th Gen Intel(R) Core(TM) i9-13900KS	NVIDIA GeForce RTX 3060	62.56	Ubuntu 22.04.4 LTS	0.0.0	gemma:2b	129.34
Linux	13th Gen Intel(R) Core(TM) i9-13900KS	NVIDIA GeForce RTX 3060	62.56	Ubuntu 22.04.4 LTS	0.0.0	gemma:7b	53.26
Linux	13th Gen Intel(R) Core(TM) i9-13900KS	NVIDIA GeForce RTX 3060	62.56	Ubuntu 22.04.4 LTS	0.0.0	llava:7b	66.63
Linux	13th Gen Intel(R) Core(TM) i9-13900KS	NVIDIA GeForce RTX 3060	62.56	Ubuntu 22.04.4 LTS	0.0.0	llava:13b	39.37
Linux	AMD EPYC Processor (with IBPB)	no_gpu	31.36	Debian GNU/Linux 12 (bookworm)	0.1.34	mistral:7b	6.34
Linux	AMD EPYC Processor (with IBPB)	no_gpu	31.36	Debian GNU/Linux 12 (bookworm)	0.1.34	llama3:8b	6.08
Linux	AMD EPYC Processor (with IBPB)	no_gpu	31.36	Debian GNU/Linux 12 (bookworm)	0.1.34	phi3:3.8b	12.25
Linux	AMD EPYC Processor (with IBPB)	no_gpu	31.36	Debian GNU/Linux 12 (bookworm)	0.1.34	gemma:2b	14.28
Linux	AMD EPYC Processor (with IBPB)	no_gpu	31.36	Debian GNU/Linux 12 (bookworm)	0.1.34	gemma:7b	5.09
Linux	AMD EPYC Processor (with IBPB)	no_gpu	31.36	Debian GNU/Linux 12 (bookworm)	0.1.34	llava:7b	6.65
Linux	11th Gen Intel(R) Core(TM) i5-11600K @ 3.90GHz	NVIDIA GeForce RTX 3080	15.56	Ubuntu 22.04.4 LTS	0.1.33	mistral:7b	77.22
Linux	11th Gen Intel(R) Core(TM) i5-11600K @ 3.90GHz	NVIDIA GeForce RTX 3080	15.56	Ubuntu 22.04.4 LTS	0.1.33	llama3:8b	63
Linux	11th Gen Intel(R) Core(TM) i5-11600K @ 3.90GHz	NVIDIA GeForce RTX 3080	15.56	Ubuntu 22.04.4 LTS	0.1.33	phi3:3.8b	100.85
Linux	11th Gen Intel(R) Core(TM) i5-11600K @ 3.90GHz	NVIDIA GeForce RTX 3080	15.56	Ubuntu 22.04.4 LTS	0.1.33	gemma:2b	117.71
Linux	11th Gen Intel(R) Core(TM) i5-11600K @ 3.90GHz	NVIDIA GeForce RTX 3080	15.56	Ubuntu 22.04.4 LTS	0.1.33	gemma:7b	72.86
Linux	11th Gen Intel(R) Core(TM) i5-11600K @ 3.90GHz	NVIDIA GeForce RTX 3080	15.56	Ubuntu 22.04.4 LTS	0.1.33	llava:7b	82.71
Linux	11th Gen Intel(R) Core(TM) i5-11600K @ 3.90GHz	NVIDIA GeForce RTX 3080	15.56	Ubuntu 22.04.4 LTS	0.1.33	llava:13b	21.88
Linux	AMD EPYC 9754 128-Core Processor	ASPEED Graphics Family	1511.46	Ubuntu 22.04.4 LTS	0.1.34	mistral:7b	17.17
Linux	AMD EPYC 9754 128-Core Processor	ASPEED Graphics Family	1511.46	Ubuntu 22.04.4 LTS	0.1.34	llama3:8b	16.75
Linux	AMD EPYC 9754 128-Core Processor	ASPEED Graphics Family	1511.46	Ubuntu 22.04.4 LTS	0.1.34	phi3:3.8b	17.92
Linux	AMD EPYC 9754 128-Core Processor	ASPEED Graphics Family	1511.46	Ubuntu 22.04.4 LTS	0.1.34	gemma:2b	26.34
Linux	AMD EPYC 9754 128-Core Processor	ASPEED Graphics Family	1511.46	Ubuntu 22.04.4 LTS	0.1.34	gemma:7b	16.79
Linux	AMD EPYC 9754 128-Core Processor	ASPEED Graphics Family	1511.46	Ubuntu 22.04.4 LTS	0.1.34	llava:7b	17.35
Linux	AMD EPYC 9754 128-Core Processor	ASPEED Graphics Family	1511.46	Ubuntu 22.04.4 LTS	0.1.34	llava:13b	13.11
Linux	12th Gen Intel(R) Core(TM) i7-12700KF	NVIDIA GeForce RTX 4070	27.42	Ubuntu 20.04.5 LTS	0.1.39	phi3:3.8b	124.54
Linux	12th Gen Intel(R) Core(TM) i7-12700KF	NVIDIA GeForce RTX 4070	27.42	Ubuntu 20.04.5 LTS	0.1.39	gemma:2b	158.72
Linux	12th Gen Intel(R) Core(TM) i7-12700KF	NVIDIA GeForce RTX 4070	27.42	Ubuntu 20.04.5 LTS	0.1.39	gemma:7b	71.06
Linux	12th Gen Intel(R) Core(TM) i7-12700KF	NVIDIA GeForce RTX 4070	27.42	Ubuntu 20.04.5 LTS	0.1.39	llava:7b	90.84
Linux	12th Gen Intel(R) Core(TM) i7-12700KF	NVIDIA GeForce RTX 4070	27.42	Ubuntu 20.04.5 LTS	0.1.39	llava:13b	54.27
Linux	Intel(R) Xeon(R) Gold 6442Y	NVIDIA A40	503.39	Ubuntu 22.04.4 LTS	0.1.39	gemma:2b	129.84
Linux	Intel(R) Xeon(R) Gold 6442Y	NVIDIA A40	503.39	Ubuntu 22.04.4 LTS	0.1.39	gemma:7b	67.51
Linux	AMD EPYC 9754 128-Core Processor	ASPEED Graphics Family	1511.46	Ubuntu 22.04.4 LTS	0.1.34	mistral:7b	15.99
Linux	AMD EPYC 9754 128-Core Processor	ASPEED Graphics Family	1511.46	Ubuntu 22.04.4 LTS	0.1.34	llama3:8b	12.77
Linux	AMD EPYC 9754 128-Core Processor	ASPEED Graphics Family	1511.46	Ubuntu 22.04.4 LTS	0.1.34	phi3:3.8b	18.27
Linux	AMD EPYC 9754 128-Core Processor	ASPEED Graphics Family	1511.46	Ubuntu 22.04.4 LTS	0.1.34	gemma:2b	23.43
Linux	AMD EPYC 9754 128-Core Processor	ASPEED Graphics Family	1511.46	Ubuntu 22.04.4 LTS	0.1.34	gemma:7b	14.87
Linux	AMD EPYC 9754 128-Core Processor	ASPEED Graphics Family	1511.46	Ubuntu 22.04.4 LTS	0.1.34	llava:7b	12.09
Linux	AMD EPYC 9754 128-Core Processor	ASPEED Graphics Family	1511.46	Ubuntu 22.04.4 LTS	0.1.34	llava:13b	13.25
Linux	Intel(R) Xeon(R) CPU @ 2.30GHz	Tesla T4	25.45	Debian GNU/Linux 11 (bullseye)	0.1.34	mistral:7b	43.6
Linux	Intel(R) Xeon(R) CPU @ 2.30GHz	Tesla T4	25.45	Debian GNU/Linux 11 (bullseye)	0.1.34	llama3:8b	36.11
Linux	Intel(R) Xeon(R) CPU @ 2.30GHz	Tesla T4	25.45	Debian GNU/Linux 11 (bullseye)	0.1.34	phi3:3.8b	54.76
Linux	Intel(R) Xeon(R) CPU @ 2.30GHz	Tesla T4	25.45	Debian GNU/Linux 11 (bullseye)	0.1.34	gemma:2b	82.57
Linux	Intel(R) Xeon(R) CPU @ 2.30GHz	Tesla T4	25.45	Debian GNU/Linux 11 (bullseye)	0.1.34	gemma:7b	30.28
Linux	Intel(R) Xeon(R) CPU @ 2.30GHz	Tesla T4	25.45	Debian GNU/Linux 11 (bullseye)	0.1.34	llava:7b	40.58
Linux	Intel(R) Xeon(R) CPU @ 2.30GHz	Tesla T4	25.45	Debian GNU/Linux 11 (bullseye)	0.1.34	llava:13b	23.3
Linux	Intel Corporation	Integrated Matrox G200eW3 Graphics Controller	1007.35	Ubuntu 22.04.4 LTS	0.1.33	mistral:7b	18.34
Linux	Intel Corporation	Integrated Matrox G200eW3 Graphics Controller	1007.35	Ubuntu 22.04.4 LTS	0.1.33	llama3:8b	12.1
Linux	Intel Corporation	Integrated Matrox G200eW3 Graphics Controller	1007.35	Ubuntu 22.04.4 LTS	0.1.33	phi3:3.8b	20.66
Linux	Intel Corporation	Integrated Matrox G200eW3 Graphics Controller	1007.35	Ubuntu 22.04.4 LTS	0.1.33	gemma:2b	32.43
Linux	Intel Corporation	Integrated Matrox G200eW3 Graphics Controller	1007.35	Ubuntu 22.04.4 LTS	0.1.33	gemma:7b	11.32
Linux	Intel Corporation	Integrated Matrox G200eW3 Graphics Controller	1007.35	Ubuntu 22.04.4 LTS	0.1.33	llava:7b	18.49
Linux	Intel Corporation	Integrated Matrox G200eW3 Graphics Controller	1007.35	Ubuntu 22.04.4 LTS	0.1.33	llava:13b	12.4
Linux	Intel Corporation	Integrated Matrox G200eW3 Graphics Controller	1007.35	Ubuntu 22.04.4 LTS	0.1.33	mistral:7b	12.38
Linux	Intel Corporation	Integrated Matrox G200eW3 Graphics Controller	1007.35	Ubuntu 22.04.4 LTS	0.1.33	llama3:8b	11.67
Linux	Intel Corporation	Integrated Matrox G200eW3 Graphics Controller	1007.35	Ubuntu 22.04.4 LTS	0.1.33	phi3:3.8b	15.97
Linux	Intel Corporation	Integrated Matrox G200eW3 Graphics Controller	1007.35	Ubuntu 22.04.4 LTS	0.1.33	gemma:2b	23.66
Linux	Intel Corporation	Integrated Matrox G200eW3 Graphics Controller	1007.35	Ubuntu 22.04.4 LTS	0.1.33	gemma:7b	11.14
Linux	Intel Corporation	Integrated Matrox G200eW3 Graphics Controller	1007.35	Ubuntu 22.04.4 LTS	0.1.33	llava:7b	12.38
Linux	Intel Corporation	Integrated Matrox G200eW3 Graphics Controller	1007.35	Ubuntu 22.04.4 LTS	0.1.33	llava:13b	7.93
Linux	11th Gen Intel(R) Core(TM) i7-11800H @ 2.30GHz	NVIDIA GeForce RTX 3070	62.61	Ubuntu 22.04.4 LTS	0.1.34	mistral:7b	75.58
Linux	11th Gen Intel(R) Core(TM) i7-11800H @ 2.30GHz	NVIDIA GeForce RTX 3070	62.61	Ubuntu 22.04.4 LTS	0.1.34	llama3:8b	66.6
Linux	11th Gen Intel(R) Core(TM) i7-11800H @ 2.30GHz	NVIDIA GeForce RTX 3070	62.61	Ubuntu 22.04.4 LTS	0.1.34	phi3:3.8b	108.76
Linux	11th Gen Intel(R) Core(TM) i7-11800H @ 2.30GHz	NVIDIA GeForce RTX 3070	62.61	Ubuntu 22.04.4 LTS	0.1.34	gemma:2b	136.65
Linux	11th Gen Intel(R) Core(TM) i7-11800H @ 2.30GHz	NVIDIA GeForce RTX 3070	62.61	Ubuntu 22.04.4 LTS	0.1.34	gemma:7b	61.42
Linux	11th Gen Intel(R) Core(TM) i7-11800H @ 2.30GHz	NVIDIA GeForce RTX 3070	62.61	Ubuntu 22.04.4 LTS	0.1.34	llava:7b	78.05
Linux	11th Gen Intel(R) Core(TM) i7-11800H @ 2.30GHz	NVIDIA GeForce RTX 3070	62.61	Ubuntu 22.04.4 LTS	0.1.34	llava:13b	14.31
Linux	Intel Corporation	Integrated Matrox G200eW3 Graphics Controller	1007.35	Ubuntu 22.04.4 LTS	0.1.33	mistral:7b	18.47
Linux	Intel Corporation	Integrated Matrox G200eW3 Graphics Controller	1007.35	Ubuntu 22.04.4 LTS	0.1.33	llama3:8b	17.23
Linux	Intel Corporation	Integrated Matrox G200eW3 Graphics Controller	1007.35	Ubuntu 22.04.4 LTS	0.1.33	phi3:3.8b	21.19
Linux	Intel Corporation	Integrated Matrox G200eW3 Graphics Controller	1007.35	Ubuntu 22.04.4 LTS	0.1.33	gemma:2b	32.55
Linux	Intel Corporation	Integrated Matrox G200eW3 Graphics Controller	1007.35	Ubuntu 22.04.4 LTS	0.1.33	gemma:7b	17.33
Linux	Intel Corporation	Integrated Matrox G200eW3 Graphics Controller	1007.35	Ubuntu 22.04.4 LTS	0.1.33	llava:7b	18.71
Linux	Intel Corporation	Integrated Matrox G200eW3 Graphics Controller	1007.35	Ubuntu 22.04.4 LTS	0.1.33	llava:13b	12.81
Linux	Intel Corporation	Integrated Matrox G200eW3 Graphics Controller	1007.35	Ubuntu 22.04.4 LTS	0.1.33	mistral:7b	17.66
Linux	Intel Corporation	Integrated Matrox G200eW3 Graphics Controller	1007.35	Ubuntu 22.04.4 LTS	0.1.33	llama3:8b	11.9
Linux	Intel Corporation	Integrated Matrox G200eW3 Graphics Controller	1007.35	Ubuntu 22.04.4 LTS	0.1.33	phi3:3.8b	20.05
Linux	Intel Corporation	Integrated Matrox G200eW3 Graphics Controller	1007.35	Ubuntu 22.04.4 LTS	0.1.33	gemma:2b	31.62
Linux	Intel Corporation	Integrated Matrox G200eW3 Graphics Controller	1007.35	Ubuntu 22.04.4 LTS	0.1.33	gemma:7b	11.24
Linux	Intel Corporation	Integrated Matrox G200eW3 Graphics Controller	1007.35	Ubuntu 22.04.4 LTS	0.1.33	llava:7b	19.14
Linux	Intel Corporation	Integrated Matrox G200eW3 Graphics Controller	1007.35	Ubuntu 22.04.4 LTS	0.1.33	llava:13b	8.08
Linux	Intel Corporation	Integrated Matrox G200eW3 Graphics Controller	1007.38	Ubuntu 22.04.4 LTS	0.1.33	mistral:7b	2.63
Linux	Intel Corporation	Integrated Matrox G200eW3 Graphics Controller	1007.38	Ubuntu 22.04.4 LTS	0.1.33	llama3:8b	4.43
Linux	Intel Corporation	Integrated Matrox G200eW3 Graphics Controller	1007.38	Ubuntu 22.04.4 LTS	0.1.33	phi3:3.8b	5.75
Linux	Intel Corporation	Integrated Matrox G200eW3 Graphics Controller	1007.38	Ubuntu 22.04.4 LTS	0.1.33	gemma:2b	3.21
Linux	Intel Corporation	Integrated Matrox G200eW3 Graphics Controller	1007.38	Ubuntu 22.04.4 LTS	0.1.33	gemma:7b	2.44
Linux	Intel Corporation	Integrated Matrox G200eW3 Graphics Controller	1007.38	Ubuntu 22.04.4 LTS	0.1.33	llava:7b	3.33
Linux	Intel Corporation	Integrated Matrox G200eW3 Graphics Controller	1007.38	Ubuntu 22.04.4 LTS	0.1.33	llava:13b	3.49
Linux	Intel Corporation	Integrated Matrox G200eW3 Graphics Controller	1007.38	Ubuntu 22.04.4 LTS	0.1.33	mistral:7b	2.79
Linux	Intel Corporation	Integrated Matrox G200eW3 Graphics Controller	1007.38	Ubuntu 22.04.4 LTS	0.1.33	llama3:8b	2.68
Linux	Intel Corporation	Integrated Matrox G200eW3 Graphics Controller	1007.38	Ubuntu 22.04.4 LTS	0.1.33	phi3:3.8b	5.41
Linux	Intel Corporation	Integrated Matrox G200eW3 Graphics Controller	1007.38	Ubuntu 22.04.4 LTS	0.1.33	gemma:2b	3.55
Linux	Intel Corporation	Integrated Matrox G200eW3 Graphics Controller	1007.38	Ubuntu 22.04.4 LTS	0.1.33	gemma:7b	4.37
Linux	Intel Corporation	Integrated Matrox G200eW3 Graphics Controller	1007.38	Ubuntu 22.04.4 LTS	0.1.33	llava:7b	2.63
Linux	Intel Corporation	Integrated Matrox G200eW3 Graphics Controller	1007.38	Ubuntu 22.04.4 LTS	0.1.33	llava:13b	3.05
Linux	Intel(R) Core(TM) i5-10500T CPU @ 2.30GHz	SVGA II Adapter	7.76	Ubuntu 24.04 LTS	0.1.34	mistral:7b	5.35
Linux	Intel(R) Core(TM) i5-10500T CPU @ 2.30GHz	SVGA II Adapter	7.76	Ubuntu 24.04 LTS	0.1.34	llama3:8b	4.91
Linux	Intel(R) Core(TM) i5-10500T CPU @ 2.30GHz	SVGA II Adapter	7.76	Ubuntu 24.04 LTS	0.1.34	phi3:3.8b	8.81
Linux	Intel(R) Core(TM) i5-10500T CPU @ 2.30GHz	SVGA II Adapter	7.76	Ubuntu 24.04 LTS	0.1.34	gemma:2b	12.89
Linux	Intel(R) Core(TM) i5-10500T CPU @ 2.30GHz	SVGA II Adapter	7.76	Ubuntu 24.04 LTS	0.1.34	gemma:7b	4.29
Linux	Intel(R) Core(TM) i5-10500T CPU @ 2.30GHz	SVGA II Adapter	7.76	Ubuntu 24.04 LTS	0.1.34	llava:7b	5.47
Linux	13th Gen Intel(R) Core(TM) i9-13900K	NVIDIA GeForce RTX 3080 Ti	31.23	Ubuntu 22.04.3 LTS	0.1.29	mistral:7b	82.08
Linux	13th Gen Intel(R) Core(TM) i9-13900K	NVIDIA GeForce RTX 3080 Ti	31.23	Ubuntu 22.04.3 LTS	0.1.29	llama3:8b	74.6
Linux	13th Gen Intel(R) Core(TM) i9-13900K	NVIDIA GeForce RTX 3080 Ti	31.23	Ubuntu 22.04.3 LTS	0.1.29	gemma:2b	130.63
Linux	13th Gen Intel(R) Core(TM) i9-13900K	NVIDIA GeForce RTX 3080 Ti	31.23	Ubuntu 22.04.3 LTS	0.1.29	gemma:7b	73.25
Linux	13th Gen Intel(R) Core(TM) i9-13900K	NVIDIA GeForce RTX 3080 Ti	31.23	Ubuntu 22.04.3 LTS	0.1.29	llava:7b	82.85
Linux	13th Gen Intel(R) Core(TM) i9-13900K	NVIDIA GeForce RTX 3080 Ti	31.23	Ubuntu 22.04.3 LTS	0.1.29	llava:13b	60.96
Linux	13th Gen Intel(R) Core(TM) i9-13900K	NVIDIA GeForce RTX 3080 Ti	31.23	Ubuntu 22.04.3 LTS	0.1.29	mistral:7b	85.56
Linux	13th Gen Intel(R) Core(TM) i9-13900K	NVIDIA GeForce RTX 3080 Ti	31.23	Ubuntu 22.04.3 LTS	0.1.29	llama3:8b	77.83
Linux	13th Gen Intel(R) Core(TM) i9-13900K	NVIDIA GeForce RTX 3080 Ti	31.23	Ubuntu 22.04.3 LTS	0.1.29	gemma:2b	136.48
Linux	13th Gen Intel(R) Core(TM) i9-13900K	NVIDIA GeForce RTX 3080 Ti	31.23	Ubuntu 22.04.3 LTS	0.1.29	gemma:7b	75.95
Linux	13th Gen Intel(R) Core(TM) i9-13900K	NVIDIA GeForce RTX 3080 Ti	31.23	Ubuntu 22.04.3 LTS	0.1.29	llava:7b	86.57
Linux	13th Gen Intel(R) Core(TM) i9-13900K	NVIDIA GeForce RTX 3080 Ti	31.23	Ubuntu 22.04.3 LTS	0.1.29	llava:13b	63.77
Linux	Intel(R) Xeon(R) Gold 6442Y	NVIDIA A40	503.39	Ubuntu 22.04.4 LTS	0.1.39	gemma:2b	128.61
Linux	Intel(R) Xeon(R) Gold 6442Y	NVIDIA A40	503.39	Ubuntu 22.04.4 LTS	0.1.39	gemma:7b	67.57
Linux	AMD Ryzen 9 3900X 12-Core Processor	NVIDIA GeForce GTX 1650	125.80	Ubuntu 20.04.6 LTS	0.1.33	mistral:7b	12.41
Linux	AMD Ryzen 9 3900X 12-Core Processor	NVIDIA GeForce GTX 1650	125.80	Ubuntu 20.04.6 LTS	0.1.33	llama3:8b	9.95
Linux	AMD Ryzen 9 3900X 12-Core Processor	NVIDIA GeForce GTX 1650	125.80	Ubuntu 20.04.6 LTS	0.1.33	phi3:3.8b	21.55
Linux	AMD Ryzen 9 3900X 12-Core Processor	NVIDIA GeForce GTX 1650	125.80	Ubuntu 20.04.6 LTS	0.1.33	gemma:2b	45.45
Linux	AMD Ryzen 9 3900X 12-Core Processor	NVIDIA GeForce GTX 1650	125.80	Ubuntu 20.04.6 LTS	0.1.33	gemma:7b	8.09
Linux	AMD Ryzen 9 3900X 12-Core Processor	NVIDIA GeForce GTX 1650	125.80	Ubuntu 20.04.6 LTS	0.1.33	llava:7b	12.9
Linux	AMD Ryzen 9 3900X 12-Core Processor	NVIDIA GeForce GTX 1650	125.80	Ubuntu 20.04.6 LTS	0.1.33	llava:13b	5.57
Linux	Intel(R) Xeon(R) CPU @ 2.80GHz	no_gpu	15.63	Debian GNU/Linux 12 (bookworm)	0.1.39	mistral:7b	2.57
Linux	Intel(R) Xeon(R) CPU @ 2.80GHz	no_gpu	15.63	Debian GNU/Linux 12 (bookworm)	0.1.39	llama3:8b	2.49
Linux	Intel(R) Xeon(R) CPU @ 2.80GHz	no_gpu	15.63	Debian GNU/Linux 12 (bookworm)	0.1.39	phi3:3.8b	4.77
Linux	Intel(R) Xeon(R) CPU @ 2.80GHz	no_gpu	15.63	Debian GNU/Linux 12 (bookworm)	0.1.39	gemma:2b	6.55
Linux	Intel(R) Xeon(R) CPU @ 2.80GHz	no_gpu	15.63	Debian GNU/Linux 12 (bookworm)	0.1.39	gemma:7b	2.15
Linux	Intel(R) Xeon(R) CPU @ 2.80GHz	no_gpu	15.63	Debian GNU/Linux 12 (bookworm)	0.1.39	llava:7b	2.72
Linux	Intel(R) Xeon(R) CPU @ 2.80GHz	no_gpu	15.63	Debian GNU/Linux 12 (bookworm)	0.1.39	llava:13b	1.54
Linux	13th Gen Intel(R) Core(TM) i7-13700K	NVIDIA GeForce RTX 2070	15.49	Ubuntu 22.04.3 LTS	0.1.34	mistral:7b	45.99
Linux	13th Gen Intel(R) Core(TM) i7-13700K	NVIDIA GeForce RTX 2070	15.49	Ubuntu 22.04.3 LTS	0.1.34	llama3:8b	37.43
Linux	13th Gen Intel(R) Core(TM) i7-13700K	NVIDIA GeForce RTX 2070	15.49	Ubuntu 22.04.3 LTS	0.1.34	phi3:3.8b	58.22
Linux	13th Gen Intel(R) Core(TM) i7-13700K	NVIDIA GeForce RTX 2070	15.49	Ubuntu 22.04.3 LTS	0.1.34	gemma:2b	70.28
Linux	13th Gen Intel(R) Core(TM) i7-13700K	NVIDIA GeForce RTX 2070	15.49	Ubuntu 22.04.3 LTS	0.1.34	gemma:7b	32.13
Linux	13th Gen Intel(R) Core(TM) i7-13700K	NVIDIA GeForce RTX 2070	15.49	Ubuntu 22.04.3 LTS	0.1.34	llava:7b	41.61
Linux	13th Gen Intel(R) Core(TM) i7-13700K	NVIDIA GeForce RTX 2070	15.49	Ubuntu 22.04.3 LTS	0.1.34	llava:13b	7.1
Linux	13th Gen Intel(R) Core(TM) i7-13700K	NVIDIA GeForce RTX 2070	15.49	Ubuntu 22.04.3 LTS	0.1.34	mistral:7b	53.84
Linux	13th Gen Intel(R) Core(TM) i7-13700K	NVIDIA GeForce RTX 2070	15.49	Ubuntu 22.04.3 LTS	0.1.34	llama3:8b	51.17
Linux	13th Gen Intel(R) Core(TM) i7-13700K	NVIDIA GeForce RTX 2070	15.49	Ubuntu 22.04.3 LTS	0.1.34	phi3:3.8b	79.5
Linux	13th Gen Intel(R) Core(TM) i7-13700K	NVIDIA GeForce RTX 2070	15.49	Ubuntu 22.04.3 LTS	0.1.34	gemma:2b	106.67
Linux	13th Gen Intel(R) Core(TM) i7-13700K	NVIDIA GeForce RTX 2070	15.49	Ubuntu 22.04.3 LTS	0.1.34	gemma:7b	44.75
Linux	13th Gen Intel(R) Core(TM) i7-13700K	NVIDIA GeForce RTX 2070	15.49	Ubuntu 22.04.3 LTS	0.1.34	llava:7b	58.94
Linux	13th Gen Intel(R) Core(TM) i7-13700K	NVIDIA GeForce RTX 2070	15.49	Ubuntu 22.04.3 LTS	0.1.34	llava:13b	9.92
Linux	Intel(R) Xeon(R) Gold 6438Y+	no_gpu	753.27	EulerOS 2.0	0.1.35	mistral:7b	6.51
Linux	Intel(R) Xeon(R) Gold 6438Y+	no_gpu	753.27	EulerOS 2.0	0.1.35	gemma:2b	17.89
Linux	Intel(R) Xeon(R) Gold 6438Y+	no_gpu	753.27	EulerOS 2.0	0.1.35	gemma:7b	5.85
Linux	Intel(R) Xeon(R) Gold 6438Y+	no_gpu	753.27	EulerOS 2.0	0.1.35	llama2:7b	7.01
Linux	Intel(R) Xeon(R) Gold 6438Y+	no_gpu	753.27	EulerOS 2.0	0.1.35	llama2:13b	4.03
Linux	Intel(R) Xeon(R) Gold 6438Y+	no_gpu	753.27	EulerOS 2.0	0.1.35	llava:7b	7.32
Linux	Intel(R) Xeon(R) Gold 6438Y+	no_gpu	753.27	EulerOS 2.0	0.1.35	llava:13b	4.18
Linux	Intel(R) Xeon(R) Gold 6442Y	NVIDIA A40	503.39	Ubuntu 22.04.4 LTS	0.1.39	gemma:7b	60.76
Linux	Intel(R) Xeon(R) Gold 5118 CPU @ 2.30GHz	Tesla V100-SXM2-16GB	502.58	"Rocky Linux 9.3 (Blue Onyx)"	0.1.38	mistral:7b	102.73
Linux	Intel(R) Xeon(R) Gold 5118 CPU @ 2.30GHz	Tesla V100-SXM2-16GB	502.58	"Rocky Linux 9.3 (Blue Onyx)"	0.1.38	llama3:8b	82.74
Linux	Intel(R) Xeon(R) Gold 5118 CPU @ 2.30GHz	Tesla V100-SXM2-16GB	502.58	"Rocky Linux 9.3 (Blue Onyx)"	0.1.38	phi3:3.8b	125.84
Linux	Intel(R) Xeon(R) Gold 5118 CPU @ 2.30GHz	Tesla V100-SXM2-16GB	502.58	"Rocky Linux 9.3 (Blue Onyx)"	0.1.38	gemma:2b	152.63
Linux	Intel(R) Xeon(R) Gold 5118 CPU @ 2.30GHz	Tesla V100-SXM2-16GB	502.58	"Rocky Linux 9.3 (Blue Onyx)"	0.1.38	gemma:7b	85.88
Linux	Intel(R) Xeon(R) Gold 5118 CPU @ 2.30GHz	Tesla V100-SXM2-16GB	502.58	"Rocky Linux 9.3 (Blue Onyx)"	0.1.38	llava:7b	106.38
Linux	Intel(R) Xeon(R) Gold 5118 CPU @ 2.30GHz	Tesla V100-SXM2-16GB	502.58	"Rocky Linux 9.3 (Blue Onyx)"	0.1.38	llava:13b	70.77
Linux	Intel(R) Xeon(R) CPU E5-2690 v4 @ 2.60GHz	Tesla M60	220.14	Ubuntu 22.04.4 LTS	0.1.39	mistral:7b	9.99
Linux	Intel(R) Xeon(R) CPU E5-2690 v4 @ 2.60GHz	Tesla M60	220.14	Ubuntu 22.04.4 LTS	0.1.39	llama3:8b	9.23
Linux	Intel(R) Xeon(R) CPU E5-2690 v4 @ 2.60GHz	Tesla M60	220.14	Ubuntu 22.04.4 LTS	0.1.39	phi3:3.8b	21.15
Linux	Intel(R) Xeon(R) CPU E5-2690 v4 @ 2.60GHz	Tesla M60	220.14	Ubuntu 22.04.4 LTS	0.1.39	gemma:2b	26.32
Linux	Intel(R) Xeon(R) CPU E5-2690 v4 @ 2.60GHz	Tesla M60	220.14	Ubuntu 22.04.4 LTS	0.1.39	gemma:7b	8.18
Linux	Intel(R) Xeon(R) CPU E5-2690 v4 @ 2.60GHz	Tesla M60	220.14	Ubuntu 22.04.4 LTS	0.1.39	llava:7b	10.05
Linux	Intel(R) Xeon(R) CPU E5-2690 v4 @ 2.60GHz	Tesla M60	220.14	Ubuntu 22.04.4 LTS	0.1.39	llava:13b	5.72
Linux	Intel(R) Core(TM) i7-7700HQ CPU @ 2.80GHz	NVIDIA GeForce GTX 1050 Ti	15.49	Ubuntu 22.04.4 LTS	0.1.39	llama3:8b	5.8
Linux	Intel(R) Core(TM) i7-7700HQ CPU @ 2.80GHz	NVIDIA GeForce GTX 1050 Ti	15.49	Ubuntu 22.04.4 LTS	0.1.39	phi3:3.8b	23.27
Linux	Intel(R) Core(TM) i7-7700HQ CPU @ 2.80GHz	NVIDIA GeForce GTX 1050 Ti	15.49	Ubuntu 22.04.4 LTS	0.1.39	gemma:2b	40.81
Linux	Intel(R) Xeon(R) Platinum 8369HC B3 CPU	NVIDIA GeForce RTX 4070	27.42	Ubuntu 20.04.5 LTS	0.1.39	mistral:7b	90.57
Linux	Intel(R) Xeon(R) Platinum 8369HC B3 CPU	NVIDIA GeForce RTX 4070	27.42	Ubuntu 20.04.5 LTS	0.1.39	llama3:8b	79.96
Linux	Intel(R) Xeon(R) Platinum 8369HC B3 CPU	NVIDIA GeForce RTX 4070	27.42	Ubuntu 20.04.5 LTS	0.1.39	phi3:3.8b	128.7
Linux	Intel(R) Xeon(R) Platinum 8369HC B3 CPU	NVIDIA GeForce RTX 4070	27.42	Ubuntu 20.04.5 LTS	0.1.39	gemma:2b	164.85
Linux	Intel(R) Xeon(R) Platinum 8369HC B3 CPU	NVIDIA GeForce RTX 4070	27.42	Ubuntu 20.04.5 LTS	0.1.39	gemma:7b	72.14
Linux	Intel(R) Xeon(R) Platinum 8369HC B3 CPU	NVIDIA GeForce RTX 4070	27.42	Ubuntu 20.04.5 LTS	0.1.39	llava:7b	93.37
Linux	Intel(R) Xeon(R) Platinum 8369HC B3 CPU	NVIDIA GeForce RTX 4070	27.42	Ubuntu 20.04.5 LTS	0.1.39	llava:13b	55.43
Linux	Intel(R) Xeon(R) Gold 5118 CPU @ 2.30GHz	Tesla V100-SXM2-16GB	502.58	"Rocky Linux 9.3 (Blue Onyx)"	0.1.38	mistral:7b	102.86
Linux	Intel(R) Xeon(R) Gold 5118 CPU @ 2.30GHz	Tesla V100-SXM2-16GB	502.58	"Rocky Linux 9.3 (Blue Onyx)"	0.1.38	llama3:8b	82.46
Linux	Intel(R) Xeon(R) Gold 5118 CPU @ 2.30GHz	Tesla V100-SXM2-16GB	502.58	"Rocky Linux 9.3 (Blue Onyx)"	0.1.38	phi3:3.8b	125.53
Linux	Intel(R) Xeon(R) Gold 5118 CPU @ 2.30GHz	Tesla V100-SXM2-16GB	502.58	"Rocky Linux 9.3 (Blue Onyx)"	0.1.38	gemma:2b	150.53
Linux	Intel(R) Xeon(R) Gold 5118 CPU @ 2.30GHz	Tesla V100-SXM2-16GB	502.58	"Rocky Linux 9.3 (Blue Onyx)"	0.1.38	gemma:7b	86.71
Linux	Intel(R) Xeon(R) Gold 5118 CPU @ 2.30GHz	Tesla V100-SXM2-16GB	502.58	"Rocky Linux 9.3 (Blue Onyx)"	0.1.38	llava:7b	107.85
Linux	Intel(R) Xeon(R) Gold 5118 CPU @ 2.30GHz	Tesla V100-SXM2-16GB	502.58	"Rocky Linux 9.3 (Blue Onyx)"	0.1.38	llava:13b	70.72
Linux	12th Gen Intel(R) Core(TM) i7-12700	NVIDIA GeForce RTX 3060 Ti	31.12	Ubuntu 20.04.6 LTS	0.1.28	mistral:7b	76.04
Linux	12th Gen Intel(R) Core(TM) i7-12700	NVIDIA GeForce RTX 3060 Ti	31.12	Ubuntu 20.04.6 LTS	0.1.28	llama3:8b	67.58
Linux	Intel(R) Core(TM) i7-7700HQ CPU @ 2.80GHz	NVIDIA GeForce GTX 1050 Ti	15.49	Ubuntu 22.04.4 LTS	0.1.39	gemma:7b	4.15
Linux	AMD Ryzen Threadripper 3970X 32-Core Processor	NVIDIA TITAN X (Pascal)	125.65	Ubuntu 24.04 LTS	0.1.36	mistral:7b	56.07
Linux	AMD Ryzen Threadripper 3970X 32-Core Processor	NVIDIA TITAN X (Pascal)	125.65	Ubuntu 24.04 LTS	0.1.36	llama3:8b	48.36
Linux	AMD Ryzen Threadripper 3970X 32-Core Processor	NVIDIA TITAN X (Pascal)	125.65	Ubuntu 24.04 LTS	0.1.36	phi3:3.8b	68.38
Linux	AMD Ryzen Threadripper 3970X 32-Core Processor	NVIDIA TITAN X (Pascal)	125.65	Ubuntu 24.04 LTS	0.1.36	gemma:2b	106.53
Linux	AMD Ryzen Threadripper 3970X 32-Core Processor	NVIDIA TITAN X (Pascal)	125.65	Ubuntu 24.04 LTS	0.1.36	gemma:7b	42.88
Linux	AMD Ryzen Threadripper 3970X 32-Core Processor	NVIDIA TITAN X (Pascal)	125.65	Ubuntu 24.04 LTS	0.1.36	llava:7b	57.84
Linux	AMD Ryzen Threadripper 3970X 32-Core Processor	NVIDIA TITAN X (Pascal)	125.65	Ubuntu 24.04 LTS	0.1.36	llava:13b	32.35
Linux	Intel(R) Core(TM) i7-7700HQ CPU @ 2.80GHz	NVIDIA GeForce GTX 1050 Ti	15.49	Ubuntu 22.04.4 LTS	0.1.39	llava:7b	7.54
Linux	Intel(R) Core(TM) i7-7700HQ CPU @ 2.80GHz	NVIDIA GeForce GTX 1050 Ti	15.49	Ubuntu 22.04.4 LTS	0.1.39	llava:13b	2.53
Linux	12th Gen Intel(R) Core(TM) i7-12700	NVIDIA GeForce RTX 3060 Ti	31.12	Ubuntu 20.04.6 LTS	0.1.39	mistral:7b	76.62
Linux	12th Gen Intel(R) Core(TM) i7-12700	NVIDIA GeForce RTX 3060 Ti	31.12	Ubuntu 20.04.6 LTS	0.1.39	llama3:8b	68.16
Linux	12th Gen Intel(R) Core(TM) i7-12700	NVIDIA GeForce RTX 3060 Ti	31.12	Ubuntu 20.04.6 LTS	0.1.39	phi3:3.8b	108.47
Linux	AMD Ryzen Threadripper 3960X 24-Core Processor	NVIDIA GeForce RTX 2060 SUPER	62.65	Arch Linux	0.1.39	mistral:7b	35.24
Linux	AMD Ryzen Threadripper 3960X 24-Core Processor	NVIDIA GeForce RTX 2060 SUPER	62.65	Arch Linux	0.1.39	llama3:8b	28.98
Linux	AMD Ryzen Threadripper 3960X 24-Core Processor	NVIDIA GeForce RTX 2060 SUPER	62.65	Arch Linux	0.1.39	phi3:3.8b	36.94
Linux	AMD Ryzen Threadripper 3960X 24-Core Processor	NVIDIA GeForce RTX 2060 SUPER	62.65	Arch Linux	0.1.39	gemma:2b	64.99
Linux	AMD Ryzen Threadripper 3960X 24-Core Processor	NVIDIA GeForce RTX 2060 SUPER	62.65	Arch Linux	0.1.39	gemma:7b	24.75
Linux	AMD Ryzen Threadripper 3960X 24-Core Processor	NVIDIA GeForce RTX 2060 SUPER	62.65	Arch Linux	0.1.39	llava:7b	33.77
Linux	AMD Ryzen Threadripper 3960X 24-Core Processor	NVIDIA GeForce RTX 2060 SUPER	62.65	Arch Linux	0.1.39	llava:13b	23.92
Linux	Intel(R) Core(TM) i9-14900K	NVIDIA GeForce RTX 4090	62.56	Ubuntu 22.04.4 LTS	0.1.37	mistral:7b	142.83
Linux	Intel(R) Core(TM) i9-14900K	NVIDIA GeForce RTX 4090	62.56	Ubuntu 22.04.4 LTS	0.1.37	llama3:8b	124.1
Linux	Intel(R) Core(TM) i9-14900K	NVIDIA GeForce RTX 4090	62.56	Ubuntu 22.04.4 LTS	0.1.37	phi3:3.8b	193.14
Linux	Intel(R) Core(TM) i9-14900K	NVIDIA GeForce RTX 4090	62.56	Ubuntu 22.04.4 LTS	0.1.37	gemma:2b	230.17
Linux	Intel(R) Core(TM) i9-14900K	NVIDIA GeForce RTX 4090	62.56	Ubuntu 22.04.4 LTS	0.1.37	gemma:7b	117.45
Linux	Intel(R) Core(TM) i9-14900K	NVIDIA GeForce RTX 4090	62.56	Ubuntu 22.04.4 LTS	0.1.37	llava:7b	145.7
Linux	Intel(R) Core(TM) i9-14900K	NVIDIA GeForce RTX 4090	62.56	Ubuntu 22.04.4 LTS	0.1.37	llava:13b	92.2
Linux	Intel(R) Core(TM) i9-14900K	NVIDIA GeForce RTX 4090	62.56	Ubuntu 22.04.4 LTS	0.1.37	mistral:7b	142.48
Linux	Intel(R) Core(TM) i9-14900K	NVIDIA GeForce RTX 4090	62.56	Ubuntu 22.04.4 LTS	0.1.37	llama3:8b	124.02
Linux	Intel(R) Core(TM) i9-14900K	NVIDIA GeForce RTX 4090	62.56	Ubuntu 22.04.4 LTS	0.1.37	phi3:3.8b	192.82
Linux	Intel(R) Core(TM) i9-14900K	NVIDIA GeForce RTX 4090	62.56	Ubuntu 22.04.4 LTS	0.1.37	gemma:2b	226.25
Linux	Intel(R) Core(TM) i9-14900K	NVIDIA GeForce RTX 4090	62.56	Ubuntu 22.04.4 LTS	0.1.37	gemma:7b	116.63
Linux	Intel(R) Core(TM) i9-14900K	NVIDIA GeForce RTX 4090	62.56	Ubuntu 22.04.4 LTS	0.1.37	llava:7b	145.87
Linux	Intel(R) Core(TM) i9-14900K	NVIDIA GeForce RTX 4090	62.56	Ubuntu 22.04.4 LTS	0.1.37	llava:13b	92.57
Linux	13th Gen Intel(R) Core(TM) i9-13900HX	Raptor Lake-S UHD Graphics	64.00	Ubuntu 23.10	0.1.37	mistral:7b	11.02
Linux	13th Gen Intel(R) Core(TM) i9-13900HX	Raptor Lake-S UHD Graphics	64.00	Ubuntu 23.10	0.1.37	llama3:8b	10.08
Linux	13th Gen Intel(R) Core(TM) i9-13900HX	Raptor Lake-S UHD Graphics	64.00	Ubuntu 23.10	0.1.37	phi3:3.8b	18.16
Linux	13th Gen Intel(R) Core(TM) i9-13900HX	Raptor Lake-S UHD Graphics	64.00	Ubuntu 23.10	0.1.37	gemma:2b	24.45
Linux	13th Gen Intel(R) Core(TM) i9-13900HX	Raptor Lake-S UHD Graphics	64.00	Ubuntu 23.10	0.1.37	gemma:7b	8.74
Linux	13th Gen Intel(R) Core(TM) i9-13900HX	Raptor Lake-S UHD Graphics	64.00	Ubuntu 23.10	0.1.37	llava:7b	11.34
Linux	13th Gen Intel(R) Core(TM) i9-13900HX	Raptor Lake-S UHD Graphics	64.00	Ubuntu 23.10	0.1.37	llava:13b	6.42
Linux	Intel(R) Core(TM) i9-9900K CPU @ 3.60GHz	no_gpu	31.26	NixOS 24.05 (Uakari)	0.1.37	mistral:7b	38.25
Linux	Intel(R) Core(TM) i9-9900K CPU @ 3.60GHz	no_gpu	31.26	NixOS 24.05 (Uakari)	0.1.37	llama3:8b	35.75
Linux	Intel(R) Core(TM) i9-9900K CPU @ 3.60GHz	no_gpu	31.26	NixOS 24.05 (Uakari)	0.1.37	phi3:3.8b	50.24
Linux	Intel(R) Core(TM) i9-9900K CPU @ 3.60GHz	no_gpu	31.26	NixOS 24.05 (Uakari)	0.1.37	gemma:2b	82.4
Linux	Intel(R) Core(TM) i9-9900K CPU @ 3.60GHz	no_gpu	31.26	NixOS 24.05 (Uakari)	0.1.37	gemma:7b	31.32
Linux	Intel(R) Core(TM) i9-9900K CPU @ 3.60GHz	no_gpu	31.26	NixOS 24.05 (Uakari)	0.1.37	llava:7b	41.58
Linux	Intel(R) Core(TM) i9-9900K CPU @ 3.60GHz	no_gpu	31.26	NixOS 24.05 (Uakari)	0.1.37	llava:13b	8.25
Linux	AMD Ryzen 5 7600 6-Core Processor	NVIDIA GeForce RTX 3090	187.32	Fedora Linux 39 (Thirty Nine)	v0.1.38	mistral:7b	144.88
Linux	AMD Ryzen 5 7600 6-Core Processor	NVIDIA GeForce RTX 3090	187.32	Fedora Linux 39 (Thirty Nine)	v0.1.38	llama3:8b	125.53
Linux	AMD Ryzen 5 7600 6-Core Processor	NVIDIA GeForce RTX 3090	187.32	Fedora Linux 39 (Thirty Nine)	v0.1.38	phi3:3.8b	174.66
Linux	AMD Ryzen 5 7600 6-Core Processor	NVIDIA GeForce RTX 3090	187.32	Fedora Linux 39 (Thirty Nine)	v0.1.38	gemma:2b	233.13
Linux	AMD Ryzen 5 7600 6-Core Processor	NVIDIA GeForce RTX 3090	187.32	Fedora Linux 39 (Thirty Nine)	v0.1.38	gemma:7b	110.31
Linux	AMD Ryzen 5 7600 6-Core Processor	NVIDIA GeForce RTX 3090	187.32	Fedora Linux 39 (Thirty Nine)	v0.1.38	llava:7b	154.88
Linux	AMD Ryzen 5 7600 6-Core Processor	NVIDIA GeForce RTX 3090	187.32	Fedora Linux 39 (Thirty Nine)	v0.1.38	llava:13b	92.85
Linux	12th Gen Intel(R) Core(TM) i7-12700K	NVIDIA GeForce RTX 3060	62.56	Ubuntu 24.04 LTS	0.1.37	mistral:7b	63.84
Linux	12th Gen Intel(R) Core(TM) i7-12700K	NVIDIA GeForce RTX 3060	62.56	Ubuntu 24.04 LTS	0.1.37	llama3:8b	56.58
Linux	12th Gen Intel(R) Core(TM) i7-12700K	NVIDIA GeForce RTX 3060	62.56	Ubuntu 24.04 LTS	0.1.37	phi3:3.8b	91.16
Linux	12th Gen Intel(R) Core(TM) i7-12700K	NVIDIA GeForce RTX 3060	62.56	Ubuntu 24.04 LTS	0.1.37	gemma:2b	122.82
Linux	12th Gen Intel(R) Core(TM) i7-12700K	NVIDIA GeForce RTX 3060	62.56	Ubuntu 24.04 LTS	0.1.37	gemma:7b	51.8
Linux	12th Gen Intel(R) Core(TM) i7-12700K	NVIDIA GeForce RTX 3060	62.56	Ubuntu 24.04 LTS	0.1.37	llava:7b	65.93
Linux	12th Gen Intel(R) Core(TM) i7-12700K	NVIDIA GeForce RTX 3060	62.56	Ubuntu 24.04 LTS	0.1.37	llava:13b	38.98
Linux	AMD EPYC 9354 32-Core Processor	NVIDIA GeForce RTX 4090	755.57	Ubuntu 22.04.3 LTS	0.1.37	mistral:7b	138.8
Linux	AMD EPYC 9354 32-Core Processor	NVIDIA GeForce RTX 4090	755.57	Ubuntu 22.04.3 LTS	0.1.37	llama3:8b	117.67
Linux	AMD EPYC 9354 32-Core Processor	NVIDIA GeForce RTX 4090	755.57	Ubuntu 22.04.3 LTS	0.1.37	phi3:3.8b	187.06
Linux	AMD EPYC 9354 32-Core Processor	NVIDIA GeForce RTX 4090	755.57	Ubuntu 22.04.3 LTS	0.1.37	gemma:2b	204.16
Linux	AMD EPYC 9354 32-Core Processor	NVIDIA GeForce RTX 4090	755.57	Ubuntu 22.04.3 LTS	0.1.37	gemma:7b	108.78
Linux	AMD EPYC 9354 32-Core Processor	NVIDIA GeForce RTX 4090	755.57	Ubuntu 22.04.3 LTS	0.1.37	llava:7b	142.45
Linux	AMD EPYC 9354 32-Core Processor	NVIDIA GeForce RTX 4090	755.57	Ubuntu 22.04.3 LTS	0.1.37	llava:13b	90.12
Linux	AMD EPYC 9354 32-Core Processor	NVIDIA GeForce RTX 4090	755.57	Ubuntu 22.04.3 LTS	0.1.37	mistral:7b	137.66
Linux	AMD EPYC 9354 32-Core Processor	NVIDIA GeForce RTX 4090	755.57	Ubuntu 22.04.3 LTS	0.1.37	llama3:8b	117.08
Linux	AMD EPYC 9354 32-Core Processor	NVIDIA GeForce RTX 4090	755.57	Ubuntu 22.04.3 LTS	0.1.37	phi3:3.8b	186.63
Linux	AMD EPYC 9354 32-Core Processor	NVIDIA GeForce RTX 4090	755.57	Ubuntu 22.04.3 LTS	0.1.37	gemma:2b	206.96
Linux	AMD EPYC 9354 32-Core Processor	NVIDIA GeForce RTX 4090	755.57	Ubuntu 22.04.3 LTS	0.1.37	gemma:7b	108.63
Linux	AMD EPYC 9354 32-Core Processor	NVIDIA GeForce RTX 4090	755.57	Ubuntu 22.04.3 LTS	0.1.37	llava:7b	144.29
Linux	AMD EPYC 9354 32-Core Processor	NVIDIA GeForce RTX 4090	755.57	Ubuntu 22.04.3 LTS	0.1.37	llava:13b	90.11
Linux	AMD EPYC 7643 48-Core Processor	NVIDIA A100 80GB PCIe	1007.79	Ubuntu 22.04.3 LTS	0.1.37	mistral:7b	128.94
Linux	AMD EPYC 7643 48-Core Processor	NVIDIA A100 80GB PCIe	1007.79	Ubuntu 22.04.3 LTS	0.1.37	llama3:8b	109.29
Linux	AMD EPYC 7643 48-Core Processor	NVIDIA A100 80GB PCIe	1007.79	Ubuntu 22.04.3 LTS	0.1.37	phi3:3.8b	138.47
Linux	AMD EPYC 7643 48-Core Processor	NVIDIA A100 80GB PCIe	1007.79	Ubuntu 22.04.3 LTS	0.1.37	gemma:2b	164.93
Linux	AMD EPYC 7643 48-Core Processor	NVIDIA A100 80GB PCIe	1007.79	Ubuntu 22.04.3 LTS	0.1.37	gemma:7b	103.43
Linux	AMD EPYC 7643 48-Core Processor	NVIDIA A100 80GB PCIe	1007.79	Ubuntu 22.04.3 LTS	0.1.37	llava:7b	132.33
Linux	AMD EPYC 7643 48-Core Processor	NVIDIA A100 80GB PCIe	1007.79	Ubuntu 22.04.3 LTS	0.1.37	llava:13b	91.53
Linux	AMD EPYC 9354 32-Core Processor	NVIDIA RTX 6000 Ada Generation	1511.50	Ubuntu 22.04.3 LTS	0.1.37	mistral:7b	131.76
Linux	AMD EPYC 9354 32-Core Processor	NVIDIA RTX 6000 Ada Generation	1511.50	Ubuntu 22.04.3 LTS	0.1.37	llama3:8b	113.51
Linux	AMD EPYC 9354 32-Core Processor	NVIDIA RTX 6000 Ada Generation	1511.50	Ubuntu 22.04.3 LTS	0.1.37	phi3:3.8b	179.84
Linux	AMD EPYC 9354 32-Core Processor	NVIDIA RTX 6000 Ada Generation	1511.50	Ubuntu 22.04.3 LTS	0.1.37	gemma:2b	199.07
Linux	AMD EPYC 9354 32-Core Processor	NVIDIA RTX 6000 Ada Generation	1511.50	Ubuntu 22.04.3 LTS	0.1.37	gemma:7b	104.56
Linux	AMD EPYC 9354 32-Core Processor	NVIDIA RTX 6000 Ada Generation	1511.50	Ubuntu 22.04.3 LTS	0.1.37	llava:7b	135.91
Linux	AMD EPYC 9354 32-Core Processor	NVIDIA RTX 6000 Ada Generation	1511.50	Ubuntu 22.04.3 LTS	0.1.37	llava:13b	85.62
Linux	AMD EPYC 7402 24-Core Processor	NVIDIA GeForce RTX 4090	503.55	Ubuntu 22.04.3 LTS	0.1.37	mistral:7b	132.89
Linux	AMD EPYC 7402 24-Core Processor	NVIDIA GeForce RTX 4090	503.55	Ubuntu 22.04.3 LTS	0.1.37	llama3:8b	104.45
Linux	AMD EPYC 7402 24-Core Processor	NVIDIA GeForce RTX 4090	503.55	Ubuntu 22.04.3 LTS	0.1.37	phi3:3.8b	174.88
Linux	AMD EPYC 7402 24-Core Processor	NVIDIA GeForce RTX 4090	503.55	Ubuntu 22.04.3 LTS	0.1.37	gemma:2b	182.41
Linux	AMD EPYC 7402 24-Core Processor	NVIDIA GeForce RTX 4090	503.55	Ubuntu 22.04.3 LTS	0.1.37	gemma:7b	102.56
Linux	AMD EPYC 7402 24-Core Processor	NVIDIA GeForce RTX 4090	503.55	Ubuntu 22.04.3 LTS	0.1.37	llava:7b	135.67
Linux	AMD EPYC 7402 24-Core Processor	NVIDIA GeForce RTX 4090	503.55	Ubuntu 22.04.3 LTS	0.1.37	llava:13b	88.77
Linux	12th Gen Intel(R) Core(TM) i5-12400	Intel Corporation	15.32	Ubuntu 22.04.4 LTS	0.1.34	mistral:7b	9.92
Linux	12th Gen Intel(R) Core(TM) i5-12400	Intel Corporation	15.32	Ubuntu 22.04.4 LTS	0.1.34	llama3:8b	9.17
Linux	12th Gen Intel(R) Core(TM) i5-12400	Intel Corporation	15.32	Ubuntu 22.04.4 LTS	0.1.34	phi3:3.8b	16.72
Linux	12th Gen Intel(R) Core(TM) i5-12400	Intel Corporation	15.32	Ubuntu 22.04.4 LTS	0.1.34	gemma:2b	22.82
Linux	12th Gen Intel(R) Core(TM) i5-12400	Intel Corporation	15.32	Ubuntu 22.04.4 LTS	0.1.34	gemma:7b	8.05
Linux	12th Gen Intel(R) Core(TM) i5-12400	Intel Corporation	15.32	Ubuntu 22.04.4 LTS	0.1.34	llava:7b	10.09
Linux	12th Gen Intel(R) Core(TM) i5-12400	Intel Corporation	15.32	Ubuntu 22.04.4 LTS	0.1.34	llava:13b	5.76
Linux	Intel(R) Core(TM) i9-9960X CPU @ 3.10GHz	NVIDIA GeForce RTX 2080 Ti	62.47	Ubuntu 22.04.4 LTS	0.1.34	mistral:7b	90.15
Linux	Intel(R) Core(TM) i9-9960X CPU @ 3.10GHz	NVIDIA GeForce RTX 2080 Ti	62.47	Ubuntu 22.04.4 LTS	0.1.34	llama3:8b	76.56
Linux	Intel(R) Core(TM) i9-9960X CPU @ 3.10GHz	NVIDIA GeForce RTX 2080 Ti	62.47	Ubuntu 22.04.4 LTS	0.1.34	phi3:3.8b	122.11
Linux	Intel(R) Core(TM) i9-9960X CPU @ 3.10GHz	NVIDIA GeForce RTX 2080 Ti	62.47	Ubuntu 22.04.4 LTS	0.1.34	gemma:2b	156.48
Linux	Intel(R) Core(TM) i9-9960X CPU @ 3.10GHz	NVIDIA GeForce RTX 2080 Ti	62.47	Ubuntu 22.04.4 LTS	0.1.34	gemma:7b	75.64
Linux	Intel(R) Core(TM) i9-9960X CPU @ 3.10GHz	NVIDIA GeForce RTX 2080 Ti	62.47	Ubuntu 22.04.4 LTS	0.1.34	llava:7b	93.8
Linux	Intel(R) Core(TM) i9-9960X CPU @ 3.10GHz	NVIDIA GeForce RTX 2080 Ti	62.47	Ubuntu 22.04.4 LTS	0.1.34	llava:13b	57.68
Linux	Intel(R) Xeon(R) CPU E5-2680 v2 @ 2.80GHz	no_gpu	31.32	Debian GNU/Linux 12 (bookworm)	0.1.37	mistral:7b	3.53
Linux	Intel(R) Xeon(R) CPU E5-2680 v2 @ 2.80GHz	no_gpu	31.32	Debian GNU/Linux 12 (bookworm)	0.1.37	llama3:8b	3.81
Linux	Intel(R) Xeon(R) CPU E5-2680 v2 @ 2.80GHz	no_gpu	31.32	Debian GNU/Linux 12 (bookworm)	0.1.37	phi3:3.8b	7.12
Linux	Intel(R) Xeon(R) CPU E5-2680 v2 @ 2.80GHz	no_gpu	31.32	Debian GNU/Linux 12 (bookworm)	0.1.37	gemma:2b	8.27
Linux	Intel(R) Xeon(R) CPU E5-2680 v2 @ 2.80GHz	no_gpu	31.32	Debian GNU/Linux 12 (bookworm)	0.1.37	gemma:7b	3.67
Linux	Intel(R) Xeon(R) CPU E5-2680 v2 @ 2.80GHz	no_gpu	31.32	Debian GNU/Linux 12 (bookworm)	0.1.37	llava:7b	3.9
Linux	Intel(R) Xeon(R) CPU E5-2680 v2 @ 2.80GHz	no_gpu	31.32	Debian GNU/Linux 12 (bookworm)	0.1.37	llava:13b	2.54
Linux	AMD Eng Sample: 100-000000059-52_48/35_Y	Vega 20 [Radeon Instinct]	31.23	Ubuntu 22.04.4 LTS	0.1.38	mistral:7b	25.72
Linux	AMD Eng Sample: 100-000000059-52_48/35_Y	Vega 20 [Radeon Instinct]	31.23	Ubuntu 22.04.4 LTS	0.1.38	llama3:8b	23.74
Linux	AMD Eng Sample: 100-000000059-52_48/35_Y	Vega 20 [Radeon Instinct]	31.23	Ubuntu 22.04.4 LTS	0.1.38	phi3:3.8b	33.01
Linux	AMD Eng Sample: 100-000000059-52_48/35_Y	Vega 20 [Radeon Instinct]	31.23	Ubuntu 22.04.4 LTS	0.1.38	gemma:2b	63.76
Linux	AMD Eng Sample: 100-000000059-52_48/35_Y	Vega 20 [Radeon Instinct]	31.23	Ubuntu 22.04.4 LTS	0.1.38	gemma:7b	21.07
Linux	AMD Eng Sample: 100-000000059-52_48/35_Y	Vega 20 [Radeon Instinct]	31.23	Ubuntu 22.04.4 LTS	0.1.38	llava:7b	26.93
Linux	AMD Eng Sample: 100-000000059-52_48/35_Y	Vega 20 [Radeon Instinct]	31.23	Ubuntu 22.04.4 LTS	0.1.38	llava:13b	15.33
Linux	AMD Ryzen 5 7600X 6-Core Processor	NVIDIA GeForce RTX 4090	15.18	Ubuntu 24.04 LTS	0.1.38	mistral:7b	113.28
Linux	AMD Ryzen 5 7600X 6-Core Processor	NVIDIA GeForce RTX 4090	15.18	Ubuntu 24.04 LTS	0.1.38	llama3:8b	99.81
Linux	AMD Ryzen 5 7600X 6-Core Processor	NVIDIA GeForce RTX 4090	15.18	Ubuntu 24.04 LTS	0.1.38	phi3:3.8b	146.42
Linux	AMD Ryzen 5 7600X 6-Core Processor	NVIDIA GeForce RTX 4090	15.18	Ubuntu 24.04 LTS	0.1.38	gemma:2b	178.98
Linux	AMD Ryzen 5 7600X 6-Core Processor	NVIDIA GeForce RTX 4090	15.18	Ubuntu 24.04 LTS	0.1.38	gemma:7b	96.48
Linux	AMD Ryzen 5 7600X 6-Core Processor	NVIDIA GeForce RTX 4090	15.18	Ubuntu 24.04 LTS	0.1.38	llava:7b	113.29
Linux	AMD Ryzen 5 7600X 6-Core Processor	NVIDIA GeForce RTX 4090	15.18	Ubuntu 24.04 LTS	0.1.38	llava:13b	78.79
Linux	AMD Ryzen 5 7600X 6-Core Processor	NVIDIA GeForce RTX 4090	15.18	Ubuntu 24.04 LTS	0.1.38	mistral:7b	118.91
Linux	AMD Ryzen 5 7600X 6-Core Processor	NVIDIA GeForce RTX 4090	15.18	Ubuntu 24.04 LTS	0.1.38	llama3:8b	105.3
Linux	AMD Ryzen 5 7600X 6-Core Processor	NVIDIA GeForce RTX 4090	15.18	Ubuntu 24.04 LTS	0.1.38	phi3:3.8b	152.95
Linux	AMD Ryzen 5 7600X 6-Core Processor	NVIDIA GeForce RTX 4090	15.18	Ubuntu 24.04 LTS	0.1.38	gemma:2b	176.64
Linux	AMD Ryzen 5 7600X 6-Core Processor	NVIDIA GeForce RTX 4090	15.18	Ubuntu 24.04 LTS	0.1.38	gemma:7b	100.23
Linux	AMD Ryzen 5 7600X 6-Core Processor	NVIDIA GeForce RTX 4090	15.18	Ubuntu 24.04 LTS	0.1.38	llava:7b	115.93
Linux	AMD Ryzen 5 7600X 6-Core Processor	NVIDIA GeForce RTX 4090	15.18	Ubuntu 24.04 LTS	0.1.38	llava:13b	80.22
Linux	AMD Ryzen 5 7600X 6-Core Processor	NVIDIA GeForce RTX 4090	15.18	Ubuntu 24.04 LTS	0.1.38	mistral:7b	114
Linux	AMD Ryzen 5 7600X 6-Core Processor	NVIDIA GeForce RTX 4090	15.18	Ubuntu 24.04 LTS	0.1.38	llama3:8b	101.1
Linux	AMD Ryzen 5 7600X 6-Core Processor	NVIDIA GeForce RTX 4090	15.18	Ubuntu 24.04 LTS	0.1.38	phi3:3.8b	146.06
Linux	AMD Ryzen 5 7600X 6-Core Processor	NVIDIA GeForce RTX 4090	15.18	Ubuntu 24.04 LTS	0.1.38	gemma:2b	172.97
Linux	AMD Ryzen 5 7600X 6-Core Processor	NVIDIA GeForce RTX 4090	15.18	Ubuntu 24.04 LTS	0.1.38	gemma:7b	97.91
Linux	AMD Ryzen 5 7600X 6-Core Processor	NVIDIA GeForce RTX 4090	15.18	Ubuntu 24.04 LTS	0.1.38	llava:7b	115.38
Linux	AMD Ryzen 5 7600X 6-Core Processor	NVIDIA GeForce RTX 4090	15.18	Ubuntu 24.04 LTS	0.1.38	llava:13b	80.29
Linux	AMD Ryzen 5 7600X 6-Core Processor	NVIDIA GeForce RTX 4070 SUPER	15.18	Ubuntu 24.04 LTS	0.1.38	mistral:7b	81.94
Linux	AMD Ryzen 5 7600X 6-Core Processor	NVIDIA GeForce RTX 4070 SUPER	15.18	Ubuntu 24.04 LTS	0.1.38	llama3:8b	72.64
Linux	AMD Ryzen 5 7600X 6-Core Processor	NVIDIA GeForce RTX 4070 SUPER	15.18	Ubuntu 24.04 LTS	0.1.38	phi3:3.8b	117.62
Linux	AMD Ryzen 5 7600X 6-Core Processor	NVIDIA GeForce RTX 4070 SUPER	15.18	Ubuntu 24.04 LTS	0.1.38	gemma:2b	133.82
Linux	AMD Ryzen 5 7600X 6-Core Processor	NVIDIA GeForce RTX 4070 SUPER	15.18	Ubuntu 24.04 LTS	0.1.38	gemma:7b	68.93
Linux	AMD Ryzen 5 7600X 6-Core Processor	NVIDIA GeForce RTX 4070 SUPER	15.18	Ubuntu 24.04 LTS	0.1.38	llava:7b	80.61
Linux	AMD Ryzen 5 7600X 6-Core Processor	NVIDIA GeForce RTX 4070 SUPER	15.18	Ubuntu 24.04 LTS	0.1.38	llava:13b	51.23
Linux	AMD Ryzen 5 7600X 6-Core Processor	NVIDIA GeForce RTX 4070 SUPER	15.18	Ubuntu 24.04 LTS	0.1.38	mistral:7b	80.75
Linux	AMD Ryzen 5 7600X 6-Core Processor	NVIDIA GeForce RTX 4070 SUPER	15.18	Ubuntu 24.04 LTS	0.1.38	llama3:8b	73.82
Linux	AMD Ryzen 5 7600X 6-Core Processor	NVIDIA GeForce RTX 4070 SUPER	15.18	Ubuntu 24.04 LTS	0.1.38	phi3:3.8b	118.9
Linux	AMD Ryzen 5 7600X 6-Core Processor	NVIDIA GeForce RTX 4070 SUPER	15.18	Ubuntu 24.04 LTS	0.1.38	gemma:2b	138.07
Linux	AMD Ryzen 5 7600X 6-Core Processor	NVIDIA GeForce RTX 4070 SUPER	15.18	Ubuntu 24.04 LTS	0.1.38	gemma:7b	69.27
Linux	AMD Ryzen 5 7600X 6-Core Processor	NVIDIA GeForce RTX 4070 SUPER	15.18	Ubuntu 24.04 LTS	0.1.38	llava:7b	83.63
Linux	AMD Ryzen 5 7600X 6-Core Processor	NVIDIA GeForce RTX 4070 SUPER	15.18	Ubuntu 24.04 LTS	0.1.38	llava:13b	51.78
Linux	AMD Ryzen 5 7600X 6-Core Processor	NVIDIA GeForce RTX 4070 SUPER	15.18	Ubuntu 24.04 LTS	0.1.38	mistral:7b	83.14
Linux	AMD Ryzen 5 7600X 6-Core Processor	NVIDIA GeForce RTX 4070 SUPER	15.18	Ubuntu 24.04 LTS	0.1.38	llama3:8b	73.75
Linux	AMD Ryzen 5 7600X 6-Core Processor	NVIDIA GeForce RTX 4070 SUPER	15.18	Ubuntu 24.04 LTS	0.1.38	phi3:3.8b	119.83
Linux	AMD Ryzen 5 7600X 6-Core Processor	NVIDIA GeForce RTX 4070 SUPER	15.18	Ubuntu 24.04 LTS	0.1.38	gemma:2b	135.59
Linux	AMD Ryzen 5 7600X 6-Core Processor	NVIDIA GeForce RTX 4070 SUPER	15.18	Ubuntu 24.04 LTS	0.1.38	gemma:7b	70.15
Linux	AMD Ryzen 5 7600X 6-Core Processor	NVIDIA GeForce RTX 4070 SUPER	15.18	Ubuntu 24.04 LTS	0.1.38	llava:7b	80.95
Linux	AMD Ryzen 5 7600X 6-Core Processor	NVIDIA GeForce RTX 4070 SUPER	15.18	Ubuntu 24.04 LTS	0.1.38	llava:13b	52.1
Linux	AMD Ryzen 5 7600X 6-Core Processor	NVIDIA GeForce RTX 4060 Ti	15.18	Ubuntu 24.04 LTS	0.1.38	mistral:7b	55
Linux	AMD Ryzen 5 7600X 6-Core Processor	NVIDIA GeForce RTX 4060 Ti	15.18	Ubuntu 24.04 LTS	0.1.38	llama3:8b	49.51
Linux	AMD Ryzen 5 7600X 6-Core Processor	NVIDIA GeForce RTX 4060 Ti	15.18	Ubuntu 24.04 LTS	0.1.38	phi3:3.8b	84.71
Linux	AMD Ryzen 5 7600X 6-Core Processor	NVIDIA GeForce RTX 4060 Ti	15.18	Ubuntu 24.04 LTS	0.1.38	gemma:2b	103.94
Linux	AMD Ryzen 5 7600X 6-Core Processor	NVIDIA GeForce RTX 4060 Ti	15.18	Ubuntu 24.04 LTS	0.1.38	gemma:7b	46.12
Linux	AMD Ryzen 5 7600X 6-Core Processor	NVIDIA GeForce RTX 4060 Ti	15.18	Ubuntu 24.04 LTS	0.1.38	llava:7b	55.57
Linux	AMD Ryzen 5 7600X 6-Core Processor	NVIDIA GeForce RTX 4060 Ti	15.18	Ubuntu 24.04 LTS	0.1.38	llava:13b	33.7
Linux	AMD Ryzen 5 7600X 6-Core Processor	NVIDIA GeForce RTX 4060 Ti	15.18	Ubuntu 24.04 LTS	0.1.38	mistral:7b	54.55
Linux	AMD Ryzen 5 7600X 6-Core Processor	NVIDIA GeForce RTX 4060 Ti	15.18	Ubuntu 24.04 LTS	0.1.38	llama3:8b	50.12
Linux	AMD Ryzen 5 7600X 6-Core Processor	NVIDIA GeForce RTX 4060 Ti	15.18	Ubuntu 24.04 LTS	0.1.38	phi3:3.8b	83.68
Linux	AMD Ryzen 5 7600X 6-Core Processor	NVIDIA GeForce RTX 4060 Ti	15.18	Ubuntu 24.04 LTS	0.1.38	gemma:2b	103.03
Linux	AMD Ryzen 5 7600X 6-Core Processor	NVIDIA GeForce RTX 4060 Ti	15.18	Ubuntu 24.04 LTS	0.1.38	gemma:7b	46.33
Linux	AMD Ryzen 5 7600X 6-Core Processor	NVIDIA GeForce RTX 4060 Ti	15.18	Ubuntu 24.04 LTS	0.1.38	llava:7b	54.92
Linux	AMD Ryzen 5 7600X 6-Core Processor	NVIDIA GeForce RTX 4060 Ti	15.18	Ubuntu 24.04 LTS	0.1.38	llava:13b	33.51
Linux	AMD Ryzen 5 7600X 6-Core Processor	NVIDIA GeForce RTX 4060 Ti	15.18	Ubuntu 24.04 LTS	0.1.38	mistral:7b	55.21
Linux	AMD Ryzen 5 7600X 6-Core Processor	NVIDIA GeForce RTX 4060 Ti	15.18	Ubuntu 24.04 LTS	0.1.38	llama3:8b	51.17
Linux	AMD Ryzen 5 7600X 6-Core Processor	NVIDIA GeForce RTX 4060 Ti	15.18	Ubuntu 24.04 LTS	0.1.38	phi3:3.8b	84.93
Linux	AMD Ryzen 5 7600X 6-Core Processor	NVIDIA GeForce RTX 4060 Ti	15.18	Ubuntu 24.04 LTS	0.1.38	gemma:2b	104.96
Linux	AMD Ryzen 5 7600X 6-Core Processor	NVIDIA GeForce RTX 4060 Ti	15.18	Ubuntu 24.04 LTS	0.1.38	gemma:7b	46.48
Linux	AMD Ryzen 5 7600X 6-Core Processor	NVIDIA GeForce RTX 4060 Ti	15.18	Ubuntu 24.04 LTS	0.1.38	llava:7b	55.76
Linux	AMD Ryzen 5 7600X 6-Core Processor	NVIDIA GeForce RTX 4060 Ti	15.18	Ubuntu 24.04 LTS	0.1.38	llava:13b	33.96
Linux	AMD Ryzen 5 7600X 6-Core Processor	NVIDIA GeForce RTX 4060	15.18	Ubuntu 24.04 LTS	0.1.38	mistral:7b	50.57
Linux	AMD Ryzen 5 7600X 6-Core Processor	NVIDIA GeForce RTX 4060	15.18	Ubuntu 24.04 LTS	0.1.38	llama3:8b	47.07
Linux	AMD Ryzen 5 7600X 6-Core Processor	NVIDIA GeForce RTX 4060	15.18	Ubuntu 24.04 LTS	0.1.38	phi3:3.8b	80.44
Linux	AMD Ryzen 5 7600X 6-Core Processor	NVIDIA GeForce RTX 4060	15.18	Ubuntu 24.04 LTS	0.1.38	gemma:2b	96.87
Linux	AMD Ryzen 5 7600X 6-Core Processor	NVIDIA GeForce RTX 4060	15.18	Ubuntu 24.04 LTS	0.1.38	gemma:7b	42.95
Linux	AMD Ryzen 5 7600X 6-Core Processor	NVIDIA GeForce RTX 4060	15.18	Ubuntu 24.04 LTS	0.1.38	llava:7b	52.95
Linux	AMD Ryzen 5 7600X 6-Core Processor	NVIDIA GeForce RTX 4060	15.18	Ubuntu 24.04 LTS	0.1.38	llava:13b	9.15
Linux	AMD Ryzen 5 7600X 6-Core Processor	NVIDIA GeForce RTX 4060	15.18	Ubuntu 24.04 LTS	0.1.38	mistral:7b	51.97
Linux	AMD Ryzen 5 7600X 6-Core Processor	NVIDIA GeForce RTX 4060	15.18	Ubuntu 24.04 LTS	0.1.38	llama3:8b	47.24
Linux	AMD Ryzen 5 7600X 6-Core Processor	NVIDIA GeForce RTX 4060	15.18	Ubuntu 24.04 LTS	0.1.38	phi3:3.8b	79.74
Linux	AMD Ryzen 5 7600X 6-Core Processor	NVIDIA GeForce RTX 4060	15.18	Ubuntu 24.04 LTS	0.1.38	gemma:2b	97.26
Linux	AMD Ryzen 5 7600X 6-Core Processor	NVIDIA GeForce RTX 4060	15.18	Ubuntu 24.04 LTS	0.1.38	gemma:7b	43.1
Linux	AMD Ryzen 5 7600X 6-Core Processor	NVIDIA GeForce RTX 4060	15.18	Ubuntu 24.04 LTS	0.1.38	llava:7b	52.33
Linux	AMD Ryzen 5 7600X 6-Core Processor	NVIDIA GeForce RTX 4060	15.18	Ubuntu 24.04 LTS	0.1.38	llava:13b	8.89
Linux	AMD Ryzen 5 7600X 6-Core Processor	NVIDIA GeForce RTX 4060	15.18	Ubuntu 24.04 LTS	0.1.38	mistral:7b	51.96
Linux	AMD Ryzen 5 7600X 6-Core Processor	NVIDIA GeForce RTX 4060	15.18	Ubuntu 24.04 LTS	0.1.38	llama3:8b	46.99
Linux	AMD Ryzen 5 7600X 6-Core Processor	NVIDIA GeForce RTX 4060	15.18	Ubuntu 24.04 LTS	0.1.38	phi3:3.8b	80.88
Linux	AMD Ryzen 5 7600X 6-Core Processor	NVIDIA GeForce RTX 4060	15.18	Ubuntu 24.04 LTS	0.1.38	gemma:2b	94.39
Linux	AMD Ryzen 5 7600X 6-Core Processor	NVIDIA GeForce RTX 4060	15.18	Ubuntu 24.04 LTS	0.1.38	gemma:7b	42.96
Linux	AMD Ryzen 5 7600X 6-Core Processor	NVIDIA GeForce RTX 4060	15.18	Ubuntu 24.04 LTS	0.1.38	llava:7b	51.57
Linux	AMD Ryzen 5 7600X 6-Core Processor	NVIDIA GeForce RTX 4060	15.18	Ubuntu 24.04 LTS	0.1.38	llava:13b	8.85
Linux	AMD Ryzen 5 7600X 6-Core Processor	NVIDIA GeForce RTX 4060	15.18	Ubuntu 24.04 LTS	0.1.38	mistral:7b	51.15
Linux	AMD Ryzen 5 7600X 6-Core Processor	NVIDIA GeForce RTX 4060	15.18	Ubuntu 24.04 LTS	0.1.38	llama3:8b	47.1
Linux	AMD Ryzen 5 7600X 6-Core Processor	NVIDIA GeForce RTX 4060	15.18	Ubuntu 24.04 LTS	0.1.38	phi3:3.8b	79.75
Linux	AMD Ryzen 5 7600X 6-Core Processor	NVIDIA GeForce RTX 4060	15.18	Ubuntu 24.04 LTS	0.1.38	gemma:2b	95.94
Linux	AMD Ryzen 5 7600X 6-Core Processor	NVIDIA GeForce RTX 4060	15.18	Ubuntu 24.04 LTS	0.1.38	gemma:7b	43.1
Linux	AMD Ryzen 5 7600X 6-Core Processor	NVIDIA GeForce RTX 4060	15.18	Ubuntu 24.04 LTS	0.1.38	llava:7b	52.73
Linux	AMD Ryzen 5 7600X 6-Core Processor	NVIDIA GeForce RTX 4060	15.18	Ubuntu 24.04 LTS	0.1.38	llava:13b	31.05
Linux	AMD Ryzen 5 7600X 6-Core Processor	NVIDIA GeForce RTX 4060	15.18	Ubuntu 24.04 LTS	0.1.38	mistral:7b	51.42
Linux	AMD Ryzen 5 7600X 6-Core Processor	NVIDIA GeForce RTX 4060	15.18	Ubuntu 24.04 LTS	0.1.38	llama3:8b	46.69
Linux	AMD Ryzen 5 7600X 6-Core Processor	NVIDIA GeForce RTX 4060	15.18	Ubuntu 24.04 LTS	0.1.38	phi3:3.8b	80.54
Linux	AMD Ryzen 5 7600X 6-Core Processor	NVIDIA GeForce RTX 4060	15.18	Ubuntu 24.04 LTS	0.1.38	gemma:2b	87.69
Linux	AMD Ryzen 5 7600X 6-Core Processor	NVIDIA GeForce RTX 4060	15.18	Ubuntu 24.04 LTS	0.1.38	gemma:7b	43.04
Linux	AMD Ryzen 5 7600X 6-Core Processor	NVIDIA GeForce RTX 4060	15.18	Ubuntu 24.04 LTS	0.1.38	llava:7b	51.52
Linux	AMD Ryzen 5 7600X 6-Core Processor	NVIDIA GeForce RTX 4060	15.18	Ubuntu 24.04 LTS	0.1.38	llava:13b	30.86
Linux	AMD Ryzen 5 7600X 6-Core Processor	NVIDIA GeForce RTX 4060	15.18	Ubuntu 24.04 LTS	0.1.38	mistral:7b	51.96
Linux	AMD Ryzen 5 7600X 6-Core Processor	NVIDIA GeForce RTX 4060	15.18	Ubuntu 24.04 LTS	0.1.38	llama3:8b	47.26
Linux	AMD Ryzen 5 7600X 6-Core Processor	NVIDIA GeForce RTX 4060	15.18	Ubuntu 24.04 LTS	0.1.38	phi3:3.8b	80.5
Linux	AMD Ryzen 5 7600X 6-Core Processor	NVIDIA GeForce RTX 4060	15.18	Ubuntu 24.04 LTS	0.1.38	gemma:2b	94.25
Linux	AMD Ryzen 5 7600X 6-Core Processor	NVIDIA GeForce RTX 4060	15.18	Ubuntu 24.04 LTS	0.1.38	gemma:7b	43.14
Linux	AMD Ryzen 5 7600X 6-Core Processor	NVIDIA GeForce RTX 4060	15.18	Ubuntu 24.04 LTS	0.1.38	llava:7b	52.63
Linux	AMD Ryzen 5 7600X 6-Core Processor	NVIDIA GeForce RTX 4060	15.18	Ubuntu 24.04 LTS	0.1.38	llava:13b	31.03
Linux	AMD Ryzen 5 7600X 6-Core Processor	NVIDIA GeForce RTX 4080 SUPER	15.18	Ubuntu 24.04 LTS	0.1.38	mistral:7b	99.6
Linux	AMD Ryzen 5 7600X 6-Core Processor	NVIDIA GeForce RTX 4080 SUPER	15.18	Ubuntu 24.04 LTS	0.1.38	llama3:8b	90.22
Linux	AMD Ryzen 5 7600X 6-Core Processor	NVIDIA GeForce RTX 4080 SUPER	15.18	Ubuntu 24.04 LTS	0.1.38	phi3:3.8b	137.47
Linux	AMD Ryzen 5 7600X 6-Core Processor	NVIDIA GeForce RTX 4080 SUPER	15.18	Ubuntu 24.04 LTS	0.1.38	gemma:2b	167.62
Linux	AMD Ryzen 5 7600X 6-Core Processor	NVIDIA GeForce RTX 4080 SUPER	15.18	Ubuntu 24.04 LTS	0.1.38	gemma:7b	85.13
Linux	AMD Ryzen 5 7600X 6-Core Processor	NVIDIA GeForce RTX 4080 SUPER	15.18	Ubuntu 24.04 LTS	0.1.38	llava:7b	95.31
Linux	AMD Ryzen 5 7600X 6-Core Processor	NVIDIA GeForce RTX 4080 SUPER	15.18	Ubuntu 24.04 LTS	0.1.38	llava:13b	64.89
Linux	AMD Ryzen 5 7600X 6-Core Processor	NVIDIA GeForce RTX 4080 SUPER	15.18	Ubuntu 24.04 LTS	0.1.38	mistral:7b	100.99
Linux	AMD Ryzen 5 7600X 6-Core Processor	NVIDIA GeForce RTX 4080 SUPER	15.18	Ubuntu 24.04 LTS	0.1.38	llama3:8b	90.84
Linux	AMD Ryzen 5 7600X 6-Core Processor	NVIDIA GeForce RTX 4080 SUPER	15.18	Ubuntu 24.04 LTS	0.1.38	phi3:3.8b	138.53
Linux	AMD Ryzen 5 7600X 6-Core Processor	NVIDIA GeForce RTX 4080 SUPER	15.18	Ubuntu 24.04 LTS	0.1.38	gemma:2b	160.23
Linux	AMD Ryzen 5 7600X 6-Core Processor	NVIDIA GeForce RTX 4080 SUPER	15.18	Ubuntu 24.04 LTS	0.1.38	gemma:7b	86.55
Linux	AMD Ryzen 5 7600X 6-Core Processor	NVIDIA GeForce RTX 4080 SUPER	15.18	Ubuntu 24.04 LTS	0.1.38	llava:7b	98.59
Linux	AMD Ryzen 5 7600X 6-Core Processor	NVIDIA GeForce RTX 4080 SUPER	15.18	Ubuntu 24.04 LTS	0.1.38	llava:13b	67.73
Linux	AMD Ryzen 5 7600X 6-Core Processor	NVIDIA GeForce RTX 4080 SUPER	15.18	Ubuntu 24.04 LTS	0.1.38	mistral:7b	103.17
Linux	AMD Ryzen 5 7600X 6-Core Processor	NVIDIA GeForce RTX 4080 SUPER	15.18	Ubuntu 24.04 LTS	0.1.38	llama3:8b	91.94
Linux	AMD Ryzen 5 7600X 6-Core Processor	NVIDIA GeForce RTX 4080 SUPER	15.18	Ubuntu 24.04 LTS	0.1.38	phi3:3.8b	140.17
Linux	AMD Ryzen 5 7600X 6-Core Processor	NVIDIA GeForce RTX 4080 SUPER	15.18	Ubuntu 24.04 LTS	0.1.38	gemma:2b	164.2
Linux	AMD Ryzen 5 7600X 6-Core Processor	NVIDIA GeForce RTX 4080 SUPER	15.18	Ubuntu 24.04 LTS	0.1.38	gemma:7b	86.93
Linux	AMD Ryzen 5 7600X 6-Core Processor	NVIDIA GeForce RTX 4080 SUPER	15.18	Ubuntu 24.04 LTS	0.1.38	llava:7b	104.2
Linux	AMD Ryzen 5 7600X 6-Core Processor	NVIDIA GeForce RTX 4080 SUPER	15.18	Ubuntu 24.04 LTS	0.1.38	llava:13b	68.1
Linux	12th Gen Intel(R) Core(TM) i5-12400	Intel Corporation	15.32	Ubuntu 22.04.4 LTS	0.1.34	mistral:7b	9.39
Linux	12th Gen Intel(R) Core(TM) i5-12400	Intel Corporation	15.32	Ubuntu 22.04.4 LTS	0.1.34	llama3:8b	8.67
Linux	12th Gen Intel(R) Core(TM) i5-12400	Intel Corporation	15.32	Ubuntu 22.04.4 LTS	0.1.34	phi3:3.8b	15.97
Linux	12th Gen Intel(R) Core(TM) i5-12400	Intel Corporation	15.32	Ubuntu 22.04.4 LTS	0.1.34	gemma:2b	22.15
Linux	12th Gen Intel(R) Core(TM) i5-12400	Intel Corporation	15.32	Ubuntu 22.04.4 LTS	0.1.34	gemma:7b	7.72
Linux	12th Gen Intel(R) Core(TM) i5-12400	Intel Corporation	15.32	Ubuntu 22.04.4 LTS	0.1.34	llava:7b	9.56
Linux	12th Gen Intel(R) Core(TM) i5-12400	Intel Corporation	15.32	Ubuntu 22.04.4 LTS	0.1.34	llava:13b	5.49
Linux	Intel(R) Core(TM) i9-9900K CPU @ 3.60GHz	NVIDIA GeForce RTX 3070 Ti	62.66	Ubuntu 22.04.4 LTS	0.1.38	mistral:7b	92.58
Linux	Intel(R) Core(TM) i9-9900K CPU @ 3.60GHz	NVIDIA GeForce RTX 3070 Ti	62.66	Ubuntu 22.04.4 LTS	0.1.38	llama3:8b	77.98
Linux	Intel(R) Core(TM) i9-9900K CPU @ 3.60GHz	NVIDIA GeForce RTX 3070 Ti	62.66	Ubuntu 22.04.4 LTS	0.1.38	phi3:3.8b	127.19
Linux	Intel(R) Core(TM) i9-9900K CPU @ 3.60GHz	NVIDIA GeForce RTX 3070 Ti	62.66	Ubuntu 22.04.4 LTS	0.1.38	gemma:2b	164.71
Linux	Intel(R) Core(TM) i9-9900K CPU @ 3.60GHz	NVIDIA GeForce RTX 3070 Ti	62.66	Ubuntu 22.04.4 LTS	0.1.38	gemma:7b	77.68
Linux	Intel(R) Core(TM) i9-9900K CPU @ 3.60GHz	NVIDIA GeForce RTX 3070 Ti	62.66	Ubuntu 22.04.4 LTS	0.1.38	llava:7b	96.28
Linux	Intel(R) Core(TM) i9-9900K CPU @ 3.60GHz	NVIDIA GeForce RTX 3070 Ti	62.66	Ubuntu 22.04.4 LTS	0.1.38	llava:13b	13.82
Linux	Intel(R) Xeon(R) CPU E5-1630 v3 @ 3.70GHz	no_gpu	62.87	Debian GNU/Linux 10 (buster)	0.1.38	mistral:7b	5.59
Linux	Intel(R) Xeon(R) CPU E5-1630 v3 @ 3.70GHz	no_gpu	62.87	Debian GNU/Linux 10 (buster)	0.1.38	llama3:8b	5.24
Linux	Intel(R) Xeon(R) CPU E5-1630 v3 @ 3.70GHz	no_gpu	62.87	Debian GNU/Linux 10 (buster)	0.1.38	phi3:3.8b	11.21
Linux	Intel(R) Xeon(R) CPU E5-1630 v3 @ 3.70GHz	no_gpu	62.87	Debian GNU/Linux 10 (buster)	0.1.38	gemma:2b	13.41
Linux	Intel(R) Xeon(R) CPU E5-1630 v3 @ 3.70GHz	no_gpu	62.87	Debian GNU/Linux 10 (buster)	0.1.38	gemma:7b	4.61
Linux	Intel(R) Xeon(R) CPU E5-1630 v3 @ 3.70GHz	no_gpu	62.87	Debian GNU/Linux 10 (buster)	0.1.38	llava:7b	5.83
Linux	Intel(R) Xeon(R) CPU E5-1630 v3 @ 3.70GHz	no_gpu	62.87	Debian GNU/Linux 10 (buster)	0.1.38	llava:13b	3.37
Linux	AMD Ryzen 7 2700 Eight-Core Processor	NVIDIA GeForce RTX 4060 Ti	62.71	Ubuntu 22.04.4 LTS	0.1.38	mistral:7b	51.65
Linux	AMD Ryzen 7 2700 Eight-Core Processor	NVIDIA GeForce RTX 4060 Ti	62.71	Ubuntu 22.04.4 LTS	0.1.38	llama3:8b	45.03
Linux	AMD Ryzen 7 2700 Eight-Core Processor	NVIDIA GeForce RTX 4060 Ti	62.71	Ubuntu 22.04.4 LTS	0.1.38	phi3:3.8b	77.23
Linux	AMD Ryzen 7 2700 Eight-Core Processor	NVIDIA GeForce RTX 4060 Ti	62.71	Ubuntu 22.04.4 LTS	0.1.38	gemma:2b	92.94
Linux	AMD Ryzen 7 2700 Eight-Core Processor	NVIDIA GeForce RTX 4060 Ti	62.71	Ubuntu 22.04.4 LTS	0.1.38	gemma:7b	39.6
Linux	AMD Ryzen 7 2700 Eight-Core Processor	NVIDIA GeForce RTX 4060 Ti	62.71	Ubuntu 22.04.4 LTS	0.1.38	llava:7b	51.5
Linux	AMD Ryzen 7 2700 Eight-Core Processor	NVIDIA GeForce RTX 4060 Ti	62.71	Ubuntu 22.04.4 LTS	0.1.38	llava:13b	31.33
Linux	Intel(R) Core(TM) Ultra 5 135HL	Intel Corporation	15.30	Ubuntu 22.04.4 LTS	0.0.0	mistral:7b	13.5
Linux	Intel(R) Core(TM) Ultra 5 135HL	Intel Corporation	15.30	Ubuntu 22.04.4 LTS	0.0.0	llama3:8b	11.61
Linux	Intel(R) Core(TM) Ultra 5 135HL	Intel Corporation	15.30	Ubuntu 22.04.4 LTS	0.0.0	phi3:3.8b	9.45
Linux	Intel(R) Core(TM) Ultra 5 135HL	Intel Corporation	15.30	Ubuntu 22.04.4 LTS	0.0.0	gemma:2b	20.67
Linux	Intel(R) Core(TM) Ultra 5 135HL	Intel Corporation	15.30	Ubuntu 22.04.4 LTS	0.0.0	gemma:7b	8.04
Linux	Intel(R) Core(TM) Ultra 5 135HL	Intel Corporation	15.30	Ubuntu 22.04.4 LTS	0.0.0	llava:7b	13.55
Linux	Intel(R) Core(TM) Ultra 5 135HL	Intel Corporation	15.30	Ubuntu 22.04.4 LTS	0.0.0	llava:13b	7.22
Linux	AMD Ryzen Threadripper PRO 5975WX 32-Cores	DG2 [Arc A380]	251.52	Arch Linux	0.1.38	mistral:7b	23.73
Linux	AMD Ryzen Threadripper PRO 5975WX 32-Cores	DG2 [Arc A380]	251.52	Arch Linux	0.1.38	llama3:8b	22.07
Linux	AMD Ryzen Threadripper PRO 5975WX 32-Cores	DG2 [Arc A380]	251.52	Arch Linux	0.1.38	phi3:3.8b	35.38
Linux	AMD Ryzen Threadripper PRO 5975WX 32-Cores	DG2 [Arc A380]	251.52	Arch Linux	0.1.38	gemma:2b	50.9
Linux	AMD Ryzen Threadripper PRO 5975WX 32-Cores	DG2 [Arc A380]	251.52	Arch Linux	0.1.38	gemma:7b	19.14
Linux	AMD Ryzen Threadripper PRO 5975WX 32-Cores	DG2 [Arc A380]	251.52	Arch Linux	0.1.38	llava:7b	24.73
Linux	AMD Ryzen Threadripper PRO 5975WX 32-Cores	DG2 [Arc A380]	251.52	Arch Linux	0.1.38	llava:13b	14.7
Linux	AMD Ryzen 5 7600X 6-Core Processor	Advanced Micro Devices, Inc. [AMD/ATI]	30.46	Ubuntu 22.04.4 LTS	0.1.38	mistral:7b	107.38
Linux	AMD Ryzen 5 7600X 6-Core Processor	Advanced Micro Devices, Inc. [AMD/ATI]	30.46	Ubuntu 22.04.4 LTS	0.1.38	llama3:8b	98.53
Linux	AMD Ryzen 5 7600X 6-Core Processor	Advanced Micro Devices, Inc. [AMD/ATI]	30.46	Ubuntu 22.04.4 LTS	0.1.38	phi3:3.8b	123.94
Linux	AMD Ryzen 5 7600X 6-Core Processor	Advanced Micro Devices, Inc. [AMD/ATI]	30.46	Ubuntu 22.04.4 LTS	0.1.38	gemma:2b	196.77
Linux	AMD Ryzen 5 7600X 6-Core Processor	Advanced Micro Devices, Inc. [AMD/ATI]	30.46	Ubuntu 22.04.4 LTS	0.1.38	gemma:7b	92.26
Linux	AMD Ryzen 5 7600X 6-Core Processor	Advanced Micro Devices, Inc. [AMD/ATI]	30.46	Ubuntu 22.04.4 LTS	0.1.38	llava:7b	109.92
Linux	AMD Ryzen 5 7600X 6-Core Processor	Advanced Micro Devices, Inc. [AMD/ATI]	30.46	Ubuntu 22.04.4 LTS	0.1.38	llava:13b	71.32
Linux	AMD Ryzen 5 7600X 6-Core Processor	Advanced Micro Devices, Inc. [AMD/ATI]	30.46	Ubuntu 22.04.4 LTS	0.1.38	mistral:7b	107.91
Linux	AMD Ryzen 5 7600X 6-Core Processor	Advanced Micro Devices, Inc. [AMD/ATI]	30.46	Ubuntu 22.04.4 LTS	0.1.38	llama3:8b	98.34
Linux	AMD Ryzen 5 7600X 6-Core Processor	Advanced Micro Devices, Inc. [AMD/ATI]	30.46	Ubuntu 22.04.4 LTS	0.1.38	phi3:3.8b	122.52
Linux	AMD Ryzen 5 7600X 6-Core Processor	Advanced Micro Devices, Inc. [AMD/ATI]	30.46	Ubuntu 22.04.4 LTS	0.1.38	gemma:2b	196.59
Linux	AMD Ryzen 5 7600X 6-Core Processor	Advanced Micro Devices, Inc. [AMD/ATI]	30.46	Ubuntu 22.04.4 LTS	0.1.38	gemma:7b	92.22
Linux	AMD Ryzen 5 7600X 6-Core Processor	Advanced Micro Devices, Inc. [AMD/ATI]	30.46	Ubuntu 22.04.4 LTS	0.1.38	llava:7b	110.28
Linux	AMD Ryzen 5 7600X 6-Core Processor	Advanced Micro Devices, Inc. [AMD/ATI]	30.46	Ubuntu 22.04.4 LTS	0.1.38	llava:13b	71.14
Linux	AMD Ryzen 5 7600X 6-Core Processor	Advanced Micro Devices, Inc. [AMD/ATI]	30.46	Ubuntu 22.04.4 LTS	0.1.38	mistral:7b	107.44
Linux	AMD Ryzen 5 7600X 6-Core Processor	Advanced Micro Devices, Inc. [AMD/ATI]	30.46	Ubuntu 22.04.4 LTS	0.1.38	llama3:8b	98.37
Linux	AMD Ryzen 5 7600X 6-Core Processor	Advanced Micro Devices, Inc. [AMD/ATI]	30.46	Ubuntu 22.04.4 LTS	0.1.38	phi3:3.8b	122.38
Linux	AMD Ryzen 5 7600X 6-Core Processor	Advanced Micro Devices, Inc. [AMD/ATI]	30.46	Ubuntu 22.04.4 LTS	0.1.38	gemma:2b	196.65
Linux	AMD Ryzen 5 7600X 6-Core Processor	Advanced Micro Devices, Inc. [AMD/ATI]	30.46	Ubuntu 22.04.4 LTS	0.1.38	gemma:7b	92.02
Linux	AMD Ryzen 5 7600X 6-Core Processor	Advanced Micro Devices, Inc. [AMD/ATI]	30.46	Ubuntu 22.04.4 LTS	0.1.38	llava:7b	109.76
Linux	AMD Ryzen 5 7600X 6-Core Processor	Advanced Micro Devices, Inc. [AMD/ATI]	30.46	Ubuntu 22.04.4 LTS	0.1.38	llava:13b	70.8
Linux	11th Gen Intel(R) Core(TM) i7-11800H @ 2.30GHz	NVIDIA GeForce RTX 3070	62.61	Ubuntu 22.04.4 LTS	0.1.38	mistral:7b	75.95
Linux	11th Gen Intel(R) Core(TM) i7-11800H @ 2.30GHz	NVIDIA GeForce RTX 3070	62.61	Ubuntu 22.04.4 LTS	0.1.38	llama3:8b	66.73
Linux	11th Gen Intel(R) Core(TM) i7-11800H @ 2.30GHz	NVIDIA GeForce RTX 3070	62.61	Ubuntu 22.04.4 LTS	0.1.38	phi3:3.8b	109.69
Linux	11th Gen Intel(R) Core(TM) i7-11800H @ 2.30GHz	NVIDIA GeForce RTX 3070	62.61	Ubuntu 22.04.4 LTS	0.1.38	gemma:2b	135.32
Linux	11th Gen Intel(R) Core(TM) i7-11800H @ 2.30GHz	NVIDIA GeForce RTX 3070	62.61	Ubuntu 22.04.4 LTS	0.1.38	gemma:7b	61.38
Linux	11th Gen Intel(R) Core(TM) i7-11800H @ 2.30GHz	NVIDIA GeForce RTX 3070	62.61	Ubuntu 22.04.4 LTS	0.1.38	llava:7b	79.08
Linux	11th Gen Intel(R) Core(TM) i7-11800H @ 2.30GHz	NVIDIA GeForce RTX 3070	62.61	Ubuntu 22.04.4 LTS	0.1.38	llava:13b	11.92
Linux	Intel(R) Xeon(R) Gold 6130 CPU @ 2.10GHz	Tesla V100-SXM2-32GB	29.36	Ubuntu 22.04.4 LTS	0.1.38	mistral:7b	95.27
Linux	Intel(R) Xeon(R) Gold 6130 CPU @ 2.10GHz	Tesla V100-SXM2-32GB	29.36	Ubuntu 22.04.4 LTS	0.1.38	llama3:8b	71.29
Linux	Intel(R) Xeon(R) Gold 6130 CPU @ 2.10GHz	Tesla V100-SXM2-32GB	29.36	Ubuntu 22.04.4 LTS	0.1.38	phi3:3.8b	116.66
Linux	Intel(R) Xeon(R) Gold 6130 CPU @ 2.10GHz	Tesla V100-SXM2-32GB	29.36	Ubuntu 22.04.4 LTS	0.1.38	gemma:2b	129.65
Linux	Intel(R) Xeon(R) Gold 6130 CPU @ 2.10GHz	Tesla V100-SXM2-32GB	29.36	Ubuntu 22.04.4 LTS	0.1.38	gemma:7b	75.45
Linux	Intel(R) Xeon(R) Gold 6130 CPU @ 2.10GHz	Tesla V100-SXM2-32GB	29.36	Ubuntu 22.04.4 LTS	0.1.38	llava:7b	99
Linux	Intel(R) Xeon(R) Gold 6130 CPU @ 2.10GHz	Tesla V100-SXM2-32GB	29.36	Ubuntu 22.04.4 LTS	0.1.38	llava:13b	66.47
Linux	AMD Ryzen 7 5700G with Radeon Graphics	NVIDIA GeForce RTX 3060	31.32	Ubuntu 22.04.4 LTS	0.1.38	mistral:7b	63
Linux	AMD Ryzen 7 5700G with Radeon Graphics	NVIDIA GeForce RTX 3060	31.32	Ubuntu 22.04.4 LTS	0.1.38	llama3:8b	55.91
Linux	AMD Ryzen 7 5700G with Radeon Graphics	NVIDIA GeForce RTX 3060	31.32	Ubuntu 22.04.4 LTS	0.1.38	phi3:3.8b	90.36
Linux	AMD Ryzen 7 5700G with Radeon Graphics	NVIDIA GeForce RTX 3060	31.32	Ubuntu 22.04.4 LTS	0.1.38	gemma:2b	115.64
Linux	AMD Ryzen 7 5700G with Radeon Graphics	NVIDIA GeForce RTX 3060	31.32	Ubuntu 22.04.4 LTS	0.1.38	gemma:7b	50.68
Linux	AMD Ryzen 7 5700G with Radeon Graphics	NVIDIA GeForce RTX 3060	31.32	Ubuntu 22.04.4 LTS	0.1.38	llava:7b	64.88
Linux	AMD Ryzen 7 5700G with Radeon Graphics	NVIDIA GeForce RTX 3060	31.32	Ubuntu 22.04.4 LTS	0.1.38	llava:13b	38.9
Linux	Intel(R) Core(TM) i9-7940X CPU @ 3.10GHz	NVIDIA TITAN Xp COLLECTORS EDITION	125.48	Debian GNU/Linux 12 (bookworm)	0.1.38	mistral:7b	65.66
Linux	Intel(R) Core(TM) i9-7940X CPU @ 3.10GHz	NVIDIA TITAN Xp COLLECTORS EDITION	125.48	Debian GNU/Linux 12 (bookworm)	0.1.38	llama3:8b	55.64
Linux	Intel(R) Core(TM) i9-7940X CPU @ 3.10GHz	NVIDIA TITAN Xp COLLECTORS EDITION	125.48	Debian GNU/Linux 12 (bookworm)	0.1.38	phi3:3.8b	82.46
Linux	Intel(R) Core(TM) i9-7940X CPU @ 3.10GHz	NVIDIA TITAN Xp COLLECTORS EDITION	125.48	Debian GNU/Linux 12 (bookworm)	0.1.38	gemma:2b	126.03
Linux	Intel(R) Core(TM) i9-7940X CPU @ 3.10GHz	NVIDIA TITAN Xp COLLECTORS EDITION	125.48	Debian GNU/Linux 12 (bookworm)	0.1.38	gemma:7b	52.22
Linux	Intel(R) Core(TM) i9-7940X CPU @ 3.10GHz	NVIDIA TITAN Xp COLLECTORS EDITION	125.48	Debian GNU/Linux 12 (bookworm)	0.1.38	llava:7b	68.92
Linux	Intel(R) Core(TM) i9-7940X CPU @ 3.10GHz	NVIDIA TITAN Xp COLLECTORS EDITION	125.48	Debian GNU/Linux 12 (bookworm)	0.1.38	llava:13b	39.98
Linux	Intel(R) Core(TM) i9-7940X CPU @ 3.10GHz	NVIDIA TITAN Xp COLLECTORS EDITION	125.48	Debian GNU/Linux 12 (bookworm)	0.1.38	mistral:7b	64.74
Linux	Intel(R) Core(TM) i9-7940X CPU @ 3.10GHz	NVIDIA TITAN Xp COLLECTORS EDITION	125.48	Debian GNU/Linux 12 (bookworm)	0.1.38	llama3:8b	55.39
Linux	Intel(R) Core(TM) i9-7940X CPU @ 3.10GHz	NVIDIA TITAN Xp COLLECTORS EDITION	125.48	Debian GNU/Linux 12 (bookworm)	0.1.38	phi3:3.8b	81.74
Linux	Intel(R) Core(TM) i9-7940X CPU @ 3.10GHz	NVIDIA TITAN Xp COLLECTORS EDITION	125.48	Debian GNU/Linux 12 (bookworm)	0.1.38	gemma:2b	125.43
Linux	Intel(R) Core(TM) i9-7940X CPU @ 3.10GHz	NVIDIA TITAN Xp COLLECTORS EDITION	125.48	Debian GNU/Linux 12 (bookworm)	0.1.38	gemma:7b	52.18
Linux	Intel(R) Core(TM) i9-7940X CPU @ 3.10GHz	NVIDIA TITAN Xp COLLECTORS EDITION	125.48	Debian GNU/Linux 12 (bookworm)	0.1.38	llava:7b	68.82
Linux	Intel(R) Core(TM) i9-7940X CPU @ 3.10GHz	NVIDIA TITAN Xp COLLECTORS EDITION	125.48	Debian GNU/Linux 12 (bookworm)	0.1.38	llava:13b	40.14
Linux	Intel(R) Core(TM) i9-7940X CPU @ 3.10GHz	NVIDIA TITAN Xp COLLECTORS EDITION	125.48	Debian GNU/Linux 12 (bookworm)	0.1.38	mistral:7b	65.9
Linux	Intel(R) Core(TM) i9-7940X CPU @ 3.10GHz	NVIDIA TITAN Xp COLLECTORS EDITION	125.48	Debian GNU/Linux 12 (bookworm)	0.1.38	llama3:8b	55.68
Linux	Intel(R) Core(TM) i9-7940X CPU @ 3.10GHz	NVIDIA TITAN Xp COLLECTORS EDITION	125.48	Debian GNU/Linux 12 (bookworm)	0.1.38	phi3:3.8b	82.6
Linux	Intel(R) Core(TM) i9-7940X CPU @ 3.10GHz	NVIDIA TITAN Xp COLLECTORS EDITION	125.48	Debian GNU/Linux 12 (bookworm)	0.1.38	gemma:2b	126.11
Linux	Intel(R) Core(TM) i9-7940X CPU @ 3.10GHz	NVIDIA TITAN Xp COLLECTORS EDITION	125.48	Debian GNU/Linux 12 (bookworm)	0.1.38	gemma:7b	52.08
Linux	Intel(R) Core(TM) i9-7940X CPU @ 3.10GHz	NVIDIA TITAN Xp COLLECTORS EDITION	125.48	Debian GNU/Linux 12 (bookworm)	0.1.38	llava:7b	69.13
Linux	Intel(R) Core(TM) i9-7940X CPU @ 3.10GHz	NVIDIA TITAN Xp COLLECTORS EDITION	125.48	Debian GNU/Linux 12 (bookworm)	0.1.38	llava:13b	40.02
Linux	Intel(R) Core(TM) i9-7940X CPU @ 3.10GHz	NVIDIA TITAN Xp COLLECTORS EDITION	125.48	Debian GNU/Linux 12 (bookworm)	0.1.38	llama3:8b	55.38
Linux	Intel(R) Core(TM) Ultra 5 135HL	Intel Corporation	15.30	Ubuntu 22.04.4 LTS	0.0.0	mistral:7b	13.87
Linux	Intel(R) Core(TM) Ultra 5 135HL	Intel Corporation	15.30	Ubuntu 22.04.4 LTS	0.0.0	llama3:8b	10.53
Linux	Intel(R) Core(TM) Ultra 5 135HL	Intel Corporation	15.30	Ubuntu 22.04.4 LTS	0.0.0	phi3:3.8b	7.29
Linux	Intel(R) Core(TM) Ultra 5 135HL	Intel Corporation	15.30	Ubuntu 22.04.4 LTS	0.0.0	gemma:2b	17.04
Linux	Intel(R) Core(TM) Ultra 5 135HL	Intel Corporation	15.30	Ubuntu 22.04.4 LTS	0.0.0	gemma:7b	8.26
Linux	Intel(R) Core(TM) Ultra 5 135HL	Intel Corporation	15.30	Ubuntu 22.04.4 LTS	0.0.0	llava:7b	13.2
Linux	Intel(R) Core(TM) Ultra 5 135HL	Intel Corporation	15.30	Ubuntu 22.04.4 LTS	0.0.0	llava:13b	7.2
Linux	11th Gen Intel(R) Core(TM) i7-11800H @ 2.30GHz	NVIDIA GeForce RTX 3070	62.61	Ubuntu 22.04.4 LTS	0.1.38	mistral:7b	75.24
Linux	11th Gen Intel(R) Core(TM) i7-11800H @ 2.30GHz	NVIDIA GeForce RTX 3070	62.61	Ubuntu 22.04.4 LTS	0.1.38	llama3:8b	66.18
Linux	11th Gen Intel(R) Core(TM) i7-11800H @ 2.30GHz	NVIDIA GeForce RTX 3070	62.61	Ubuntu 22.04.4 LTS	0.1.38	phi3:3.8b	107.26
Linux	11th Gen Intel(R) Core(TM) i7-11800H @ 2.30GHz	NVIDIA GeForce RTX 3070	62.61	Ubuntu 22.04.4 LTS	0.1.38	gemma:2b	132.5
Linux	11th Gen Intel(R) Core(TM) i7-11800H @ 2.30GHz	NVIDIA GeForce RTX 3070	62.61	Ubuntu 22.04.4 LTS	0.1.38	gemma:7b	60.74
Linux	11th Gen Intel(R) Core(TM) i7-11800H @ 2.30GHz	NVIDIA GeForce RTX 3070	62.61	Ubuntu 22.04.4 LTS	0.1.38	llava:7b	78.11
Linux	11th Gen Intel(R) Core(TM) i7-11800H @ 2.30GHz	NVIDIA GeForce RTX 3070	62.61	Ubuntu 22.04.4 LTS	0.1.38	llava:13b	12.86
Linux	AMD EPYC 7V12 64-Core Processor	Tesla T4	108.06	Ubuntu 22.04.4 LTS	0.1.38	mistral:7b	48.17
Linux	AMD EPYC 7V12 64-Core Processor	Tesla T4	108.06	Ubuntu 22.04.4 LTS	0.1.38	llama3:8b	41.52
Linux	AMD EPYC 7V12 64-Core Processor	Tesla T4	108.06	Ubuntu 22.04.4 LTS	0.1.38	phi3:3.8b	64.32
Linux	AMD EPYC 7V12 64-Core Processor	Tesla T4	108.06	Ubuntu 22.04.4 LTS	0.1.38	gemma:2b	94.6
Linux	AMD EPYC 7V12 64-Core Processor	Tesla T4	108.06	Ubuntu 22.04.4 LTS	0.1.38	gemma:7b	35.2
Linux	AMD EPYC 7V12 64-Core Processor	Tesla T4	108.06	Ubuntu 22.04.4 LTS	0.1.38	llava:7b	45.58
Linux	AMD EPYC 7V12 64-Core Processor	Tesla T4	108.06	Ubuntu 22.04.4 LTS	0.1.38	llava:13b	26.06
Linux	11th Gen Intel(R) Core(TM) i7-11800H @ 2.30GHz	NVIDIA GeForce RTX 3070	62.61	Ubuntu 22.04.4 LTS	0.1.38	mistral:7b	75.42
Linux	11th Gen Intel(R) Core(TM) i7-11800H @ 2.30GHz	NVIDIA GeForce RTX 3070	62.61	Ubuntu 22.04.4 LTS	0.1.38	llama3:8b	66.01
Linux	11th Gen Intel(R) Core(TM) i7-11800H @ 2.30GHz	NVIDIA GeForce RTX 3070	62.61	Ubuntu 22.04.4 LTS	0.1.38	phi3:3.8b	108.23
Linux	11th Gen Intel(R) Core(TM) i7-11800H @ 2.30GHz	NVIDIA GeForce RTX 3070	62.61	Ubuntu 22.04.4 LTS	0.1.38	gemma:2b	132.11
Linux	11th Gen Intel(R) Core(TM) i7-11800H @ 2.30GHz	NVIDIA GeForce RTX 3070	62.61	Ubuntu 22.04.4 LTS	0.1.38	gemma:7b	60.77
Linux	11th Gen Intel(R) Core(TM) i7-11800H @ 2.30GHz	NVIDIA GeForce RTX 3070	62.61	Ubuntu 22.04.4 LTS	0.1.38	llava:7b	78.05
Linux	11th Gen Intel(R) Core(TM) i7-11800H @ 2.30GHz	NVIDIA GeForce RTX 3070	62.61	Ubuntu 22.04.4 LTS	0.1.38	llava:13b	12.92
Linux	AMD Ryzen 5 1600 Six-Core Processor	no_gpu	46.97	Debian GNU/Linux 12 (bookworm)	0.1.33	mistral:7b	5.87
Linux	AMD Ryzen 5 1600 Six-Core Processor	no_gpu	46.97	Debian GNU/Linux 12 (bookworm)	0.1.33	llama3:8b	5.55
Linux	AMD Ryzen 5 1600 Six-Core Processor	no_gpu	46.97	Debian GNU/Linux 12 (bookworm)	0.1.33	phi3:3.8b	10.12
Linux	AMD Ryzen 5 1600 Six-Core Processor	no_gpu	46.97	Debian GNU/Linux 12 (bookworm)	0.1.33	gemma:2b	12.7
Linux	AMD Ryzen 5 1600 Six-Core Processor	no_gpu	46.97	Debian GNU/Linux 12 (bookworm)	0.1.33	gemma:7b	4.68
Linux	AMD Ryzen 5 1600 Six-Core Processor	no_gpu	46.97	Debian GNU/Linux 12 (bookworm)	0.1.33	llava:7b	6.06
Linux	AMD Ryzen 5 1600 Six-Core Processor	no_gpu	46.97	Debian GNU/Linux 12 (bookworm)	0.1.33	llava:13b	3.43
Linux	13th Gen Intel(R) Core(TM) i9-13900K	NVIDIA GeForce RTX 3090	188.40	Fedora Linux 40 (Workstation Edition)	0.1.32	mistral:7b	125
Linux	13th Gen Intel(R) Core(TM) i9-13900K	NVIDIA GeForce RTX 3090	188.40	Fedora Linux 40 (Workstation Edition)	0.1.32	llama3:8b	109.65
Linux	13th Gen Intel(R) Core(TM) i9-13900K	NVIDIA GeForce RTX 3090	188.40	Fedora Linux 40 (Workstation Edition)	0.1.32	phi3:3.8b	165.32
Linux	13th Gen Intel(R) Core(TM) i9-13900K	NVIDIA GeForce RTX 3090	188.40	Fedora Linux 40 (Workstation Edition)	0.1.32	gemma:2b	197.51
Linux	13th Gen Intel(R) Core(TM) i9-13900K	NVIDIA GeForce RTX 3090	188.40	Fedora Linux 40 (Workstation Edition)	0.1.32	gemma:7b	105.2
Linux	13th Gen Intel(R) Core(TM) i9-13900K	NVIDIA GeForce RTX 3090	188.40	Fedora Linux 40 (Workstation Edition)	0.1.32	llava:7b	129.79
Linux	13th Gen Intel(R) Core(TM) i9-13900K	NVIDIA GeForce RTX 3090	188.40	Fedora Linux 40 (Workstation Edition)	0.1.32	llava:13b	82.62
Linux	Intel(R) Core(TM) i7-10750H CPU @ 2.60GHz	NVIDIA GeForce GTX 1650 Ti	15.36	Ubuntu 20.04.6 LTS	0.1.32	phi3:3.8b	24.75
Linux	13th Gen Intel(R) Core(TM) i9-13900K	NVIDIA GeForce RTX 3090	188.40	Fedora Linux 40 (Workstation Edition)	0.1.38	mistral:7b	126.24
Linux	13th Gen Intel(R) Core(TM) i9-13900K	NVIDIA GeForce RTX 3090	188.40	Fedora Linux 40 (Workstation Edition)	0.1.38	llama3:8b	110.72
Linux	13th Gen Intel(R) Core(TM) i9-13900K	NVIDIA GeForce RTX 3090	188.40	Fedora Linux 40 (Workstation Edition)	0.1.38	phi3:3.8b	168.48
Linux	13th Gen Intel(R) Core(TM) i9-13900K	NVIDIA GeForce RTX 3090	188.40	Fedora Linux 40 (Workstation Edition)	0.1.38	gemma:2b	208.21
Linux	13th Gen Intel(R) Core(TM) i9-13900K	NVIDIA GeForce RTX 3090	188.40	Fedora Linux 40 (Workstation Edition)	0.1.38	gemma:7b	105.46
Linux	13th Gen Intel(R) Core(TM) i9-13900K	NVIDIA GeForce RTX 3090	188.40	Fedora Linux 40 (Workstation Edition)	0.1.38	llava:7b	129.81
Linux	13th Gen Intel(R) Core(TM) i9-13900K	NVIDIA GeForce RTX 3090	188.40	Fedora Linux 40 (Workstation Edition)	0.1.38	llava:13b	82.13
Linux	Intel(R) Core(TM) Ultra 5 135HL	Intel Corporation	15.30	Ubuntu 22.04.4 LTS	0.0.0	mistral:7b	12.18
Linux	Intel(R) Core(TM) Ultra 5 135HL	Intel Corporation	15.30	Ubuntu 22.04.4 LTS	0.0.0	llama3:8b	12.72
Linux	Intel(R) Core(TM) Ultra 5 135HL	Intel Corporation	15.30	Ubuntu 22.04.4 LTS	0.0.0	phi3:3.8b	9.61
Linux	Intel(R) Core(TM) Ultra 5 135HL	Intel Corporation	15.30	Ubuntu 22.04.4 LTS	0.0.0	gemma:2b	20.46
Linux	Intel(R) Core(TM) Ultra 5 135HL	Intel Corporation	15.30	Ubuntu 22.04.4 LTS	0.0.0	gemma:7b	7.66
Linux	Intel(R) Core(TM) Ultra 5 135HL	Intel Corporation	15.30	Ubuntu 22.04.4 LTS	0.0.0	llava:7b	13.12
Linux	Intel(R) Core(TM) Ultra 5 135HL	Intel Corporation	15.30	Ubuntu 22.04.4 LTS	0.0.0	llava:13b	7.46
Linux	AMD Ryzen 9 5950X 16-Core Processor	NVIDIA GeForce RTX 3090	125.70	Ubuntu 22.04.4 LTS	0.1.37	mistral:7b	116.47
Linux	AMD Ryzen 9 5950X 16-Core Processor	NVIDIA GeForce RTX 3090	125.70	Ubuntu 22.04.4 LTS	0.1.37	llama3:8b	101.5
Linux	AMD Ryzen 9 5950X 16-Core Processor	NVIDIA GeForce RTX 3090	125.70	Ubuntu 22.04.4 LTS	0.1.37	phi3:3.8b	153.45
Linux	AMD Ryzen 9 5950X 16-Core Processor	NVIDIA GeForce RTX 3090	125.70	Ubuntu 22.04.4 LTS	0.1.37	gemma:2b	192.21
Linux	AMD Ryzen 9 5950X 16-Core Processor	NVIDIA GeForce RTX 3090	125.70	Ubuntu 22.04.4 LTS	0.1.37	gemma:7b	95.83
Linux	AMD Ryzen 9 5950X 16-Core Processor	NVIDIA GeForce RTX 3090	125.70	Ubuntu 22.04.4 LTS	0.1.37	llava:7b	118.17
Linux	AMD Ryzen 9 5950X 16-Core Processor	NVIDIA GeForce RTX 3090	125.70	Ubuntu 22.04.4 LTS	0.1.37	llava:13b	73.77
Linux	AMD EPYC 7402 24-Core Processor	NVIDIA RTX A5000	251.73	"Ubuntu 22.04.4 LTS"	0.0.0	mistral:7b	100.76
Linux	AMD EPYC 7402 24-Core Processor	NVIDIA RTX A5000	251.73	"Ubuntu 22.04.4 LTS"	0.0.0	llama3:8b	82.66
Linux	AMD EPYC 7402 24-Core Processor	NVIDIA RTX A5000	251.73	"Ubuntu 22.04.4 LTS"	0.0.0	phi3:3.8b	133.37
Linux	AMD EPYC 7402 24-Core Processor	NVIDIA RTX A5000	251.73	"Ubuntu 22.04.4 LTS"	0.0.0	gemma:2b	152.19
Linux	AMD EPYC 7402 24-Core Processor	NVIDIA RTX A5000	251.73	"Ubuntu 22.04.4 LTS"	0.0.0	gemma:7b	80.3
Linux	AMD EPYC 7402 24-Core Processor	NVIDIA RTX A5000	251.73	"Ubuntu 22.04.4 LTS"	0.0.0	llava:7b	104.11
Linux	AMD EPYC 7402 24-Core Processor	NVIDIA RTX A5000	251.73	"Ubuntu 22.04.4 LTS"	0.0.0	llava:13b	65.58
Linux	AMD Ryzen 7 5700X 8-Core Processor	NVIDIA GeForce RTX 4060 Ti	7.76	Ubuntu 22.04.4 LTS	0.1.38	mistral:7b	41.96
Linux	AMD Ryzen 7 5700X 8-Core Processor	NVIDIA GeForce RTX 4060 Ti	7.76	Ubuntu 22.04.4 LTS	0.1.38	llama3:8b	40.07
Linux	AMD Ryzen 7 5700X 8-Core Processor	NVIDIA GeForce RTX 4060 Ti	7.76	Ubuntu 22.04.4 LTS	0.1.38	phi3:3.8b	56.91
Linux	AMD Ryzen 7 5700X 8-Core Processor	NVIDIA GeForce RTX 4060 Ti	7.76	Ubuntu 22.04.4 LTS	0.1.38	gemma:2b	72.35
Linux	AMD Ryzen 7 5700X 8-Core Processor	NVIDIA GeForce RTX 4060 Ti	7.76	Ubuntu 22.04.4 LTS	0.1.38	gemma:7b	38.02
Linux	AMD Ryzen 7 5700X 8-Core Processor	NVIDIA GeForce RTX 4060 Ti	7.76	Ubuntu 22.04.4 LTS	0.1.38	llava:7b	42.79
Linux	AMD Ryzen 7 5700X 8-Core Processor	NVIDIA GeForce RTX 4060 Ti	7.76	Ubuntu 22.04.4 LTS	0.1.38	mistral:7b	42
Linux	AMD Ryzen 7 5700X 8-Core Processor	NVIDIA GeForce RTX 4060 Ti	7.76	Ubuntu 22.04.4 LTS	0.1.38	llama3:8b	39.28
Linux	AMD Ryzen 7 5700X 8-Core Processor	NVIDIA GeForce RTX 4060 Ti	7.76	Ubuntu 22.04.4 LTS	0.1.38	phi3:3.8b	58.83
Linux	AMD Ryzen 7 5700X 8-Core Processor	NVIDIA GeForce RTX 4060 Ti	7.76	Ubuntu 22.04.4 LTS	0.1.38	gemma:2b	73.32
Linux	AMD Ryzen 7 5700X 8-Core Processor	NVIDIA GeForce RTX 4060 Ti	7.76	Ubuntu 22.04.4 LTS	0.1.38	gemma:7b	39.26
Linux	AMD Ryzen 7 5700X 8-Core Processor	NVIDIA GeForce RTX 4060 Ti	7.76	Ubuntu 22.04.4 LTS	0.1.38	llava:7b	45.69
Linux	AMD Ryzen 7 5700X 8-Core Processor	NVIDIA GeForce RTX 4060 Ti	7.76	Ubuntu 22.04.4 LTS	0.1.38	mistral:7b	44.49
Linux	AMD Ryzen 7 5700X 8-Core Processor	NVIDIA GeForce RTX 4060 Ti	7.76	Ubuntu 22.04.4 LTS	0.1.38	llama3:8b	38.98
Linux	AMD Ryzen 7 5700X 8-Core Processor	NVIDIA GeForce RTX 4060 Ti	7.76	Ubuntu 22.04.4 LTS	0.1.38	phi3:3.8b	57.24
Linux	AMD Ryzen 7 5700X 8-Core Processor	NVIDIA GeForce RTX 4060 Ti	7.76	Ubuntu 22.04.4 LTS	0.1.38	gemma:2b	72.26
Linux	AMD Ryzen 7 5700X 8-Core Processor	NVIDIA GeForce RTX 4060 Ti	7.76	Ubuntu 22.04.4 LTS	0.1.38	gemma:7b	39.57
Linux	AMD Ryzen 7 5700X 8-Core Processor	NVIDIA GeForce RTX 4060 Ti	7.76	Ubuntu 22.04.4 LTS	0.1.38	llava:7b	44.04
Linux	Intel(R) Xeon(R) Platinum 8369HC B3 CPU	NVIDIA GeForce RTX 4070	27.42	Ubuntu 20.04.5 LTS	0.1.39	mistral:7b	90.32
Linux	Intel(R) Xeon(R) Platinum 8369HC B3 CPU	NVIDIA GeForce RTX 4070	27.42	Ubuntu 20.04.5 LTS	0.1.39	llama3:8b	79.79
Linux	Intel(R) Xeon(R) Platinum 8369HC B3 CPU	NVIDIA GeForce RTX 4070	27.42	Ubuntu 20.04.5 LTS	0.1.39	phi3:3.8b	130.75
Linux	Intel(R) Xeon(R) Platinum 8369HC B3 CPU	NVIDIA GeForce RTX 4070	27.42	Ubuntu 20.04.5 LTS	0.1.39	gemma:2b	164.8
Linux	Intel(R) Xeon(R) Platinum 8369HC B3 CPU	NVIDIA GeForce RTX 4070	27.42	Ubuntu 20.04.5 LTS	0.1.39	gemma:7b	71.99
Linux	Intel(R) Xeon(R) Platinum 8369HC B3 CPU	NVIDIA GeForce RTX 4070	27.42	Ubuntu 20.04.5 LTS	0.1.39	llava:7b	93.53
Linux	Intel(R) Xeon(R) Platinum 8369HC B3 CPU	NVIDIA GeForce RTX 4070	27.42	Ubuntu 20.04.5 LTS	0.1.39	llava:13b	55.49
Linux	Intel(R) Xeon(R) Gold 6442Y	NVIDIA A40	503.39	Ubuntu 22.04.4 LTS	0.1.38	gemma:2b	42.26
Linux	Intel(R) Xeon(R) Gold 6442Y	NVIDIA A40	503.39	Ubuntu 22.04.4 LTS	0.1.38	gemma:7b	16.74
Linux	Intel(R) Xeon(R) Gold 6442Y	NVIDIA A40	503.39	Ubuntu 22.04.4 LTS	0.1.38	mistral:7b	20.89
Linux	Intel(R) Xeon(R) Gold 6442Y	NVIDIA A40	503.39	Ubuntu 22.04.4 LTS	0.1.38	llama3:8b	22.66
Linux	Intel(R) Xeon(R) Gold 6442Y	NVIDIA A40	503.39	Ubuntu 22.04.4 LTS	0.1.38	phi3:3.8b	26.9
Linux	Intel(R) Xeon(R) Gold 6442Y	NVIDIA A40	503.39	Ubuntu 22.04.4 LTS	0.1.38	llava:7b	23.67
Linux	Intel(R) Core(TM) i3-10105T CPU @ 3.00GHz	CometLake-S GT2 [UHD Graphics 630]	22.91	"Rocky Linux 9.4 (Blue Onyx)"	0.1.38	mistral:7b	5.55
Linux	Intel(R) Core(TM) i3-10105T CPU @ 3.00GHz	CometLake-S GT2 [UHD Graphics 630]	22.91	"Rocky Linux 9.4 (Blue Onyx)"	0.1.38	llama3:8b	5.19
Linux	Intel(R) Core(TM) i3-10105T CPU @ 3.00GHz	CometLake-S GT2 [UHD Graphics 630]	22.91	"Rocky Linux 9.4 (Blue Onyx)"	0.1.38	phi3:3.8b	9.37
Linux	Intel(R) Core(TM) i3-10105T CPU @ 3.00GHz	CometLake-S GT2 [UHD Graphics 630]	22.91	"Rocky Linux 9.4 (Blue Onyx)"	0.1.38	gemma:2b	13.23
Linux	Intel(R) Core(TM) i3-10105T CPU @ 3.00GHz	CometLake-S GT2 [UHD Graphics 630]	22.91	"Rocky Linux 9.4 (Blue Onyx)"	0.1.38	gemma:7b	4.62
Linux	Intel(R) Core(TM) i3-10105T CPU @ 3.00GHz	CometLake-S GT2 [UHD Graphics 630]	22.91	"Rocky Linux 9.4 (Blue Onyx)"	0.1.38	llava:7b	5.82
Linux	Intel(R) Core(TM) i3-10105T CPU @ 3.00GHz	CometLake-S GT2 [UHD Graphics 630]	22.91	"Rocky Linux 9.4 (Blue Onyx)"	0.1.38	llava:13b	3.29
Linux	13th Gen Intel(R) Core(TM) i9-13900HX	NVIDIA GeForce RTX 3090	7.62	"Ubuntu 22.04.4 LTS"	0.1.38	mistral:7b	121.55
Linux	13th Gen Intel(R) Core(TM) i9-13900HX	NVIDIA GeForce RTX 3090	7.62	"Ubuntu 22.04.4 LTS"	0.1.38	llama3:8b	103.46
Linux	13th Gen Intel(R) Core(TM) i9-13900HX	NVIDIA GeForce RTX 3090	7.62	"Ubuntu 22.04.4 LTS"	0.1.38	phi3:3.8b	159.05
Linux	13th Gen Intel(R) Core(TM) i9-13900HX	NVIDIA GeForce RTX 3090	7.62	"Ubuntu 22.04.4 LTS"	0.1.38	gemma:2b	185.19
Linux	13th Gen Intel(R) Core(TM) i9-13900HX	NVIDIA GeForce RTX 3090	7.62	"Ubuntu 22.04.4 LTS"	0.1.38	gemma:7b	97.9
Linux	13th Gen Intel(R) Core(TM) i9-13900HX	NVIDIA GeForce RTX 3090	7.62	"Ubuntu 22.04.4 LTS"	0.1.38	llava:7b	124.94
Linux	13th Gen Intel(R) Core(TM) i9-13900HX	NVIDIA GeForce RTX 3090	7.62	"Ubuntu 22.04.4 LTS"	0.1.38	mistral:7b	99.72
Linux	13th Gen Intel(R) Core(TM) i9-13900HX	NVIDIA GeForce RTX 3090	7.62	"Ubuntu 22.04.4 LTS"	0.1.38	llama3:8b	90.6
Linux	13th Gen Intel(R) Core(TM) i9-13900HX	NVIDIA GeForce RTX 3090	7.62	"Ubuntu 22.04.4 LTS"	0.1.38	phi3:3.8b	131.64
Linux	13th Gen Intel(R) Core(TM) i9-13900HX	NVIDIA GeForce RTX 3090	7.62	"Ubuntu 22.04.4 LTS"	0.1.38	gemma:2b	175.63
Linux	13th Gen Intel(R) Core(TM) i9-13900HX	NVIDIA GeForce RTX 3090	7.62	"Ubuntu 22.04.4 LTS"	0.1.38	gemma:7b	75.79
Linux	13th Gen Intel(R) Core(TM) i9-13900HX	NVIDIA GeForce RTX 3090	7.62	"Ubuntu 22.04.4 LTS"	0.1.38	llava:7b	108.57
Linux	13th Gen Intel(R) Core(TM) i9-13900HX	NVIDIA GeForce RTX 3090	7.62	"Ubuntu 22.04.4 LTS"	0.1.38	mistral:7b	69.56
Linux	13th Gen Intel(R) Core(TM) i9-13900HX	NVIDIA GeForce RTX 3090	7.62	"Ubuntu 22.04.4 LTS"	0.1.38	llama3:8b	63.64
Linux	13th Gen Intel(R) Core(TM) i9-13900HX	NVIDIA GeForce RTX 3090	7.62	"Ubuntu 22.04.4 LTS"	0.1.38	phi3:3.8b	94.18
Linux	13th Gen Intel(R) Core(TM) i9-13900HX	NVIDIA GeForce RTX 3090	7.62	"Ubuntu 22.04.4 LTS"	0.1.38	gemma:2b	164.09
Linux	13th Gen Intel(R) Core(TM) i9-13900HX	NVIDIA GeForce RTX 3090	7.62	"Ubuntu 22.04.4 LTS"	0.1.38	gemma:7b	53.31
Linux	13th Gen Intel(R) Core(TM) i9-13900HX	NVIDIA GeForce RTX 3090	7.62	"Ubuntu 22.04.4 LTS"	0.1.38	llava:7b	74.62
Linux	Intel(R) Core(TM) i5-9300H CPU @ 2.40GHz	NVIDIA GeForce GTX 1650	7.70	Ubuntu 22.04.4 LTS	0.1.33	mistral:7b	4.69
Linux	Intel(R) Core(TM) i5-9300H CPU @ 2.40GHz	NVIDIA GeForce GTX 1650	7.70	Ubuntu 22.04.4 LTS	0.1.33	llama3:8b	3.55
Linux	Intel(R) Core(TM) i5-9300H CPU @ 2.40GHz	NVIDIA GeForce GTX 1650	7.70	Ubuntu 22.04.4 LTS	0.1.33	phi3:3.8b	13.89
Linux	Intel(R) Core(TM) i5-9300H CPU @ 2.40GHz	NVIDIA GeForce GTX 1650	7.70	Ubuntu 22.04.4 LTS	0.1.33	gemma:2b	33.56
Linux	Intel(R) Core(TM) i5-9300H CPU @ 2.40GHz	NVIDIA GeForce GTX 1650	7.70	Ubuntu 22.04.4 LTS	0.1.33	gemma:7b	2.6
Linux	Intel(R) Core(TM) i5-9300H CPU @ 2.40GHz	NVIDIA GeForce GTX 1650	7.70	Ubuntu 22.04.4 LTS	0.1.33	llava:7b	4.09
Linux	Intel(R) Xeon(R) Platinum 8369HC B3 CPU	NVIDIA GeForce RTX 4070	27.42	Ubuntu 20.04.5 LTS	0.1.39	mistral:7b	90.61
Linux	Intel(R) Xeon(R) Platinum 8369HC B3 CPU	NVIDIA GeForce RTX 4070	27.42	Ubuntu 20.04.5 LTS	0.1.39	llama3:8b	79.91
Linux	Intel(R) Xeon(R) Platinum 8369HC B3 CPU	NVIDIA GeForce RTX 4070	27.42	Ubuntu 20.04.5 LTS	0.1.39	phi3:3.8b	127.16
Linux	Intel(R) Xeon(R) Platinum 8369HC B3 CPU	NVIDIA GeForce RTX 4070	27.42	Ubuntu 20.04.5 LTS	0.1.39	gemma:2b	159.89
Linux	Intel(R) Xeon(R) Platinum 8369HC B3 CPU	NVIDIA GeForce RTX 4070	27.42	Ubuntu 20.04.5 LTS	0.1.39	gemma:7b	71.9
Linux	Intel(R) Xeon(R) Platinum 8369HC B3 CPU	NVIDIA GeForce RTX 4070	27.42	Ubuntu 20.04.5 LTS	0.1.39	llava:7b	93.76
Linux	Intel(R) Xeon(R) Platinum 8369HC B3 CPU	NVIDIA GeForce RTX 4070	27.42	Ubuntu 20.04.5 LTS	0.1.39	llava:13b	55.5
Linux	AMD Ryzen 9 7940HS w/ Radeon 780M Graphics	Phoenix1	58.67	Arch Linux	0.1.38	mistral:7b	29.68
Linux	AMD Ryzen 9 7940HS w/ Radeon 780M Graphics	Phoenix1	58.67	Arch Linux	0.1.38	llama3:8b	27.85
Linux	AMD Ryzen 9 7940HS w/ Radeon 780M Graphics	Phoenix1	58.67	Arch Linux	0.1.38	phi3:3.8b	37.28
Linux	AMD Ryzen 9 7940HS w/ Radeon 780M Graphics	Phoenix1	58.67	Arch Linux	0.1.38	gemma:2b	66.77
Linux	AMD Ryzen 9 7940HS w/ Radeon 780M Graphics	Phoenix1	58.67	Arch Linux	0.1.38	gemma:7b	25.95
Linux	AMD Ryzen 9 7940HS w/ Radeon 780M Graphics	Phoenix1	58.67	Arch Linux	0.1.38	llava:7b	31.44
Linux	AMD Ryzen 9 7940HS w/ Radeon 780M Graphics	Phoenix1	58.67	Arch Linux	0.1.38	llava:13b	11.3
Linux	Intel(R) Xeon(R) Gold 6442Y	NVIDIA A40	503.39	Ubuntu 22.04.4 LTS	0.1.38	mistral:7b	94.18
Linux	Intel(R) Xeon(R) Gold 6442Y	NVIDIA A40	503.39	Ubuntu 22.04.4 LTS	0.1.38	gemma:7b	77.33
Linux	Intel(R) Xeon(R) Gold 6442Y	NVIDIA A40	503.39	Ubuntu 22.04.4 LTS	0.1.38	llama3:8b	82.42
Linux	Intel(R) Xeon(R) Gold 6442Y	NVIDIA A40	503.39	Ubuntu 22.04.4 LTS	0.1.38	phi3:3.8b	127.43
Linux	Intel(R) Xeon(R) Gold 6442Y	NVIDIA A40	503.39	Ubuntu 22.04.4 LTS	0.1.38	llava:13b	59.9
Linux	AMD Ryzen 9 7940HS w/ Radeon 780M Graphics	Phoenix1	58.67	Arch Linux	0.1.38	mistral:7b	41.43
Linux	AMD Ryzen 9 7940HS w/ Radeon 780M Graphics	Phoenix1	58.67	Arch Linux	0.1.38	llama3:8b	39.14
Linux	AMD Ryzen 9 7940HS w/ Radeon 780M Graphics	Phoenix1	58.67	Arch Linux	0.1.38	phi3:3.8b	52.26
Linux	AMD Ryzen 9 7940HS w/ Radeon 780M Graphics	Phoenix1	58.67	Arch Linux	0.1.38	gemma:2b	92.51
Linux	AMD Ryzen 9 7940HS w/ Radeon 780M Graphics	Phoenix1	58.67	Arch Linux	0.1.38	gemma:7b	36.93
Linux	AMD Ryzen 9 7940HS w/ Radeon 780M Graphics	Phoenix1	58.67	Arch Linux	0.1.38	llava:7b	43.64
Linux	AMD Ryzen 9 7940HS w/ Radeon 780M Graphics	Phoenix1	58.67	Arch Linux	0.1.38	llava:13b	4.64
Linux	AMD Ryzen 9 7940HS w/ Radeon 780M Graphics	Phoenix1	58.67	Arch Linux	0.1.38	mistral:7b	41.25
Linux	AMD Ryzen 9 7940HS w/ Radeon 780M Graphics	Phoenix1	58.67	Arch Linux	0.1.38	llama3:8b	39.19
Linux	AMD Ryzen 9 7940HS w/ Radeon 780M Graphics	Phoenix1	58.67	Arch Linux	0.1.38	phi3:3.8b	51.7
Linux	AMD Ryzen 9 7940HS w/ Radeon 780M Graphics	Phoenix1	58.67	Arch Linux	0.1.38	gemma:2b	92.49
Linux	AMD Ryzen 9 7940HS w/ Radeon 780M Graphics	Phoenix1	58.67	Arch Linux	0.1.38	gemma:7b	36.76
Linux	AMD Ryzen 9 7940HS w/ Radeon 780M Graphics	Phoenix1	58.67	Arch Linux	0.1.38	llava:7b	43.84
Linux	AMD Ryzen 9 7940HS w/ Radeon 780M Graphics	Phoenix1	58.67	Arch Linux	0.1.38	llava:13b	4.35
Linux	11th Gen Intel(R) Core(TM) i7-11800H @ 2.30GHz	NVIDIA GeForce RTX 3070 Laptop GPU	31.18	"Ubuntu 22.04.4 LTS"	0.1.37	llama3:8b	54.22
Linux	11th Gen Intel(R) Core(TM) i7-11800H @ 2.30GHz	NVIDIA GeForce RTX 3070 Laptop GPU	31.18	"Ubuntu 22.04.4 LTS"	0.1.37	llama3:8b	53.84
Linux	AMD EPYC 75F3 32-Core Processor	NVIDIA GeForce RTX 4090	503.71	Ubuntu 22.04.3 LTS	0.1.38	mistral:7b	138.53
Linux	AMD EPYC 75F3 32-Core Processor	NVIDIA GeForce RTX 4090	503.71	Ubuntu 22.04.3 LTS	0.1.38	llama3:8b	119.69
Linux	AMD EPYC 75F3 32-Core Processor	NVIDIA GeForce RTX 4090	503.71	Ubuntu 22.04.3 LTS	0.1.38	phi3:3.8b	185.05
Linux	AMD EPYC 75F3 32-Core Processor	NVIDIA GeForce RTX 4090	503.71	Ubuntu 22.04.3 LTS	0.1.38	gemma:2b	211.02
Linux	AMD EPYC 75F3 32-Core Processor	NVIDIA GeForce RTX 4090	503.71	Ubuntu 22.04.3 LTS	0.1.38	gemma:7b	103.61
Linux	AMD EPYC 75F3 32-Core Processor	NVIDIA GeForce RTX 4090	503.71	Ubuntu 22.04.3 LTS	0.1.38	llava:7b	142.37
Linux	AMD EPYC 75F3 32-Core Processor	NVIDIA GeForce RTX 4090	503.71	Ubuntu 22.04.3 LTS	0.1.38	llava:13b	90.31
Linux	unknown	NVIDIA A100 80GB PCIe	1021.17	Ubuntu 22.04.4 LTS	0.2.1	mistral:7b	127.23
Linux	unknown	NVIDIA A100 80GB PCIe	1021.17	Ubuntu 22.04.4 LTS	0.2.1	llama3:8b	88.93
Linux	unknown	NVIDIA A100 80GB PCIe	1021.17	Ubuntu 22.04.4 LTS	0.2.1	phi3:3.8b	133.04
Linux	unknown	NVIDIA A100 80GB PCIe	1021.17	Ubuntu 22.04.4 LTS	0.2.1	gemma2:9b	54.01
Linux	unknown	NVIDIA A100 80GB PCIe	1021.17	Ubuntu 22.04.4 LTS	0.2.1	gemma2:27b	38.25
Linux	unknown	NVIDIA A100 80GB PCIe	1021.17	Ubuntu 22.04.4 LTS	0.2.1	llava:7b	121.27
Linux	unknown	NVIDIA A100 80GB PCIe	1021.17	Ubuntu 22.04.4 LTS	0.2.1	llava:13b	86.06
Linux	12th Gen Intel(R) Core(TM) i7-12700KF	NVIDIA GeForce RTX 4070	27.42	Ubuntu 20.04.5 LTS	0.1.39	mistral:7b	87.96
Linux	12th Gen Intel(R) Core(TM) i7-12700KF	NVIDIA GeForce RTX 4070	27.42	Ubuntu 20.04.5 LTS	0.1.39	llama3:8b	78
Linux	12th Gen Intel(R) Core(TM) i7-12700KF	NVIDIA GeForce RTX 4070	27.42	Ubuntu 20.04.5 LTS	0.1.39	phi3:3.8b	126.41
Linux	12th Gen Intel(R) Core(TM) i7-12700KF	NVIDIA GeForce RTX 4070	27.42	Ubuntu 20.04.5 LTS	0.1.39	gemma:2b	159.4
Linux	12th Gen Intel(R) Core(TM) i7-12700KF	NVIDIA GeForce RTX 4070	27.42	Ubuntu 20.04.5 LTS	0.1.39	gemma:7b	71.13
Linux	12th Gen Intel(R) Core(TM) i7-12700KF	NVIDIA GeForce RTX 4070	27.42	Ubuntu 20.04.5 LTS	0.1.39	llava:7b	91.42
Linux	12th Gen Intel(R) Core(TM) i7-12700KF	NVIDIA GeForce RTX 4070	27.42	Ubuntu 20.04.5 LTS	0.1.39	llava:13b	53.86
Linux	12th Gen Intel(R) Core(TM) i7-12700	NVIDIA GeForce RTX 3060 Ti	31.12	Ubuntu 20.04.6 LTS	0.1.28	mistral:7b	76.09
Linux	12th Gen Intel(R) Core(TM) i7-12700	NVIDIA GeForce RTX 3060 Ti	31.12	Ubuntu 20.04.6 LTS	0.1.28	llama3:8b	67.59
Linux	12th Gen Intel(R) Core(TM) i7-12700	NVIDIA GeForce RTX 3060 Ti	31.12	Ubuntu 20.04.6 LTS	0.1.39	mistral:7b	76.52
Linux	12th Gen Intel(R) Core(TM) i7-12700	NVIDIA GeForce RTX 3060 Ti	31.12	Ubuntu 20.04.6 LTS	0.1.39	llama3:8b	68.37
Linux	12th Gen Intel(R) Core(TM) i7-12700	NVIDIA GeForce RTX 3060 Ti	31.12	Ubuntu 20.04.6 LTS	0.1.39	phi3:3.8b	109.07
Linux	11th Gen Intel(R) Core(TM) i7-1165G7 @ 2.80GHz	Microsoft Corporation	7.65	Ubuntu 22.04.4 LTS	0.1.38	mistral:7b	2.78
Linux	11th Gen Intel(R) Core(TM) i7-1165G7 @ 2.80GHz	Microsoft Corporation	7.65	Ubuntu 22.04.4 LTS	0.1.38	llama3:8b	2.47
Linux	11th Gen Intel(R) Core(TM) i7-1165G7 @ 2.80GHz	Microsoft Corporation	7.65	Ubuntu 22.04.4 LTS	0.1.38	phi3:3.8b	4.2
Linux	11th Gen Intel(R) Core(TM) i7-1165G7 @ 2.80GHz	Microsoft Corporation	7.65	Ubuntu 22.04.4 LTS	0.1.38	gemma:2b	6.74
Linux	11th Gen Intel(R) Core(TM) i7-1165G7 @ 2.80GHz	Microsoft Corporation	7.65	Ubuntu 22.04.4 LTS	0.1.38	gemma:7b	2
Linux	11th Gen Intel(R) Core(TM) i7-1165G7 @ 2.80GHz	Microsoft Corporation	7.65	Ubuntu 22.04.4 LTS	0.1.38	llava:7b	2.74
Linux	Intel(R) Xeon(R) Platinum 8370C CPU @ 2.80GHz	unknown	125.81	Ubuntu 22.04.4 LTS	0.1.39	mistral:7b	19.43
Linux	Intel(R) Xeon(R) Platinum 8370C CPU @ 2.80GHz	unknown	125.81	Ubuntu 22.04.4 LTS	0.1.39	llama3:8b	19.47
Linux	Intel(R) Xeon(R) Platinum 8370C CPU @ 2.80GHz	unknown	125.81	Ubuntu 22.04.4 LTS	0.1.39	phi3:3.8b	17.99
Linux	Intel(R) Xeon(R) Platinum 8370C CPU @ 2.80GHz	unknown	125.81	Ubuntu 22.04.4 LTS	0.1.39	gemma:2b	38.08
Linux	Intel(R) Xeon(R) Platinum 8370C CPU @ 2.80GHz	unknown	125.81	Ubuntu 22.04.4 LTS	0.1.39	gemma:7b	10.61
Linux	Intel(R) Xeon(R) Platinum 8370C CPU @ 2.80GHz	unknown	125.81	Ubuntu 22.04.4 LTS	0.1.39	llava:7b	19.09
Linux	Intel(R) Xeon(R) Platinum 8370C CPU @ 2.80GHz	unknown	125.81	Ubuntu 22.04.4 LTS	0.1.39	llava:13b	8.78
Linux	AMD Ryzen 7 PRO 4750U with Radeon Graphics	no_gpu	36.92	"Fedora Linux 40 (Workstation Edition)"	0.1.41	mistral:7b	5.68
Linux	AMD Ryzen 7 PRO 4750U with Radeon Graphics	no_gpu	36.92	"Fedora Linux 40 (Workstation Edition)"	0.1.41	llama3:8b	3.09
Linux	AMD Ryzen 7 PRO 4750U with Radeon Graphics	no_gpu	36.92	"Fedora Linux 40 (Workstation Edition)"	0.1.41	phi3:3.8b	8.52
Linux	AMD Ryzen 7 PRO 4750U with Radeon Graphics	no_gpu	36.92	"Fedora Linux 40 (Workstation Edition)"	0.1.41	gemma:2b	9.05
Linux	AMD Ryzen 7 PRO 4750U with Radeon Graphics	no_gpu	36.92	"Fedora Linux 40 (Workstation Edition)"	0.1.41	gemma:7b	3
Linux	AMD Ryzen 7 PRO 4750U with Radeon Graphics	no_gpu	36.92	"Fedora Linux 40 (Workstation Edition)"	0.1.41	llava:7b	4.91
Linux	AMD Ryzen 7 PRO 4750U with Radeon Graphics	no_gpu	36.92	"Fedora Linux 40 (Workstation Edition)"	0.1.41	llava:13b	2.39
Linux	Intel(R) Core(TM) i9-14900K	NVIDIA GeForce RTX 3090	62.54	Ubuntu 24.04 LTS	0.2.1	mistral:7b	125.69
Linux	Intel(R) Core(TM) i9-14900K	NVIDIA GeForce RTX 3090	62.54	Ubuntu 24.04 LTS	0.2.1	llama3:8b	111.46
Linux	Intel(R) Xeon(R) CPU E5-2640 v4 @ 2.40GHz	SVGA II Adapter	31.13	"AlmaLinux 8.9 (Midnight Oncilla)"	0.1.39	mistral:7b	7.79
Linux	Intel(R) Xeon(R) CPU E5-2640 v4 @ 2.40GHz	SVGA II Adapter	31.13	"AlmaLinux 8.9 (Midnight Oncilla)"	0.1.39	llama3:8b	7.35
Linux	Intel(R) Xeon(R) CPU E5-2640 v4 @ 2.40GHz	SVGA II Adapter	31.13	"AlmaLinux 8.9 (Midnight Oncilla)"	0.1.39	phi3:3.8b	13.56
Linux	Intel(R) Xeon(R) CPU E5-2640 v4 @ 2.40GHz	SVGA II Adapter	31.13	"AlmaLinux 8.9 (Midnight Oncilla)"	0.1.39	gemma:2b	18.52
Linux	Intel(R) Xeon(R) CPU E5-2640 v4 @ 2.40GHz	SVGA II Adapter	31.13	"AlmaLinux 8.9 (Midnight Oncilla)"	0.1.39	gemma:7b	6.56
Linux	Intel(R) Xeon(R) CPU E5-2640 v4 @ 2.40GHz	SVGA II Adapter	31.13	"AlmaLinux 8.9 (Midnight Oncilla)"	0.1.39	llava:7b	8.26
Linux	Intel(R) Xeon(R) CPU E5-2640 v4 @ 2.40GHz	SVGA II Adapter	31.13	"AlmaLinux 8.9 (Midnight Oncilla)"	0.1.39	llava:13b	4.78
Linux	Intel(R) Xeon(R) Gold 5118 CPU @ 2.30GHz	Tesla V100-SXM2-16GB	502.58	"Rocky Linux 9.3 (Blue Onyx)"	0.1.38	llama3:8b	82.56
Linux	Intel(R) Core(TM) i5-10500T CPU @ 2.30GHz	SVGA II Adapter	7.76	Ubuntu 24.04 LTS	0.1.34	mistral:7b	5.2
Linux	Intel(R) Core(TM) i5-10500T CPU @ 2.30GHz	SVGA II Adapter	7.76	Ubuntu 24.04 LTS	0.1.34	llama3:8b	4.85
Linux	Intel(R) Core(TM) i5-10500T CPU @ 2.30GHz	SVGA II Adapter	7.76	Ubuntu 24.04 LTS	0.1.34	phi3:3.8b	8.47
Linux	Intel(R) Core(TM) i5-10500T CPU @ 2.30GHz	SVGA II Adapter	7.76	Ubuntu 24.04 LTS	0.1.34	gemma:2b	12.68
Linux	Intel(R) Core(TM) i5-10500T CPU @ 2.30GHz	SVGA II Adapter	7.76	Ubuntu 24.04 LTS	0.1.34	gemma:7b	4.25
Linux	Intel(R) Core(TM) i5-10500T CPU @ 2.30GHz	SVGA II Adapter	7.76	Ubuntu 24.04 LTS	0.1.34	llava:7b	5.44
Linux	Intel(R) Core(TM) i9-9900K CPU @ 3.60GHz	NVIDIA GeForce RTX 2080 Ti	62.64	Debian GNU/Linux 12 (bookworm)	0.1.41	mistral:7b	90.08
Linux	Intel(R) Core(TM) i9-9900K CPU @ 3.60GHz	NVIDIA GeForce RTX 2080 Ti	62.64	Debian GNU/Linux 12 (bookworm)	0.1.41	llama3:8b	76.98
Linux	Intel(R) Core(TM) i9-9900K CPU @ 3.60GHz	NVIDIA GeForce RTX 2080 Ti	62.64	Debian GNU/Linux 12 (bookworm)	0.1.41	phi3:3.8b	119.8
Linux	Intel(R) Core(TM) i9-9900K CPU @ 3.60GHz	NVIDIA GeForce RTX 2080 Ti	62.64	Debian GNU/Linux 12 (bookworm)	0.1.41	gemma:2b	154.75
Linux	Intel(R) Core(TM) i9-9900K CPU @ 3.60GHz	NVIDIA GeForce RTX 2080 Ti	62.64	Debian GNU/Linux 12 (bookworm)	0.1.41	gemma:7b	75.24
Linux	Intel(R) Core(TM) i9-9900K CPU @ 3.60GHz	NVIDIA GeForce RTX 2080 Ti	62.64	Debian GNU/Linux 12 (bookworm)	0.1.41	llava:7b	94.86
Linux	Intel(R) Core(TM) i9-9900K CPU @ 3.60GHz	NVIDIA GeForce RTX 2080 Ti	62.64	Debian GNU/Linux 12 (bookworm)	0.1.41	llava:13b	57.65
Linux	Intel(R) Core(TM) i9-14900K	NVIDIA GeForce RTX 3090	62.54	Ubuntu 24.04 LTS	0.2.1	phi3:3.8b	168.08
Linux	Intel(R) Core(TM) i9-14900K	NVIDIA GeForce RTX 3090	62.54	Ubuntu 24.04 LTS	0.2.1	gemma:2b	209.77
Linux	Intel(R) Core(TM) i9-14900K	NVIDIA GeForce RTX 3090	62.54	Ubuntu 24.04 LTS	0.2.1	gemma:7b	105.62
Linux	Intel(R) Core(TM) i9-14900K	NVIDIA GeForce RTX 3090	62.54	Ubuntu 24.04 LTS	0.2.1	llava:7b	131.32
Linux	Intel(R) Core(TM) i9-14900K	NVIDIA GeForce RTX 3090	62.54	Ubuntu 24.04 LTS	0.2.1	llava:13b	82.82
Linux	AMD Ryzen 5 5600H with Radeon Graphics	NVIDIA GeForce RTX 3060 Laptop GPU	7.43	Ubuntu 22.04.4 LTS	0.2.1	mistral:7b	39.68
Linux	AMD Ryzen 5 5600H with Radeon Graphics	NVIDIA GeForce RTX 3060 Laptop GPU	7.43	Ubuntu 22.04.4 LTS	0.2.1	llama3:8b	19.3
Linux	AMD Ryzen 5 5600H with Radeon Graphics	NVIDIA GeForce RTX 3060 Laptop GPU	7.43	Ubuntu 22.04.4 LTS	0.2.1	phi3:3.8b	60.94
Linux	AMD Ryzen 5 5600H with Radeon Graphics	NVIDIA GeForce RTX 3060 Laptop GPU	7.43	Ubuntu 22.04.4 LTS	0.2.1	gemma:2b	72.78
Linux	AMD Ryzen 5 5600H with Radeon Graphics	NVIDIA GeForce RTX 3060 Laptop GPU	7.43	Ubuntu 22.04.4 LTS	0.2.1	gemma:7b	6.81
Linux	AMD Ryzen 5 5600H with Radeon Graphics	NVIDIA GeForce RTX 3060 Laptop GPU	7.43	Ubuntu 22.04.4 LTS	0.2.1	llava:7b	25.26
Linux	AMD Ryzen 7 7800X3D 8-Core Processor	no_gpu	61.91	Nobara Linux 40 (KDE Plasma)	0.2.1	mistral:7b	9.9
Linux	AMD Ryzen 7 7800X3D 8-Core Processor	no_gpu	61.91	Nobara Linux 40 (KDE Plasma)	0.2.1	llama3:8b	9.74
Linux	AMD Ryzen 7 7800X3D 8-Core Processor	no_gpu	61.91	Nobara Linux 40 (KDE Plasma)	0.2.1	phi3:3.8b	16.95
Linux	AMD Ryzen 7 7800X3D 8-Core Processor	no_gpu	61.91	Nobara Linux 40 (KDE Plasma)	0.2.1	gemma:2b	23.76
Linux	AMD Ryzen 7 7800X3D 8-Core Processor	no_gpu	61.91	Nobara Linux 40 (KDE Plasma)	0.2.1	gemma:7b	8.25
Linux	AMD Ryzen 7 7800X3D 8-Core Processor	no_gpu	61.91	Nobara Linux 40 (KDE Plasma)	0.2.1	llava:7b	10.82
Linux	AMD Ryzen 7 7800X3D 8-Core Processor	no_gpu	61.91	Nobara Linux 40 (KDE Plasma)	0.2.1	llava:13b	6.08
Linux	Intel(R) Core(TM) i9-14900K	NVIDIA GeForce RTX 3090	62.54	Ubuntu 24.04 LTS	0.2.1	mistral:7b	126.17
Linux	Intel(R) Core(TM) i9-14900K	NVIDIA GeForce RTX 3090	62.54	Ubuntu 24.04 LTS	0.2.1	llama3:8b	111.65
Linux	Intel(R) Core(TM) i9-14900K	NVIDIA GeForce RTX 3090	62.54	Ubuntu 24.04 LTS	0.2.1	phi3:3.8b	176.55
Linux	Intel(R) Core(TM) i9-14900K	NVIDIA GeForce RTX 3090	62.54	Ubuntu 24.04 LTS	0.2.1	gemma:2b	216.11
Linux	Intel(R) Core(TM) i9-14900K	NVIDIA GeForce RTX 3090	62.54	Ubuntu 24.04 LTS	0.2.1	gemma:7b	106.76
Linux	Intel(R) Core(TM) i9-14900K	NVIDIA GeForce RTX 3090	62.54	Ubuntu 24.04 LTS	0.2.1	llava:7b	130.77
Linux	Intel(R) Core(TM) i9-14900K	NVIDIA GeForce RTX 3090	62.54	Ubuntu 24.04 LTS	0.2.1	llava:13b	82.33
Linux	Intel(R) Core(TM) i5-8500T CPU @ 2.10GHz	NVIDIA Quadro P1000	28.20	Debian GNU/Linux 12 (bookworm)	0.1.45	mistral:7b	9.46
Linux	AMD Ryzen 7 8700G w/ Radeon 780M Graphics	Advanced Micro Devices, Inc. [AMD/ATI]	23.17	Ubuntu 22.04.4 LTS	0.1.41	mistral:7b	13.42
Linux	AMD Ryzen 7 8700G w/ Radeon 780M Graphics	Advanced Micro Devices, Inc. [AMD/ATI]	23.17	Ubuntu 22.04.4 LTS	0.1.41	llama3:8b	12.86
Linux	AMD Ryzen 7 8700G w/ Radeon 780M Graphics	Advanced Micro Devices, Inc. [AMD/ATI]	23.17	Ubuntu 22.04.4 LTS	0.1.41	phi3:3.8b	23.88
Linux	AMD Ryzen 7 8700G w/ Radeon 780M Graphics	Advanced Micro Devices, Inc. [AMD/ATI]	23.17	Ubuntu 22.04.4 LTS	0.1.41	gemma:2b	31.1
Linux	AMD Ryzen 7 8700G w/ Radeon 780M Graphics	Advanced Micro Devices, Inc. [AMD/ATI]	23.17	Ubuntu 22.04.4 LTS	0.1.41	gemma:7b	10.55
Linux	AMD Ryzen 7 8700G w/ Radeon 780M Graphics	Advanced Micro Devices, Inc. [AMD/ATI]	23.17	Ubuntu 22.04.4 LTS	0.1.41	llava:7b	14.19
Linux	AMD Ryzen 7 8700G w/ Radeon 780M Graphics	Advanced Micro Devices, Inc. [AMD/ATI]	23.17	Ubuntu 22.04.4 LTS	0.1.41	llava:13b	7.63
Linux	Intel(R) Core(TM) i5-8500T CPU @ 2.10GHz	NVIDIA Quadro P1000	28.20	Debian GNU/Linux 12 (bookworm)	0.1.45	llama3:8b	7.69
Linux	Intel(R) Core(TM) i5-8500T CPU @ 2.10GHz	NVIDIA Quadro P1000	28.20	Debian GNU/Linux 12 (bookworm)	0.1.45	phi3:3.8b	19.53
Linux	Intel(R) Core(TM) i5-8500T CPU @ 2.10GHz	NVIDIA Quadro P1000	28.20	Debian GNU/Linux 12 (bookworm)	0.1.45	gemma:2b	30.1
Linux	Intel(R) Core(TM) i5-8500T CPU @ 2.10GHz	NVIDIA Quadro P1000	28.20	Debian GNU/Linux 12 (bookworm)	0.1.45	gemma:7b	5.64
Linux	Intel(R) Core(TM) i5-8500T CPU @ 2.10GHz	NVIDIA Quadro P1000	28.20	Debian GNU/Linux 12 (bookworm)	0.1.45	llava:7b	8.88
Linux	Intel(R) Core(TM) i5-7500T CPU @ 2.70GHz	NVIDIA CMP 30HX	31.23	Ubuntu 24.04 LTS	0.1.39	mistral:7b	19.97
Linux	Intel(R) Core(TM) i5-7500T CPU @ 2.70GHz	NVIDIA CMP 30HX	31.23	Ubuntu 24.04 LTS	0.1.39	llama3:8b	19.69
Linux	Intel(R) Core(TM) i5-7500T CPU @ 2.70GHz	NVIDIA CMP 30HX	31.23	Ubuntu 24.04 LTS	0.1.39	phi3:3.8b	24.23
Linux	Intel(R) Core(TM) i5-7500T CPU @ 2.70GHz	NVIDIA CMP 30HX	31.23	Ubuntu 24.04 LTS	0.1.39	gemma:2b	51.18
Linux	Intel(R) Core(TM) i5-7500T CPU @ 2.70GHz	NVIDIA CMP 30HX	31.23	Ubuntu 24.04 LTS	0.1.39	gemma:7b	8.22
Linux	Intel(R) Core(TM) i5-7500T CPU @ 2.70GHz	NVIDIA CMP 30HX	31.23	Ubuntu 24.04 LTS	0.1.39	llava:7b	30.2
Linux	Intel(R) Core(TM) i5-7500T CPU @ 2.70GHz	NVIDIA CMP 30HX	31.23	Ubuntu 24.04 LTS	0.1.39	llava:13b	4.11
Linux	AMD Ryzen 7 3700X 8-Core Processor	unknown	31.25	Ubuntu 22.04.4 LTS	0.1.41	mistral:7b	4.1
Linux	AMD Ryzen 7 3700X 8-Core Processor	unknown	31.25	Ubuntu 22.04.4 LTS	0.1.41	llama3:8b	3.85
Linux	AMD Ryzen 7 3700X 8-Core Processor	unknown	31.25	Ubuntu 22.04.4 LTS	0.1.41	phi3:3.8b	6.62
Linux	AMD Ryzen 7 3700X 8-Core Processor	unknown	31.25	Ubuntu 22.04.4 LTS	0.1.41	gemma:2b	9.75
Linux	AMD Ryzen 7 3700X 8-Core Processor	unknown	31.25	Ubuntu 22.04.4 LTS	0.1.41	gemma:7b	3.26
Linux	AMD Ryzen 7 3700X 8-Core Processor	unknown	31.25	Ubuntu 22.04.4 LTS	0.1.41	llava:7b	4.15
Linux	AMD Ryzen 7 3700X 8-Core Processor	unknown	31.25	Ubuntu 22.04.4 LTS	0.1.41	llava:13b	2.3
Linux	Intel(R) Xeon(R) Gold 6133 CPU @ 2.50GHz	NVIDIA GeForce RTX 4090	94.26	Ubuntu 22.04.3 LTS	0.1.41	mistral:7b	114.56
Linux	Intel(R) Xeon(R) Gold 6133 CPU @ 2.50GHz	NVIDIA GeForce RTX 4090	94.26	Ubuntu 22.04.3 LTS	0.1.41	llama3:8b	85.29
Linux	Intel(R) Xeon(R) Gold 6133 CPU @ 2.50GHz	NVIDIA GeForce RTX 4090	94.26	Ubuntu 22.04.3 LTS	0.1.41	phi3:3.8b	138.21
Linux	Intel(R) Xeon(R) Gold 6133 CPU @ 2.50GHz	NVIDIA GeForce RTX 4090	94.26	Ubuntu 22.04.3 LTS	0.1.41	gemma:2b	147.94
Linux	Intel(R) Xeon(R) Gold 6133 CPU @ 2.50GHz	NVIDIA GeForce RTX 4090	94.26	Ubuntu 22.04.3 LTS	0.1.41	gemma:7b	88.08
Linux	Intel(R) Xeon(R) Gold 6133 CPU @ 2.50GHz	NVIDIA GeForce RTX 4090	94.26	Ubuntu 22.04.3 LTS	0.1.41	llava:7b	123.71
Linux	Intel(R) Xeon(R) Gold 6133 CPU @ 2.50GHz	NVIDIA GeForce RTX 4090	94.26	Ubuntu 22.04.3 LTS	0.1.41	llava:13b	2.77
Linux	Intel(R) Core(TM) i7-3770S CPU @ 3.10GHz	NVIDIA GeForce GTX 1660 SUPER	15.61	Ubuntu 22.04.4 LTS	0.1.41	mistral:7b	26.49
Linux	Intel(R) Core(TM) i7-3770S CPU @ 3.10GHz	NVIDIA GeForce GTX 1660 SUPER	15.61	Ubuntu 22.04.4 LTS	0.1.41	llama3:8b	24.91
Linux	Intel(R) Core(TM) i7-3770S CPU @ 3.10GHz	NVIDIA GeForce GTX 1660 SUPER	15.61	Ubuntu 22.04.4 LTS	0.1.41	phi3:3.8b	41.71
Linux	Intel(R) Core(TM) i7-3770S CPU @ 3.10GHz	NVIDIA GeForce GTX 1660 SUPER	15.61	Ubuntu 22.04.4 LTS	0.1.41	gemma:2b	64.37
Linux	Intel(R) Core(TM) i7-3770S CPU @ 3.10GHz	NVIDIA GeForce GTX 1660 SUPER	15.61	Ubuntu 22.04.4 LTS	0.1.41	gemma:7b	6.54
Linux	Intel(R) Core(TM) i7-3770S CPU @ 3.10GHz	NVIDIA GeForce GTX 1660 SUPER	15.61	Ubuntu 22.04.4 LTS	0.1.41	llava:7b	40.21
Linux	Intel(R) Core(TM) i7-3770S CPU @ 3.10GHz	NVIDIA GeForce GTX 1660 SUPER	15.61	Ubuntu 22.04.4 LTS	0.1.41	llava:13b	3.13
Linux	AMD Ryzen 7 3750H with Radeon Vega Mobile Gfx	no_gpu	31.25	Debian GNU/Linux 12 (bookworm)	0.1.39	mistral:7b	3.93
Linux	AMD Ryzen 7 3750H with Radeon Vega Mobile Gfx	no_gpu	31.25	Debian GNU/Linux 12 (bookworm)	0.1.39	llama3:8b	3.63
Linux	AMD Ryzen 7 3750H with Radeon Vega Mobile Gfx	no_gpu	31.25	Debian GNU/Linux 12 (bookworm)	0.1.39	phi3:3.8b	8.08
Linux	AMD Ryzen 7 3750H with Radeon Vega Mobile Gfx	no_gpu	31.25	Debian GNU/Linux 12 (bookworm)	0.1.39	gemma:2b	9.58
Linux	AMD Ryzen 7 3750H with Radeon Vega Mobile Gfx	no_gpu	31.25	Debian GNU/Linux 12 (bookworm)	0.1.39	gemma:7b	3.07
Linux	AMD Ryzen 7 3750H with Radeon Vega Mobile Gfx	no_gpu	31.25	Debian GNU/Linux 12 (bookworm)	0.1.39	llava:7b	3.96
Linux	AMD Ryzen 7 3750H with Radeon Vega Mobile Gfx	no_gpu	31.25	Debian GNU/Linux 12 (bookworm)	0.1.39	llava:13b	2.27
Linux	Intel(R) Xeon(R) Gold 6133 CPU @ 2.50GHz	NVIDIA GeForce RTX 4090	94.26	Ubuntu 22.04.3 LTS	0.1.41	mistral:7b	114.51
Linux	Intel(R) Xeon(R) Gold 6133 CPU @ 2.50GHz	NVIDIA GeForce RTX 4090	94.26	Ubuntu 22.04.3 LTS	0.1.41	llama3:8b	43.53
Linux	Intel(R) Xeon(R) Gold 6133 CPU @ 2.50GHz	NVIDIA GeForce RTX 4090	94.26	Ubuntu 22.04.3 LTS	0.1.41	phi3:3.8b	136.98
Linux	Intel(R) Xeon(R) Gold 6133 CPU @ 2.50GHz	NVIDIA GeForce RTX 4090	94.26	Ubuntu 22.04.3 LTS	0.1.41	gemma:2b	151.25
Linux	Intel(R) Xeon(R) Gold 6133 CPU @ 2.50GHz	NVIDIA GeForce RTX 4090	94.26	Ubuntu 22.04.3 LTS	0.1.41	gemma:7b	87.63
Linux	Intel(R) Xeon(R) Gold 6133 CPU @ 2.50GHz	NVIDIA GeForce RTX 4090	94.26	Ubuntu 22.04.3 LTS	0.1.41	llava:7b	125.47
Linux	Intel(R) Xeon(R) Gold 6133 CPU @ 2.50GHz	NVIDIA GeForce RTX 4090	94.26	Ubuntu 22.04.3 LTS	0.1.41	llava:13b	63.08
Linux	AMD Ryzen 9 7950X 16-Core Processor	NVIDIA GeForce RTX 3060	62.36	Ubuntu 22.04.4 LTS	0.1.41	mistral:7b	55.91
Linux	AMD Ryzen 9 7950X 16-Core Processor	NVIDIA GeForce RTX 3060	62.36	Ubuntu 22.04.4 LTS	0.1.41	llama3:8b	51.91
Linux	AMD Ryzen 9 7950X 16-Core Processor	NVIDIA GeForce RTX 3060	62.36	Ubuntu 22.04.4 LTS	0.1.41	phi3:3.8b	79.83
Linux	AMD Ryzen 9 7950X 16-Core Processor	NVIDIA GeForce RTX 3060	62.36	Ubuntu 22.04.4 LTS	0.1.41	gemma:2b	104.42
Linux	AMD Ryzen 9 7950X 16-Core Processor	NVIDIA GeForce RTX 3060	62.36	Ubuntu 22.04.4 LTS	0.1.41	gemma:7b	49
Linux	AMD Ryzen 9 7950X 16-Core Processor	NVIDIA GeForce RTX 3060	62.36	Ubuntu 22.04.4 LTS	0.1.41	llava:7b	57.6
Linux	AMD Ryzen 9 7950X 16-Core Processor	NVIDIA GeForce RTX 3060	62.36	Ubuntu 22.04.4 LTS	0.1.41	llava:13b	36.08
Linux	Intel(R) Core(TM) i7-3770S CPU @ 3.10GHz	NVIDIA GeForce GTX 1660 SUPER	15.61	Ubuntu 22.04.4 LTS	0.1.41	mistral:7b	26.44
Linux	Intel(R) Core(TM) i7-3770S CPU @ 3.10GHz	NVIDIA GeForce GTX 1660 SUPER	15.61	Ubuntu 22.04.4 LTS	0.1.41	llama3:8b	24.44
Linux	Intel(R) Core(TM) i7-3770S CPU @ 3.10GHz	NVIDIA GeForce GTX 1660 SUPER	15.61	Ubuntu 22.04.4 LTS	0.1.41	phi3:3.8b	37.53
Linux	Intel(R) Core(TM) i7-3770S CPU @ 3.10GHz	NVIDIA GeForce GTX 1660 SUPER	15.61	Ubuntu 22.04.4 LTS	0.1.41	gemma:2b	64.67
Linux	Intel(R) Core(TM) i7-3770S CPU @ 3.10GHz	NVIDIA GeForce GTX 1660 SUPER	15.61	Ubuntu 22.04.4 LTS	0.1.41	gemma:7b	6.61
Linux	Intel(R) Core(TM) i7-3770S CPU @ 3.10GHz	NVIDIA GeForce GTX 1660 SUPER	15.61	Ubuntu 22.04.4 LTS	0.1.41	llava:7b	39.94
Linux	Intel(R) Core(TM) i7-3770S CPU @ 3.10GHz	NVIDIA GeForce GTX 1660 SUPER	15.61	Ubuntu 22.04.4 LTS	0.1.41	llava:13b	3.18
Linux	Intel(R) Xeon(R) Gold 6240R CPU @ 2.40GHz	NVIDIA RTX A6000	187.53	Ubuntu 22.04.4 LTS	0.1.39	mistral:7b	101.44
Linux	Intel(R) Xeon(R) Gold 6240R CPU @ 2.40GHz	NVIDIA RTX A6000	187.53	Ubuntu 22.04.4 LTS	0.1.39	llama3:8b	84.27
Linux	Intel(R) Xeon(R) Gold 6240R CPU @ 2.40GHz	NVIDIA RTX A6000	187.53	Ubuntu 22.04.4 LTS	0.1.39	phi3:3.8b	134.64
Linux	Intel(R) Xeon(R) Gold 6240R CPU @ 2.40GHz	NVIDIA RTX A6000	187.53	Ubuntu 22.04.4 LTS	0.1.39	gemma:2b	167.22
Linux	Intel(R) Xeon(R) Gold 6240R CPU @ 2.40GHz	NVIDIA RTX A6000	187.53	Ubuntu 22.04.4 LTS	0.1.39	gemma:7b	85.36
Linux	Intel(R) Xeon(R) Gold 6240R CPU @ 2.40GHz	NVIDIA RTX A6000	187.53	Ubuntu 22.04.4 LTS	0.1.39	llava:7b	104.74
Linux	Intel(R) Xeon(R) Gold 6240R CPU @ 2.40GHz	NVIDIA RTX A6000	187.53	Ubuntu 22.04.4 LTS	0.1.39	llava:13b	66.52
Linux	Intel(R) Xeon(R) Platinum 8358P CPU @ 2.60GHz	SVGA II Adapter	7.71	Ubuntu 24.04 LTS	0.1.41	mistral:7b	15.64
Linux	Intel(R) Xeon(R) Platinum 8358P CPU @ 2.60GHz	SVGA II Adapter	7.71	Ubuntu 24.04 LTS	0.1.41	llama3:8b	15.52
Linux	Intel(R) Xeon(R) Platinum 8358P CPU @ 2.60GHz	SVGA II Adapter	7.71	Ubuntu 24.04 LTS	0.1.41	phi3:3.8b	22.14
Linux	Intel(R) Xeon(R) Platinum 8358P CPU @ 2.60GHz	SVGA II Adapter	7.71	Ubuntu 24.04 LTS	0.1.41	gemma:2b	32.47
Linux	Intel(R) Xeon(R) Platinum 8358P CPU @ 2.60GHz	SVGA II Adapter	7.71	Ubuntu 24.04 LTS	0.1.41	gemma:7b	13.02
Linux	Intel(R) Xeon(R) Platinum 8358P CPU @ 2.60GHz	SVGA II Adapter	7.71	Ubuntu 24.04 LTS	0.1.41	llava:7b	16.46
Linux	Intel(R) Core(TM) i9-7980XE CPU @ 2.60GHz	NVIDIA GeForce RTX 3090	62.48	Ubuntu 20.04.6 LTS	0.1.41	mistral:7b	116.43
Linux	Intel(R) Core(TM) i9-7980XE CPU @ 2.60GHz	NVIDIA GeForce RTX 3090	62.48	Ubuntu 20.04.6 LTS	0.1.41	llama3:8b	94.38
Linux	Intel(R) Core(TM) i9-7980XE CPU @ 2.60GHz	NVIDIA GeForce RTX 3090	62.48	Ubuntu 20.04.6 LTS	0.1.41	phi3:3.8b	150.82
Linux	Intel(R) Core(TM) i9-7980XE CPU @ 2.60GHz	NVIDIA GeForce RTX 3090	62.48	Ubuntu 20.04.6 LTS	0.1.41	gemma:2b	183.27
Linux	Intel(R) Core(TM) i9-7980XE CPU @ 2.60GHz	NVIDIA GeForce RTX 3090	62.48	Ubuntu 20.04.6 LTS	0.1.41	gemma:7b	98.03
Linux	Intel(R) Core(TM) i9-7980XE CPU @ 2.60GHz	NVIDIA GeForce RTX 3090	62.48	Ubuntu 20.04.6 LTS	0.1.41	llava:7b	119.69
Linux	Intel(R) Core(TM) i9-7980XE CPU @ 2.60GHz	NVIDIA GeForce RTX 3090	62.48	Ubuntu 20.04.6 LTS	0.1.41	llava:13b	77.75
Linux	AMD Ryzen 7 7840U w/ Radeon 780M Graphics	no_gpu	30.66	"Arch Linux"	0.1.41	mistral:7b	6.09
Linux	AMD Ryzen 7 7840U w/ Radeon 780M Graphics	no_gpu	30.66	"Arch Linux"	0.1.41	llama3:8b	5.64
Linux	AMD Ryzen 7 7840U w/ Radeon 780M Graphics	no_gpu	30.66	"Arch Linux"	0.1.41	phi3:3.8b	9.54
Linux	AMD Ryzen 7 7840U w/ Radeon 780M Graphics	no_gpu	30.66	"Arch Linux"	0.1.41	gemma:2b	14.14
Linux	AMD Ryzen 7 7840U w/ Radeon 780M Graphics	no_gpu	30.66	"Arch Linux"	0.1.41	gemma:7b	4.85
Linux	AMD Ryzen 7 7840U w/ Radeon 780M Graphics	no_gpu	30.66	"Arch Linux"	0.1.41	llava:7b	6.21
Linux	AMD Ryzen 7 7840U w/ Radeon 780M Graphics	no_gpu	30.66	"Arch Linux"	0.1.41	llava:13b	3.43
Linux	Intel(R) Core(TM) i7-10700KF CPU @ 3.80GHz	NVIDIA GeForce RTX 3090	62.71	Ubuntu 22.04.4 LTS	0.1.41	mistral:7b	110.49
Linux	Intel(R) Core(TM) i7-10700KF CPU @ 3.80GHz	NVIDIA GeForce RTX 3090	62.71	Ubuntu 22.04.4 LTS	0.1.41	llama3:8b	92.84
Linux	Intel(R) Core(TM) i7-10700KF CPU @ 3.80GHz	NVIDIA GeForce RTX 3090	62.71	Ubuntu 22.04.4 LTS	0.1.41	phi3:3.8b	140.35
Linux	Intel(R) Core(TM) i7-10700KF CPU @ 3.80GHz	NVIDIA GeForce RTX 3090	62.71	Ubuntu 22.04.4 LTS	0.1.41	gemma:2b	175.36
Linux	Intel(R) Core(TM) i7-10700KF CPU @ 3.80GHz	NVIDIA GeForce RTX 3090	62.71	Ubuntu 22.04.4 LTS	0.1.41	gemma:7b	93.58
Linux	Intel(R) Core(TM) i7-10700KF CPU @ 3.80GHz	NVIDIA GeForce RTX 3090	62.71	Ubuntu 22.04.4 LTS	0.1.41	llava:7b	115.3
Linux	Intel(R) Core(TM) i7-10700KF CPU @ 3.80GHz	NVIDIA GeForce RTX 3090	62.71	Ubuntu 22.04.4 LTS	0.1.41	llava:13b	73.36
Linux	AMD EPYC 7V12 64-Core Processor	Tesla T4	27.41	Ubuntu 20.04.6 LTS	0.1.42	mistral:7b	46.87
Linux	AMD EPYC 7V12 64-Core Processor	Tesla T4	27.41	Ubuntu 20.04.6 LTS	0.1.42	llama3:8b	40.91
Linux	AMD EPYC 7V12 64-Core Processor	Tesla T4	27.41	Ubuntu 20.04.6 LTS	0.1.42	phi3:3.8b	61.54
Linux	AMD EPYC 7V12 64-Core Processor	Tesla T4	27.41	Ubuntu 20.04.6 LTS	0.1.42	gemma:2b	95
Linux	AMD EPYC 7V12 64-Core Processor	Tesla T4	27.41	Ubuntu 20.04.6 LTS	0.1.42	gemma:7b	35.5
Linux	AMD EPYC 7V12 64-Core Processor	Tesla T4	27.41	Ubuntu 20.04.6 LTS	0.1.42	llava:7b	47.31
Linux	AMD EPYC 7V12 64-Core Processor	Tesla T4	27.41	Ubuntu 20.04.6 LTS	0.1.42	llava:13b	27.2
Linux	Intel(R) Core(TM) i7-10700KF CPU @ 3.80GHz	NVIDIA GeForce RTX 3090	62.71	Ubuntu 22.04.4 LTS	0.1.41	llama3:8b	94.25
Linux	Intel(R) Core(TM) i7-10700KF CPU @ 3.80GHz	NVIDIA GeForce RTX 3090	62.71	Ubuntu 22.04.4 LTS	0.1.41	gemma:7b	95.04
Linux	Intel(R) Core(TM) i7-10700KF CPU @ 3.80GHz	NVIDIA GeForce RTX 3090	62.71	Ubuntu 22.04.4 LTS	0.1.42	llama3:8b	92.85
Linux	Intel(R) Core(TM) i7-10700KF CPU @ 3.80GHz	NVIDIA GeForce RTX 3090	62.71	Ubuntu 22.04.4 LTS	0.1.42	gemma:7b	93.08
Linux	Intel(R) Core(TM) i7-10700KF CPU @ 3.80GHz	NVIDIA GeForce RTX 3090	62.71	Ubuntu 22.04.4 LTS	0.1.42	llama3:8b	92.81
Linux	Intel(R) Core(TM) i7-10700KF CPU @ 3.80GHz	NVIDIA GeForce RTX 3090	62.71	Ubuntu 22.04.4 LTS	0.1.42	gemma:7b	92.15
Linux	12th Gen Intel(R) Core(TM) i7-12700H	NVIDIA GeForce RTX 3050 Ti Laptop GPU	31.01	Ubuntu 22.04.2 LTS	0.1.42	mistral:7b	17.57
Linux	12th Gen Intel(R) Core(TM) i7-12700H	NVIDIA GeForce RTX 3050 Ti Laptop GPU	31.01	Ubuntu 22.04.2 LTS	0.1.42	llama3:8b	11.62
Linux	12th Gen Intel(R) Core(TM) i7-12700H	NVIDIA GeForce RTX 3050 Ti Laptop GPU	31.01	Ubuntu 22.04.2 LTS	0.1.42	phi3:3.8b	43.75
Linux	12th Gen Intel(R) Core(TM) i7-12700H	NVIDIA GeForce RTX 3050 Ti Laptop GPU	31.01	Ubuntu 22.04.2 LTS	0.1.42	gemma:2b	70.98
Linux	12th Gen Intel(R) Core(TM) i7-12700H	NVIDIA GeForce RTX 3050 Ti Laptop GPU	31.01	Ubuntu 22.04.2 LTS	0.1.42	gemma:7b	8.02
Linux	12th Gen Intel(R) Core(TM) i7-12700H	NVIDIA GeForce RTX 3050 Ti Laptop GPU	31.01	Ubuntu 22.04.2 LTS	0.1.42	llava:7b	13.26
Linux	12th Gen Intel(R) Core(TM) i7-12700H	NVIDIA GeForce RTX 3050 Ti Laptop GPU	31.01	Ubuntu 22.04.2 LTS	0.1.42	llava:13b	5.01
Linux	Intel(R) Xeon(R) Gold 5115 CPU @ 2.40GHz	Tesla P40	502.55	Ubuntu 22.04.4 LTS	0.1.42	mistral:7b	47.88
Linux	Intel(R) Xeon(R) Gold 5115 CPU @ 2.40GHz	Tesla P40	502.55	Ubuntu 22.04.4 LTS	0.1.42	llama3:8b	41.62
Linux	Intel(R) Xeon(R) Gold 5115 CPU @ 2.40GHz	Tesla P40	502.55	Ubuntu 22.04.4 LTS	0.1.42	phi3:3.8b	60.73
Linux	Intel(R) Xeon(R) Gold 5115 CPU @ 2.40GHz	Tesla P40	502.55	Ubuntu 22.04.4 LTS	0.1.42	gemma:2b	99.81
Linux	Intel(R) Xeon(R) Gold 5115 CPU @ 2.40GHz	Tesla P40	502.55	Ubuntu 22.04.4 LTS	0.1.42	gemma:7b	38.34
Linux	Intel(R) Xeon(R) Gold 5115 CPU @ 2.40GHz	Tesla P40	502.55	Ubuntu 22.04.4 LTS	0.1.42	llava:7b	51.29
Linux	Intel(R) Xeon(R) Gold 5115 CPU @ 2.40GHz	Tesla P40	502.55	Ubuntu 22.04.4 LTS	0.1.42	llava:13b	29.42
Linux	Intel(R) Xeon(R) Gold 5118 CPU @ 2.30GHz	Tesla V100-SXM2-16GB	502.58	"Rocky Linux 9.3 (Blue Onyx)"	0.1.38	gemma:2b	154.99
Linux	12th Gen Intel(R) Core(TM) i7-12700KF	NVIDIA GeForce RTX 4070	27.42	Ubuntu 20.04.5 LTS	0.1.39	mistral:7b	85.37
Linux	12th Gen Intel(R) Core(TM) i7-12700KF	NVIDIA GeForce RTX 4070	27.42	Ubuntu 20.04.5 LTS	0.1.39	llama3:8b	76.09
Linux	12th Gen Intel(R) Core(TM) i7-12700KF	NVIDIA GeForce RTX 4070	27.42	Ubuntu 20.04.5 LTS	0.1.39	phi3:3.8b	120.33
Linux	12th Gen Intel(R) Core(TM) i7-12700KF	NVIDIA GeForce RTX 4070	27.42	Ubuntu 20.04.5 LTS	0.1.39	gemma:2b	158.25
Linux	12th Gen Intel(R) Core(TM) i7-12700KF	NVIDIA GeForce RTX 4070	27.42	Ubuntu 20.04.5 LTS	0.1.39	gemma:7b	69.34
Linux	12th Gen Intel(R) Core(TM) i7-12700KF	NVIDIA GeForce RTX 4070	27.42	Ubuntu 20.04.5 LTS	0.1.39	llava:7b	90.35
Linux	12th Gen Intel(R) Core(TM) i7-12700KF	NVIDIA GeForce RTX 4070	27.42	Ubuntu 20.04.5 LTS	0.1.39	llava:13b	53.25
Linux	12th Gen Intel(R) Core(TM) i7-12700KF	NVIDIA GeForce RTX 4070	27.42	Ubuntu 20.04.5 LTS	0.1.39	mistral:7b	85.65
Linux	12th Gen Intel(R) Core(TM) i7-12700KF	NVIDIA GeForce RTX 4070	27.42	Ubuntu 20.04.5 LTS	0.1.39	llama3:8b	75.29
Linux	12th Gen Intel(R) Core(TM) i7-12700KF	NVIDIA GeForce RTX 4070	27.42	Ubuntu 20.04.5 LTS	0.1.39	phi3:3.8b	122.11
Linux	12th Gen Intel(R) Core(TM) i7-12700KF	NVIDIA GeForce RTX 4070	27.42	Ubuntu 20.04.5 LTS	0.1.39	gemma:2b	158.09
Linux	12th Gen Intel(R) Core(TM) i7-12700KF	NVIDIA GeForce RTX 4070	27.42	Ubuntu 20.04.5 LTS	0.1.39	gemma:7b	69.88
Linux	12th Gen Intel(R) Core(TM) i7-12700KF	NVIDIA GeForce RTX 4070	27.42	Ubuntu 20.04.5 LTS	0.1.39	llava:7b	88.89
Linux	12th Gen Intel(R) Core(TM) i7-12700KF	NVIDIA GeForce RTX 4070	27.42	Ubuntu 20.04.5 LTS	0.1.39	llava:13b	53.24
Linux	12th Gen Intel(R) Core(TM) i7-12700KF	NVIDIA GeForce RTX 4070	27.42	Ubuntu 20.04.5 LTS	0.1.39	mistral:7b	87.06
Linux	12th Gen Intel(R) Core(TM) i7-12700KF	NVIDIA GeForce RTX 4070	27.42	Ubuntu 20.04.5 LTS	0.1.39	llama3:8b	76.29
Linux	12th Gen Intel(R) Core(TM) i7-12700KF	NVIDIA GeForce RTX 4070	27.42	Ubuntu 20.04.5 LTS	0.1.39	phi3:3.8b	120.99
Linux	12th Gen Intel(R) Core(TM) i7-12700KF	NVIDIA GeForce RTX 4070	27.42	Ubuntu 20.04.5 LTS	0.1.39	gemma:2b	157.9
Linux	12th Gen Intel(R) Core(TM) i7-12700KF	NVIDIA GeForce RTX 4070	27.42	Ubuntu 20.04.5 LTS	0.1.39	gemma:7b	69.95
Linux	12th Gen Intel(R) Core(TM) i7-12700KF	NVIDIA GeForce RTX 4070	27.42	Ubuntu 20.04.5 LTS	0.1.39	llava:7b	90.31
Linux	12th Gen Intel(R) Core(TM) i7-12700KF	NVIDIA GeForce RTX 4070	27.42	Ubuntu 20.04.5 LTS	0.1.39	llava:13b	52.1
Linux	AMD Ryzen 9 3900X 12-Core Processor	NVIDIA GeForce GTX 1080 Ti	62.72	Pop!_OS 22.04 LTS	0.1.42	mistral:7b	46.18
Linux	AMD Ryzen 9 3900X 12-Core Processor	NVIDIA GeForce GTX 1080 Ti	62.72	Pop!_OS 22.04 LTS	0.1.42	llama3:8b	42.79
Linux	AMD Ryzen 9 3900X 12-Core Processor	NVIDIA GeForce GTX 1080 Ti	62.72	Pop!_OS 22.04 LTS	0.1.42	phi3:3.8b	51.22
Linux	AMD Ryzen 9 3900X 12-Core Processor	NVIDIA GeForce GTX 1080 Ti	62.72	Pop!_OS 22.04 LTS	0.1.42	qwen:1.8b	102.69
Linux	AMD Ryzen 9 3900X 12-Core Processor	NVIDIA GeForce GTX 1080 Ti	62.72	Pop!_OS 22.04 LTS	0.1.42	phi:2.7b	83.02
Linux	AMD Ryzen 9 3900X 12-Core Processor	NVIDIA GeForce GTX 1080 Ti	62.72	Pop!_OS 22.04 LTS	0.1.42	gemma:2b	105.39
Linux	AMD Ryzen 9 3900X 12-Core Processor	NVIDIA GeForce GTX 1080 Ti	62.72	Pop!_OS 22.04 LTS	0.1.42	gemma:7b	40.34
Linux	AMD Ryzen 9 3900X 12-Core Processor	NVIDIA GeForce GTX 1080 Ti	62.72	Pop!_OS 22.04 LTS	0.1.42	llava:7b	6.79
Linux	AMD Ryzen 9 3900X 12-Core Processor	NVIDIA GeForce GTX 1080 Ti	62.72	Pop!_OS 22.04 LTS	0.1.42	llava:13b	6.44
Linux	Intel(R) Core(TM) Ultra 9 185H	NVIDIA GeForce RTX 4050 Laptop GPU	46.80	Ubuntu 22.04.3 LTS	0.1.41	mistral:7b	32.56
Linux	Intel(R) Core(TM) Ultra 9 185H	NVIDIA GeForce RTX 4050 Laptop GPU	46.80	Ubuntu 22.04.3 LTS	0.1.41	llama3:8b	17.85
Linux	Intel(R) Core(TM) Ultra 9 185H	NVIDIA GeForce RTX 4050 Laptop GPU	46.80	Ubuntu 22.04.3 LTS	0.1.41	phi3:3.8b	45.92
Linux	Intel(R) Core(TM) Ultra 9 185H	NVIDIA GeForce RTX 4050 Laptop GPU	46.80	Ubuntu 22.04.3 LTS	0.1.41	gemma:2b	62.9
Linux	Intel(R) Core(TM) Ultra 9 185H	NVIDIA GeForce RTX 4050 Laptop GPU	46.80	Ubuntu 22.04.3 LTS	0.1.41	gemma:7b	11.38
Linux	Intel(R) Core(TM) Ultra 9 185H	NVIDIA GeForce RTX 4050 Laptop GPU	46.80	Ubuntu 22.04.3 LTS	0.1.41	llava:7b	25.04
Linux	Intel(R) Core(TM) Ultra 9 185H	NVIDIA GeForce RTX 4050 Laptop GPU	46.80	Ubuntu 22.04.3 LTS	0.1.41	llava:13b	6.23
Linux	Intel(R) Core(TM) i7-7700T CPU @ 2.90GHz	Quadro P4000	31.30	"Ubuntu 22.04.4 LTS"	0.1.42	mistral:7b	32.41
Linux	Intel(R) Core(TM) i7-7700T CPU @ 2.90GHz	Quadro P4000	31.30	"Ubuntu 22.04.4 LTS"	0.1.42	llama3:8b	29.33
Linux	Intel(R) Core(TM) i7-7700T CPU @ 2.90GHz	Quadro P4000	31.30	"Ubuntu 22.04.4 LTS"	0.1.42	phi3:3.8b	42.44
Linux	Intel(R) Core(TM) i7-7700T CPU @ 2.90GHz	Quadro P4000	31.30	"Ubuntu 22.04.4 LTS"	0.1.42	gemma:2b	73.38
Linux	Intel(R) Core(TM) i7-7700T CPU @ 2.90GHz	Quadro P4000	31.30	"Ubuntu 22.04.4 LTS"	0.1.42	gemma:7b	27.67
Linux	Intel(R) Core(TM) i7-7700T CPU @ 2.90GHz	Quadro P4000	31.30	"Ubuntu 22.04.4 LTS"	0.1.42	llava:7b	34.82
Linux	Intel(R) Core(TM) i7-7700T CPU @ 2.90GHz	Quadro P4000	31.30	"Ubuntu 22.04.4 LTS"	0.1.42	llava:13b	5.75
Linux	12th Gen Intel(R) Core(TM) i7-12700KF	NVIDIA GeForce RTX 4070	27.42	Ubuntu 20.04.5 LTS	0.1.39	mistral:7b	86.08
Linux	12th Gen Intel(R) Core(TM) i7-12700KF	NVIDIA GeForce RTX 4070	27.42	Ubuntu 20.04.5 LTS	0.1.39	llama3:8b	75.54
Linux	12th Gen Intel(R) Core(TM) i7-12700KF	NVIDIA GeForce RTX 4070	27.42	Ubuntu 20.04.5 LTS	0.1.39	phi3:3.8b	122.47
Linux	12th Gen Intel(R) Core(TM) i7-12700KF	NVIDIA GeForce RTX 4070	27.42	Ubuntu 20.04.5 LTS	0.1.39	gemma:2b	136.3
Linux	12th Gen Intel(R) Core(TM) i7-12700KF	NVIDIA GeForce RTX 4070	27.42	Ubuntu 20.04.5 LTS	0.1.39	gemma:7b	69.93
Linux	12th Gen Intel(R) Core(TM) i7-12700KF	NVIDIA GeForce RTX 4070	27.42	Ubuntu 20.04.5 LTS	0.1.39	llava:7b	90.53
Linux	12th Gen Intel(R) Core(TM) i7-12700KF	NVIDIA GeForce RTX 4070	27.42	Ubuntu 20.04.5 LTS	0.1.39	llava:13b	53.12
Linux	12th Gen Intel(R) Core(TM) i7-12700KF	NVIDIA GeForce RTX 4070	27.42	Ubuntu 20.04.5 LTS	0.1.39	mistral:7b	86.8
Linux	12th Gen Intel(R) Core(TM) i7-12700KF	NVIDIA GeForce RTX 4070	27.42	Ubuntu 20.04.5 LTS	0.1.39	llama3:8b	76.3
Linux	12th Gen Intel(R) Core(TM) i7-12700KF	NVIDIA GeForce RTX 4070	27.42	Ubuntu 20.04.5 LTS	0.1.39	phi3:3.8b	122.68
Linux	12th Gen Intel(R) Core(TM) i7-12700KF	NVIDIA GeForce RTX 4070	27.42	Ubuntu 20.04.5 LTS	0.1.39	gemma:2b	150.93
Linux	12th Gen Intel(R) Core(TM) i7-12700KF	NVIDIA GeForce RTX 4070	27.42	Ubuntu 20.04.5 LTS	0.1.39	gemma:7b	67.93
Linux	12th Gen Intel(R) Core(TM) i7-12700KF	NVIDIA GeForce RTX 4070	27.42	Ubuntu 20.04.5 LTS	0.1.39	llava:7b	90.43
Linux	12th Gen Intel(R) Core(TM) i7-12700KF	NVIDIA GeForce RTX 4070	27.42	Ubuntu 20.04.5 LTS	0.1.39	llava:13b	52.98
Linux	12th Gen Intel(R) Core(TM) i7-12700KF	NVIDIA GeForce RTX 4070	27.42	Ubuntu 20.04.5 LTS	0.1.39	mistral:7b	82.9
Linux	12th Gen Intel(R) Core(TM) i7-12700KF	NVIDIA GeForce RTX 4070	27.42	Ubuntu 20.04.5 LTS	0.1.39	llama3:8b	76.54
Linux	12th Gen Intel(R) Core(TM) i7-12700KF	NVIDIA GeForce RTX 4070	27.42	Ubuntu 20.04.5 LTS	0.1.39	phi3:3.8b	122.56
Linux	12th Gen Intel(R) Core(TM) i7-12700KF	NVIDIA GeForce RTX 4070	27.42	Ubuntu 20.04.5 LTS	0.1.39	gemma:2b	158.54
Linux	12th Gen Intel(R) Core(TM) i7-12700KF	NVIDIA GeForce RTX 4070	27.42	Ubuntu 20.04.5 LTS	0.1.39	gemma:7b	69.69
Linux	12th Gen Intel(R) Core(TM) i7-12700KF	NVIDIA GeForce RTX 4070	27.42	Ubuntu 20.04.5 LTS	0.1.39	llava:7b	86.06
Linux	12th Gen Intel(R) Core(TM) i7-12700KF	NVIDIA GeForce RTX 4070	27.42	Ubuntu 20.04.5 LTS	0.1.39	llava:13b	53.57
Linux	AMD Ryzen 5 5600G with Radeon Graphics	NVIDIA GeForce RTX 3060	93.67	"Fedora Linux 40 (Workstation Edition)"	0.1.48	mistral:7b	65.62
Linux	AMD Ryzen 5 5600G with Radeon Graphics	NVIDIA GeForce RTX 3060	93.67	"Fedora Linux 40 (Workstation Edition)"	0.1.48	llama3:8b	58.99
Linux	AMD Ryzen 5 7600X 6-Core Processor	NVIDIA GeForce RTX 4070	30.95	Ubuntu 22.04.4 LTS	0.1.43	mistral:7b	69.2
Linux	AMD Ryzen 5 7600X 6-Core Processor	NVIDIA GeForce RTX 4070	30.95	Ubuntu 22.04.4 LTS	0.1.43	llama3:8b	66.19
Linux	AMD Ryzen 5 7600X 6-Core Processor	NVIDIA GeForce RTX 4070	30.95	Ubuntu 22.04.4 LTS	0.1.43	phi3:3.8b	101.36
Linux	AMD Ryzen 5 7600X 6-Core Processor	NVIDIA GeForce RTX 4070	30.95	Ubuntu 22.04.4 LTS	0.1.43	gemma:2b	111.29
Linux	AMD Ryzen 5 7600X 6-Core Processor	NVIDIA GeForce RTX 4070	30.95	Ubuntu 22.04.4 LTS	0.1.43	gemma:7b	60.13
Linux	AMD Ryzen 5 7600X 6-Core Processor	NVIDIA GeForce RTX 4070	30.95	Ubuntu 22.04.4 LTS	0.1.43	llava:7b	74.27
Linux	AMD Ryzen 5 7600X 6-Core Processor	NVIDIA GeForce RTX 4070	30.95	Ubuntu 22.04.4 LTS	0.1.43	llava:13b	46.26
Linux	Intel(R) Core(TM) i9-14900K	NVIDIA GeForce RTX 4070 SUPER	62.54	Ubuntu 24.04 LTS	0.1.43	mistral:7b	90.21
Linux	Intel(R) Core(TM) i9-14900K	NVIDIA GeForce RTX 4070 SUPER	62.54	Ubuntu 24.04 LTS	0.1.43	llama3:8b	80.3
Linux	Intel(R) Core(TM) i9-14900K	NVIDIA GeForce RTX 4070 SUPER	62.54	Ubuntu 24.04 LTS	0.1.43	phi3:3.8b	130.85
Linux	Intel(R) Core(TM) i9-14900K	NVIDIA GeForce RTX 4070 SUPER	62.54	Ubuntu 24.04 LTS	0.1.43	gemma:2b	170.18
Linux	Intel(R) Core(TM) i9-14900K	NVIDIA GeForce RTX 4070 SUPER	62.54	Ubuntu 24.04 LTS	0.1.43	gemma:7b	73.78
Linux	Intel(R) Core(TM) i9-14900K	NVIDIA GeForce RTX 4070 SUPER	62.54	Ubuntu 24.04 LTS	0.1.43	llava:7b	93.84
Linux	Intel(R) Core(TM) i9-14900K	NVIDIA GeForce RTX 4070 SUPER	62.54	Ubuntu 24.04 LTS	0.1.43	llava:13b	54.95
Linux	AMD Ryzen 5 5600G with Radeon Graphics	NVIDIA GeForce RTX 3060	93.67	"Fedora Linux 40 (Workstation Edition)"	0.1.48	phi3:3.8b	96.5
Linux	AMD Ryzen 5 5600G with Radeon Graphics	NVIDIA GeForce RTX 3060	93.67	"Fedora Linux 40 (Workstation Edition)"	0.1.48	gemma:2b	127.36
Linux	AMD Ryzen 5 5600G with Radeon Graphics	NVIDIA GeForce RTX 3060	93.67	"Fedora Linux 40 (Workstation Edition)"	0.1.48	gemma:7b	53.56
Linux	AMD Ryzen 5 5600G with Radeon Graphics	NVIDIA GeForce RTX 3060	93.67	"Fedora Linux 40 (Workstation Edition)"	0.1.48	llava:7b	67.78
Linux	AMD Ryzen 5 5600G with Radeon Graphics	NVIDIA GeForce RTX 3060	93.67	"Fedora Linux 40 (Workstation Edition)"	0.1.48	llava:13b	40.44
Linux	AMD Ryzen 7 7840HS w/ Radeon 780M Graphics	Phoenix1	38.55	Ubuntu 24.04 LTS	0.1.48	mistral:7b	7.31
Linux	AMD Ryzen 7 7800X3D 8-Core Processor	NVIDIA GeForce RTX 3090	60.54	Arch Linux	0.1.44	mistral:7b	127.73
Linux	AMD Ryzen 7 7800X3D 8-Core Processor	NVIDIA GeForce RTX 3090	60.54	Arch Linux	0.1.44	llama3:8b	111.97
Linux	AMD Ryzen 7 7800X3D 8-Core Processor	NVIDIA GeForce RTX 3090	60.54	Arch Linux	0.1.44	phi3:3.8b	169.49
Linux	AMD Ryzen 7 7800X3D 8-Core Processor	NVIDIA GeForce RTX 3090	60.54	Arch Linux	0.1.44	gemma:2b	207.47
Linux	AMD Ryzen 7 7800X3D 8-Core Processor	NVIDIA GeForce RTX 3090	60.54	Arch Linux	0.1.44	gemma:7b	105.09
Linux	AMD Ryzen 7 7800X3D 8-Core Processor	NVIDIA GeForce RTX 3090	60.54	Arch Linux	0.1.44	llava:7b	131.51
Linux	AMD Ryzen 7 7800X3D 8-Core Processor	NVIDIA GeForce RTX 3090	60.54	Arch Linux	0.1.44	llava:13b	82.94
Linux	Intel(R) Core(TM) i7-4720HQ CPU @ 2.60GHz	NVIDIA GeForce GTX 850M	7.67	Ubuntu 20.04.6 LTS	0.1.44	mistral:7b	3.05
Linux	Intel(R) Core(TM) i7-4720HQ CPU @ 2.60GHz	NVIDIA GeForce GTX 850M	7.67	Ubuntu 20.04.6 LTS	0.1.44	llama3:8b	3.02
Linux	Intel(R) Core(TM) i7-4720HQ CPU @ 2.60GHz	NVIDIA GeForce GTX 850M	7.67	Ubuntu 20.04.6 LTS	0.1.44	phi3:3.8b	5.22
Linux	Intel(R) Core(TM) i7-4720HQ CPU @ 2.60GHz	NVIDIA GeForce GTX 850M	7.67	Ubuntu 20.04.6 LTS	0.1.44	gemma:2b	6.99
Linux	Intel(R) Core(TM) i7-4720HQ CPU @ 2.60GHz	NVIDIA GeForce GTX 850M	7.67	Ubuntu 20.04.6 LTS	0.1.44	gemma:7b	2.67
Linux	Intel(R) Core(TM) i7-4720HQ CPU @ 2.60GHz	NVIDIA GeForce GTX 850M	7.67	Ubuntu 20.04.6 LTS	0.1.44	llava:7b	3.13
Linux	AMD EPYC 7R32	NVIDIA A10G	62.11	Ubuntu 22.04.3 LTS	0.1.41	mistral:7b	81.07
Linux	AMD EPYC 7R32	NVIDIA A10G	62.11	Ubuntu 22.04.3 LTS	0.1.41	llama3:8b	68.09
Linux	AMD EPYC 7R32	NVIDIA A10G	62.11	Ubuntu 22.04.3 LTS	0.1.41	phi3:3.8b	108.9
Linux	AMD EPYC 7R32	NVIDIA A10G	62.11	Ubuntu 22.04.3 LTS	0.1.41	gemma:2b	134.39
Linux	AMD EPYC 7R32	NVIDIA A10G	62.11	Ubuntu 22.04.3 LTS	0.1.41	gemma:7b	66.58
Linux	AMD EPYC 7R32	NVIDIA A10G	62.11	Ubuntu 22.04.3 LTS	0.1.41	llava:7b	85.63
Linux	AMD EPYC 7R32	NVIDIA A10G	62.11	Ubuntu 22.04.3 LTS	0.1.41	llava:13b	51.64
Linux	AMD Ryzen 7 7800X3D 8-Core Processor	NVIDIA GeForce RTX 3090	60.54	Arch Linux	0.1.44	mistral:7b	126.74
Linux	AMD Ryzen 7 7800X3D 8-Core Processor	NVIDIA GeForce RTX 3090	60.54	Arch Linux	0.1.44	llama3:8b	112.16
Linux	AMD Ryzen 7 7800X3D 8-Core Processor	NVIDIA GeForce RTX 3090	60.54	Arch Linux	0.1.44	phi3:3.8b	168.85
Linux	AMD Ryzen 7 7800X3D 8-Core Processor	NVIDIA GeForce RTX 3090	60.54	Arch Linux	0.1.44	gemma:2b	206.47
Linux	AMD Ryzen 7 7800X3D 8-Core Processor	NVIDIA GeForce RTX 3090	60.54	Arch Linux	0.1.44	gemma:7b	105.41
Linux	AMD Ryzen 7 7800X3D 8-Core Processor	NVIDIA GeForce RTX 3090	60.54	Arch Linux	0.1.44	llava:7b	131.21
Linux	AMD Ryzen 7 7800X3D 8-Core Processor	NVIDIA GeForce RTX 3090	60.54	Arch Linux	0.1.44	llava:13b	83.07
Linux	AMD Ryzen 7 7840HS w/ Radeon 780M Graphics	Phoenix1	38.55	Ubuntu 24.04 LTS	0.1.48	llama3:8b	9.74
Linux	AMD Ryzen 7 7840HS w/ Radeon 780M Graphics	Phoenix1	38.55	Ubuntu 24.04 LTS	0.1.48	phi3:3.8b	18.04
Linux	Intel(R) Xeon(R) CPU E3-1231 v3 @ 3.40GHz	NVIDIA GeForce GTX 1080 Ti	15.53	Ubuntu 24.04 LTS	0.1.44	mistral:7b	52.96
Linux	Intel(R) Xeon(R) CPU E3-1231 v3 @ 3.40GHz	NVIDIA GeForce GTX 1080 Ti	15.53	Ubuntu 24.04 LTS	0.1.44	llama3:8b	47.06
Linux	Intel(R) Xeon(R) CPU E3-1231 v3 @ 3.40GHz	NVIDIA GeForce GTX 1080 Ti	15.53	Ubuntu 24.04 LTS	0.1.44	phi3:3.8b	69.01
Linux	Intel(R) Xeon(R) CPU E3-1231 v3 @ 3.40GHz	NVIDIA GeForce GTX 1080 Ti	15.53	Ubuntu 24.04 LTS	0.1.44	gemma:2b	110.02
Linux	Intel(R) Xeon(R) CPU E3-1231 v3 @ 3.40GHz	NVIDIA GeForce GTX 1080 Ti	15.53	Ubuntu 24.04 LTS	0.1.44	gemma:7b	44.41
Linux	Intel(R) Xeon(R) CPU E3-1231 v3 @ 3.40GHz	NVIDIA GeForce GTX 1080 Ti	15.53	Ubuntu 24.04 LTS	0.1.44	llava:7b	59.56
Linux	Intel(R) Xeon(R) CPU E3-1231 v3 @ 3.40GHz	NVIDIA GeForce GTX 1080 Ti	15.53	Ubuntu 24.04 LTS	0.1.44	llava:13b	35.47
Linux	12th Gen Intel(R) Core(TM) i7-12700K	AlderLake-S GT1	15.39	Ubuntu 22.04.4 LTS	0.1.44	mistral:7b	71.1
Linux	12th Gen Intel(R) Core(TM) i7-12700K	AlderLake-S GT1	15.39	Ubuntu 22.04.4 LTS	0.1.44	llama3:8b	66.89
Linux	12th Gen Intel(R) Core(TM) i7-12700K	AlderLake-S GT1	15.39	Ubuntu 22.04.4 LTS	0.1.44	phi3:3.8b	82.21
Linux	12th Gen Intel(R) Core(TM) i7-12700K	AlderLake-S GT1	15.39	Ubuntu 22.04.4 LTS	0.1.44	gemma:2b	145.2
Linux	12th Gen Intel(R) Core(TM) i7-12700K	AlderLake-S GT1	15.39	Ubuntu 22.04.4 LTS	0.1.44	gemma:7b	59.31
Linux	12th Gen Intel(R) Core(TM) i7-12700K	AlderLake-S GT1	15.39	Ubuntu 22.04.4 LTS	0.1.44	llava:7b	74.34
Linux	12th Gen Intel(R) Core(TM) i7-12700K	AlderLake-S GT1	15.39	Ubuntu 22.04.4 LTS	0.1.44	llava:13b	45.05
Linux	Intel(R) Xeon(R) CPU E3-1231 v3 @ 3.40GHz	NVIDIA GeForce GTX 1080 Ti	15.53	Ubuntu 24.04 LTS	0.1.44	mistral:7b	57.41
Linux	Intel(R) Xeon(R) CPU E3-1231 v3 @ 3.40GHz	NVIDIA GeForce GTX 1080 Ti	15.53	Ubuntu 24.04 LTS	0.1.44	llama3:8b	48.74
Linux	Intel(R) Xeon(R) CPU E3-1231 v3 @ 3.40GHz	NVIDIA GeForce GTX 1080 Ti	15.53	Ubuntu 24.04 LTS	0.1.44	phi3:3.8b	73.39
Linux	Intel(R) Xeon(R) CPU E3-1231 v3 @ 3.40GHz	NVIDIA GeForce GTX 1080 Ti	15.53	Ubuntu 24.04 LTS	0.1.44	gemma:2b	114.17
Linux	Intel(R) Xeon(R) CPU E3-1231 v3 @ 3.40GHz	NVIDIA GeForce GTX 1080 Ti	15.53	Ubuntu 24.04 LTS	0.1.44	gemma:7b	46.96
Linux	Intel(R) Xeon(R) CPU E3-1231 v3 @ 3.40GHz	NVIDIA GeForce GTX 1080 Ti	15.53	Ubuntu 24.04 LTS	0.1.44	llava:7b	61.15
Linux	Intel(R) Xeon(R) CPU E3-1231 v3 @ 3.40GHz	NVIDIA GeForce GTX 1080 Ti	15.53	Ubuntu 24.04 LTS	0.1.44	llava:13b	35.94
Linux	Intel(R) Xeon(R) Gold 6342 CPU @ 2.80GHz	NVIDIA A100-SXM4-80GB	88.37	Ubuntu 22.04.4 LTS	0.1.44	mistral:7b	129.26
Linux	Intel(R) Xeon(R) Gold 6342 CPU @ 2.80GHz	NVIDIA A100-SXM4-80GB	88.37	Ubuntu 22.04.4 LTS	0.1.44	llama3:8b	102.97
Linux	Intel(R) Xeon(R) Gold 6342 CPU @ 2.80GHz	NVIDIA A100-SXM4-80GB	88.37	Ubuntu 22.04.4 LTS	0.1.44	phi3:3.8b	137.37
Linux	Intel(R) Xeon(R) Gold 6342 CPU @ 2.80GHz	NVIDIA A100-SXM4-80GB	88.37	Ubuntu 22.04.4 LTS	0.1.44	gemma:2b	158.12
Linux	Intel(R) Xeon(R) Gold 6342 CPU @ 2.80GHz	NVIDIA A100-SXM4-80GB	88.37	Ubuntu 22.04.4 LTS	0.1.44	gemma:7b	100.61
Linux	Intel(R) Xeon(R) Gold 6342 CPU @ 2.80GHz	NVIDIA A100-SXM4-80GB	88.37	Ubuntu 22.04.4 LTS	0.1.44	llava:7b	130.37
Linux	Intel(R) Xeon(R) Gold 6342 CPU @ 2.80GHz	NVIDIA A100-SXM4-80GB	88.37	Ubuntu 22.04.4 LTS	0.1.44	llava:13b	90.83
Linux	Intel(R) Core(TM) i5-8250U CPU @ 1.60GHz	NVIDIA GeForce MX130	31.08	Fedora release 40 (Forty)	0.1.42	mistral:7b	1.97
Linux	Intel(R) Core(TM) i5-8250U CPU @ 1.60GHz	NVIDIA GeForce MX130	31.08	Fedora release 40 (Forty)	0.1.42	llama3:8b	2.22
Linux	Intel(R) Core(TM) i5-8250U CPU @ 1.60GHz	NVIDIA GeForce MX130	31.08	Fedora release 40 (Forty)	0.1.42	phi3:3.8b	3.87
Linux	Intel(R) Core(TM) i5-8250U CPU @ 1.60GHz	NVIDIA GeForce MX130	31.08	Fedora release 40 (Forty)	0.1.42	gemma:2b	5.27
Linux	Intel(R) Core(TM) i5-8250U CPU @ 1.60GHz	NVIDIA GeForce MX130	31.08	Fedora release 40 (Forty)	0.1.42	gemma:7b	2.31
Linux	Intel(R) Core(TM) i5-8250U CPU @ 1.60GHz	NVIDIA GeForce MX130	31.08	Fedora release 40 (Forty)	0.1.42	llava:7b	2.45
Linux	Intel(R) Core(TM) i5-8250U CPU @ 1.60GHz	NVIDIA GeForce MX130	31.08	Fedora release 40 (Forty)	0.1.42	llava:13b	1.67
Linux	AMD Ryzen 7 7840U w/ Radeon 780M Graphics	no_gpu	30.66	"Arch Linux"	0.1.41	mistral:7b	4.98
Linux	AMD Ryzen 7 7840U w/ Radeon 780M Graphics	no_gpu	30.66	"Arch Linux"	0.1.41	llama3:8b	5.16
Linux	AMD Ryzen 7 7840U w/ Radeon 780M Graphics	no_gpu	30.66	"Arch Linux"	0.1.41	phi3:3.8b	9.58
Linux	AMD Ryzen 7 7840U w/ Radeon 780M Graphics	no_gpu	30.66	"Arch Linux"	0.1.41	gemma:2b	13.71
Linux	AMD Ryzen 7 7840U w/ Radeon 780M Graphics	no_gpu	30.66	"Arch Linux"	0.1.41	gemma:7b	4.88
Linux	AMD Ryzen 7 7840U w/ Radeon 780M Graphics	no_gpu	30.66	"Arch Linux"	0.1.41	llava:7b	6.21
Linux	AMD Ryzen 7 7840U w/ Radeon 780M Graphics	no_gpu	30.66	"Arch Linux"	0.1.41	llava:13b	3.45
Linux	AMD Ryzen 7 7840U w/ Radeon 780M Graphics	no_gpu	30.66	"Arch Linux"	0.1.44	mistral:7b	6.07
Linux	AMD Ryzen 7 7840U w/ Radeon 780M Graphics	no_gpu	30.66	"Arch Linux"	0.1.44	llama3:8b	5.67
Linux	AMD Ryzen 7 7840U w/ Radeon 780M Graphics	no_gpu	30.66	"Arch Linux"	0.1.44	phi3:3.8b	9.66
Linux	AMD Ryzen 7 7840U w/ Radeon 780M Graphics	no_gpu	30.66	"Arch Linux"	0.1.44	gemma:2b	13.62
Linux	AMD Ryzen 7 7840U w/ Radeon 780M Graphics	no_gpu	30.66	"Arch Linux"	0.1.44	gemma:7b	4.89
Linux	AMD Ryzen 7 7840U w/ Radeon 780M Graphics	no_gpu	30.66	"Arch Linux"	0.1.44	llava:7b	6.26
Linux	AMD Ryzen 7 7840U w/ Radeon 780M Graphics	no_gpu	30.66	"Arch Linux"	0.1.44	llava:13b	3.45
Linux	12th Gen Intel(R) Core(TM) i7-12700KF	NVIDIA GeForce RTX 4070	27.42	Ubuntu 20.04.5 LTS	0.1.39	mistral:7b	89.05
Linux	12th Gen Intel(R) Core(TM) i7-12700KF	NVIDIA GeForce RTX 4070	27.42	Ubuntu 20.04.5 LTS	0.1.39	llama3:8b	78.49
Linux	12th Gen Intel(R) Core(TM) i7-12700KF	NVIDIA GeForce RTX 4070	27.42	Ubuntu 20.04.5 LTS	0.1.39	phi3:3.8b	125.24
Linux	12th Gen Intel(R) Core(TM) i7-12700KF	NVIDIA GeForce RTX 4070	27.42	Ubuntu 20.04.5 LTS	0.1.39	gemma:2b	159.15
Linux	12th Gen Intel(R) Core(TM) i7-12700KF	NVIDIA GeForce RTX 4070	27.42	Ubuntu 20.04.5 LTS	0.1.39	gemma:7b	70.84
Linux	12th Gen Intel(R) Core(TM) i7-12700KF	NVIDIA GeForce RTX 4070	27.42	Ubuntu 20.04.5 LTS	0.1.39	llava:7b	88.13
Linux	12th Gen Intel(R) Core(TM) i7-12700KF	NVIDIA GeForce RTX 4070	27.42	Ubuntu 20.04.5 LTS	0.1.39	llava:13b	54.24
Linux	Intel(R) Xeon(R) Platinum 8369HC B3 CPU	NVIDIA GeForce RTX 4070	27.42	Ubuntu 20.04.5 LTS	0.1.39	mistral:7b	87.22
Linux	Intel(R) Xeon(R) Platinum 8369HC B3 CPU	NVIDIA GeForce RTX 4070	27.42	Ubuntu 20.04.5 LTS	0.1.39	llama3:8b	77
Linux	Intel(R) Xeon(R) Platinum 8369HC B3 CPU	NVIDIA GeForce RTX 4070	27.42	Ubuntu 20.04.5 LTS	0.1.39	phi3:3.8b	124.4
Linux	Intel(R) Xeon(R) Platinum 8369HC B3 CPU	NVIDIA GeForce RTX 4070	27.42	Ubuntu 20.04.5 LTS	0.1.39	gemma:2b	159.55
Linux	Intel(R) Xeon(R) Platinum 8369HC B3 CPU	NVIDIA GeForce RTX 4070	27.42	Ubuntu 20.04.5 LTS	0.1.39	gemma:7b	70.72
Linux	Intel(R) Xeon(R) Platinum 8369HC B3 CPU	NVIDIA GeForce RTX 4070	27.42	Ubuntu 20.04.5 LTS	0.1.39	llava:7b	90.2
Linux	Intel(R) Xeon(R) Platinum 8369HC B3 CPU	NVIDIA GeForce RTX 4070	27.42	Ubuntu 20.04.5 LTS	0.1.39	llava:13b	54.22
Linux	Intel(R) Xeon(R) Platinum 8369HC B3 CPU	NVIDIA GeForce RTX 4070	27.42	Ubuntu 20.04.5 LTS	0.1.39	mistral:7b	86.99
Linux	Intel(R) Xeon(R) Platinum 8369HC B3 CPU	NVIDIA GeForce RTX 4070	27.42	Ubuntu 20.04.5 LTS	0.1.39	llama3:8b	77.01
Linux	Intel(R) Xeon(R) Platinum 8369HC B3 CPU	NVIDIA GeForce RTX 4070	27.42	Ubuntu 20.04.5 LTS	0.1.39	phi3:3.8b	124.25
Linux	Intel(R) Xeon(R) Platinum 8369HC B3 CPU	NVIDIA GeForce RTX 4070	27.42	Ubuntu 20.04.5 LTS	0.1.39	gemma:2b	159.54
Linux	Intel(R) Xeon(R) Platinum 8369HC B3 CPU	NVIDIA GeForce RTX 4070	27.42	Ubuntu 20.04.5 LTS	0.1.39	gemma:7b	70.88
Linux	Intel(R) Xeon(R) Platinum 8369HC B3 CPU	NVIDIA GeForce RTX 4070	27.42	Ubuntu 20.04.5 LTS	0.1.39	llava:7b	91.01
Linux	Intel(R) Xeon(R) Platinum 8369HC B3 CPU	NVIDIA GeForce RTX 4070	27.42	Ubuntu 20.04.5 LTS	0.1.39	llava:13b	53.91
Linux	12th Gen Intel(R) Core(TM) i7-12700KF	NVIDIA GeForce RTX 4070	27.42	Ubuntu 20.04.5 LTS	0.1.39	mistral:7b	87.47
Linux	12th Gen Intel(R) Core(TM) i7-12700KF	NVIDIA GeForce RTX 4070	27.42	Ubuntu 20.04.5 LTS	0.1.39	llama3:8b	77.2
Linux	12th Gen Intel(R) Core(TM) i7-12700KF	NVIDIA GeForce RTX 4070	27.42	Ubuntu 20.04.5 LTS	0.1.39	phi3:3.8b	124.3
Linux	12th Gen Intel(R) Core(TM) i7-12700KF	NVIDIA GeForce RTX 4070	27.42	Ubuntu 20.04.5 LTS	0.1.39	gemma:2b	158.55
Linux	12th Gen Intel(R) Core(TM) i7-12700KF	NVIDIA GeForce RTX 4070	27.42	Ubuntu 20.04.5 LTS	0.1.39	gemma:7b	70.67
Linux	12th Gen Intel(R) Core(TM) i7-12700KF	NVIDIA GeForce RTX 4070	27.42	Ubuntu 20.04.5 LTS	0.1.39	llava:7b	89.85
Linux	12th Gen Intel(R) Core(TM) i7-12700KF	NVIDIA GeForce RTX 4070	27.42	Ubuntu 20.04.5 LTS	0.1.39	llava:13b	52.92
Linux	Intel(R) Xeon(R) Platinum 8369HC B3 CPU	NVIDIA GeForce RTX 4070	27.42	Ubuntu 20.04.5 LTS	0.1.39	mistral:7b	87.06
Linux	Intel(R) Xeon(R) Platinum 8369HC B3 CPU	NVIDIA GeForce RTX 4070	27.42	Ubuntu 20.04.5 LTS	0.1.39	llama3:8b	77.32
Linux	Intel(R) Xeon(R) Platinum 8369HC B3 CPU	NVIDIA GeForce RTX 4070	27.42	Ubuntu 20.04.5 LTS	0.1.39	phi3:3.8b	120.83
Linux	Intel(R) Xeon(R) Platinum 8369HC B3 CPU	NVIDIA GeForce RTX 4070	27.42	Ubuntu 20.04.5 LTS	0.1.39	gemma:2b	159.21
Linux	Intel(R) Xeon(R) Platinum 8369HC B3 CPU	NVIDIA GeForce RTX 4070	27.42	Ubuntu 20.04.5 LTS	0.1.39	gemma:7b	69.85
Linux	Intel(R) Xeon(R) Platinum 8369HC B3 CPU	NVIDIA GeForce RTX 4070	27.42	Ubuntu 20.04.5 LTS	0.1.39	llava:7b	90.52
Linux	Intel(R) Xeon(R) Platinum 8369HC B3 CPU	NVIDIA GeForce RTX 4070	27.42	Ubuntu 20.04.5 LTS	0.1.39	llava:13b	54.38
Linux	12th Gen Intel(R) Core(TM) i7-12700KF	NVIDIA GeForce RTX 4070	27.42	Ubuntu 20.04.5 LTS	0.1.39	mistral:7b	87.19
Linux	12th Gen Intel(R) Core(TM) i7-12700KF	NVIDIA GeForce RTX 4070	27.42	Ubuntu 20.04.5 LTS	0.1.39	llama3:8b	77.08
Linux	12th Gen Intel(R) Core(TM) i7-12700KF	NVIDIA GeForce RTX 4070	27.42	Ubuntu 20.04.5 LTS	0.1.39	phi3:3.8b	123.6
Linux	12th Gen Intel(R) Core(TM) i7-12700KF	NVIDIA GeForce RTX 4070	27.42	Ubuntu 20.04.5 LTS	0.1.39	gemma:2b	159.62
Linux	12th Gen Intel(R) Core(TM) i7-12700KF	NVIDIA GeForce RTX 4070	27.42	Ubuntu 20.04.5 LTS	0.1.39	gemma:7b	69.6
Linux	12th Gen Intel(R) Core(TM) i7-12700KF	NVIDIA GeForce RTX 4070	27.42	Ubuntu 20.04.5 LTS	0.1.39	llava:7b	90.64
Linux	12th Gen Intel(R) Core(TM) i7-12700KF	NVIDIA GeForce RTX 4070	27.42	Ubuntu 20.04.5 LTS	0.1.39	llava:13b	54.25
Linux	AMD Ryzen Z1 Extreme	Basic Render Driver	5.64	Ubuntu 24.04 LTS	0.1.44	phi3:3.8b	8.63
Linux	AMD Ryzen Z1 Extreme	Basic Render Driver	5.64	Ubuntu 24.04 LTS	0.1.44	phi:2.7b	9.53
Linux	AMD Ryzen Z1 Extreme	Basic Render Driver	5.64	Ubuntu 24.04 LTS	0.1.44	gemma:2b	12.36
Linux	AMD Ryzen 5 3600 6-Core Processor	NVIDIA GeForce RTX 2070 SUPER	23.47	Ubuntu 22.04.4 LTS	0.1.44	mistral:7b	38.6
Linux	AMD Ryzen 5 3600 6-Core Processor	NVIDIA GeForce RTX 2070 SUPER	23.47	Ubuntu 22.04.4 LTS	0.1.44	llama3:8b	35.51
Linux	AMD Ryzen 5 3600 6-Core Processor	NVIDIA GeForce RTX 2070 SUPER	23.47	Ubuntu 22.04.4 LTS	0.1.44	phi3:3.8b	54.23
Linux	AMD Ryzen 5 3600 6-Core Processor	NVIDIA GeForce RTX 2070 SUPER	23.47	Ubuntu 22.04.4 LTS	0.1.44	gemma:2b	73.98
Linux	AMD Ryzen 5 3600 6-Core Processor	NVIDIA GeForce RTX 2070 SUPER	23.47	Ubuntu 22.04.4 LTS	0.1.44	gemma:7b	38.34
Linux	AMD Ryzen 5 3600 6-Core Processor	NVIDIA GeForce RTX 2070 SUPER	23.47	Ubuntu 22.04.4 LTS	0.1.44	llava:7b	40.87
Linux	AMD Ryzen 5 3600 6-Core Processor	NVIDIA GeForce RTX 2070 SUPER	23.47	Ubuntu 22.04.4 LTS	0.1.44	llava:13b	6.39
Linux	AMD Ryzen 9 3900X 12-Core Processor	NVIDIA GeForce RTX 2060	125.70	Debian GNU/Linux 12 (bookworm)	0.0.0	mistral:7b	131.99
Linux	AMD Ryzen 9 3900X 12-Core Processor	NVIDIA GeForce RTX 2060	125.70	Debian GNU/Linux 12 (bookworm)	0.0.0	llama3:8b	109.16
Linux	AMD Ryzen 9 3900X 12-Core Processor	NVIDIA GeForce RTX 2060	125.70	Debian GNU/Linux 12 (bookworm)	0.0.0	phi3:3.8b	175.1
Linux	AMD Ryzen 9 3900X 12-Core Processor	NVIDIA GeForce RTX 2060	125.70	Debian GNU/Linux 12 (bookworm)	0.0.0	gemma:2b	203.84
Linux	AMD Ryzen 9 3900X 12-Core Processor	NVIDIA GeForce RTX 2060	125.70	Debian GNU/Linux 12 (bookworm)	0.0.0	gemma:7b	105.52
Linux	AMD Ryzen 9 3900X 12-Core Processor	NVIDIA GeForce RTX 2060	125.70	Debian GNU/Linux 12 (bookworm)	0.0.0	llava:7b	136.57
Linux	AMD Ryzen 9 3900X 12-Core Processor	NVIDIA GeForce RTX 2060	125.70	Debian GNU/Linux 12 (bookworm)	0.0.0	llava:13b	86.94
Linux	AMD Ryzen 9 3900X 12-Core Processor	NVIDIA GeForce RTX 2060	125.70	Debian GNU/Linux 12 (bookworm)	0.0.0	mistral:7b	131.76
Linux	AMD Ryzen 9 3900X 12-Core Processor	NVIDIA GeForce RTX 2060	125.70	Debian GNU/Linux 12 (bookworm)	0.0.0	llama3:8b	108.95
Linux	AMD Ryzen 9 3900X 12-Core Processor	NVIDIA GeForce RTX 2060	125.70	Debian GNU/Linux 12 (bookworm)	0.0.0	phi3:3.8b	174.66
Linux	AMD Ryzen 9 3900X 12-Core Processor	NVIDIA GeForce RTX 2060	125.70	Debian GNU/Linux 12 (bookworm)	0.0.0	gemma:2b	203.53
Linux	AMD Ryzen 9 3900X 12-Core Processor	NVIDIA GeForce RTX 2060	125.70	Debian GNU/Linux 12 (bookworm)	0.0.0	gemma:7b	105.97
Linux	AMD Ryzen 9 3900X 12-Core Processor	NVIDIA GeForce RTX 2060	125.70	Debian GNU/Linux 12 (bookworm)	0.0.0	llava:7b	138.48
Linux	AMD Ryzen 9 3900X 12-Core Processor	NVIDIA GeForce RTX 2060	125.70	Debian GNU/Linux 12 (bookworm)	0.0.0	llava:13b	86.94
Linux	AMD Ryzen 9 7950X 16-Core Processor	no_gpu	187.90	Debian GNU/Linux 12 (bookworm)	0.1.44	mistral:7b	9.8
Linux	AMD Ryzen 9 7950X 16-Core Processor	no_gpu	187.90	Debian GNU/Linux 12 (bookworm)	0.1.44	llama3:8b	9.05
Linux	AMD Ryzen 9 7950X 16-Core Processor	no_gpu	187.90	Debian GNU/Linux 12 (bookworm)	0.1.44	phi3:3.8b	14.86
Linux	AMD Ryzen 9 7950X 16-Core Processor	no_gpu	187.90	Debian GNU/Linux 12 (bookworm)	0.1.44	gemma:2b	22.43
Linux	AMD Ryzen 9 7950X 16-Core Processor	no_gpu	187.90	Debian GNU/Linux 12 (bookworm)	0.1.44	gemma:7b	7.69
Linux	AMD Ryzen 9 7950X 16-Core Processor	no_gpu	187.90	Debian GNU/Linux 12 (bookworm)	0.1.44	llava:7b	10
Linux	AMD Ryzen 9 7950X 16-Core Processor	no_gpu	187.90	Debian GNU/Linux 12 (bookworm)	0.1.44	llava:13b	5.67
Linux	AMD Ryzen 9 5950X 16-Core Processor	no_gpu	125.72	Debian GNU/Linux 12 (bookworm)	0.1.44	mistral:7b	8.38
Linux	AMD Ryzen 9 5950X 16-Core Processor	no_gpu	125.72	Debian GNU/Linux 12 (bookworm)	0.1.44	llama3:8b	7.77
Linux	AMD Ryzen 9 5950X 16-Core Processor	no_gpu	125.72	Debian GNU/Linux 12 (bookworm)	0.1.44	phi3:3.8b	12.95
Linux	AMD Ryzen 9 5950X 16-Core Processor	no_gpu	125.72	Debian GNU/Linux 12 (bookworm)	0.1.44	gemma:2b	19.66
Linux	AMD Ryzen 9 5950X 16-Core Processor	no_gpu	125.72	Debian GNU/Linux 12 (bookworm)	0.1.44	gemma:7b	6.71
Linux	AMD Ryzen 9 5950X 16-Core Processor	no_gpu	125.72	Debian GNU/Linux 12 (bookworm)	0.1.44	llava:7b	8.58
Linux	AMD Ryzen 9 5950X 16-Core Processor	no_gpu	125.72	Debian GNU/Linux 12 (bookworm)	0.1.44	llava:13b	4.72
Linux	AMD Ryzen 9 5950X 16-Core Processor	no_gpu	125.72	Debian GNU/Linux 12 (bookworm)	0.1.44	mistral:7b	7.98
Linux	AMD Ryzen 9 5950X 16-Core Processor	no_gpu	125.72	Debian GNU/Linux 12 (bookworm)	0.1.44	llama3:8b	7.4
Linux	AMD Ryzen 9 5950X 16-Core Processor	no_gpu	125.72	Debian GNU/Linux 12 (bookworm)	0.1.44	phi3:3.8b	12.52
Linux	AMD Ryzen 9 5950X 16-Core Processor	no_gpu	125.72	Debian GNU/Linux 12 (bookworm)	0.1.44	gemma:2b	18.61
Linux	AMD Ryzen 9 5950X 16-Core Processor	no_gpu	125.72	Debian GNU/Linux 12 (bookworm)	0.1.44	gemma:7b	6.34
Linux	AMD Ryzen 9 5950X 16-Core Processor	no_gpu	125.72	Debian GNU/Linux 12 (bookworm)	0.1.44	llava:7b	8.19
Linux	AMD Ryzen 9 5950X 16-Core Processor	no_gpu	125.72	Debian GNU/Linux 12 (bookworm)	0.1.44	llava:13b	4.53
Linux	AMD Ryzen Z1 Extreme	Basic Render Driver	11.39	Ubuntu 24.04 LTS	0.1.44	mistral:7b	6.82
Linux	AMD Ryzen Z1 Extreme	Basic Render Driver	11.39	Ubuntu 24.04 LTS	0.1.44	llama3:8b	6.32
Linux	AMD Ryzen Z1 Extreme	Basic Render Driver	11.39	Ubuntu 24.04 LTS	0.1.44	phi3:3.8b	14.08
Linux	AMD Ryzen Z1 Extreme	Basic Render Driver	11.39	Ubuntu 24.04 LTS	0.1.44	gemma:2b	15.16
Linux	AMD Ryzen Z1 Extreme	Basic Render Driver	11.39	Ubuntu 24.04 LTS	0.1.44	gemma:7b	5.24
Linux	AMD Ryzen Z1 Extreme	Basic Render Driver	11.39	Ubuntu 24.04 LTS	0.1.44	llava:7b	6.66
Linux	unknown	NVIDIA GH200 480GB	572.05	Rocky Linux 9.4 (Blue Onyx)	0.1.42	mistral:7b	149.38
Linux	unknown	NVIDIA GH200 480GB	572.05	Rocky Linux 9.4 (Blue Onyx)	0.1.42	llama3:8b	135.18
Linux	unknown	NVIDIA GH200 480GB	572.05	Rocky Linux 9.4 (Blue Onyx)	0.1.42	phi3:3.8b	159.13
Linux	unknown	NVIDIA GH200 480GB	572.05	Rocky Linux 9.4 (Blue Onyx)	0.1.42	gemma:2b	240.56
Linux	unknown	NVIDIA GH200 480GB	572.05	Rocky Linux 9.4 (Blue Onyx)	0.1.42	gemma:7b	143.88
Linux	unknown	NVIDIA GH200 480GB	572.05	Rocky Linux 9.4 (Blue Onyx)	0.1.42	llava:7b	175.63
Linux	unknown	NVIDIA GH200 480GB	572.05	Rocky Linux 9.4 (Blue Onyx)	0.1.42	llava:13b	121.84
Linux	Intel(R) Core(TM) i7-7700 CPU @ 3.60GHz	NVIDIA GeForce RTX 2060	47.05	Ubuntu 24.04 LTS	0.1.41	mistral:7b	57.22
Linux	Intel(R) Core(TM) i7-7700 CPU @ 3.60GHz	NVIDIA GeForce RTX 2060	47.05	Ubuntu 24.04 LTS	0.1.41	llama3:8b	48.35
Linux	Intel(R) Core(TM) i7-7700 CPU @ 3.60GHz	NVIDIA GeForce RTX 2060	47.05	Ubuntu 24.04 LTS	0.1.41	phi3:3.8b	76.4
Linux	Intel(R) Core(TM) i7-7700 CPU @ 3.60GHz	NVIDIA GeForce RTX 2060	47.05	Ubuntu 24.04 LTS	0.1.41	gemma:2b	98.78
Linux	Intel(R) Core(TM) i7-7700 CPU @ 3.60GHz	NVIDIA GeForce RTX 2060	47.05	Ubuntu 24.04 LTS	0.1.41	gemma:7b	11.66
Linux	Intel(R) Core(TM) i7-7700 CPU @ 3.60GHz	NVIDIA GeForce RTX 2060	47.05	Ubuntu 24.04 LTS	0.1.41	llava:7b	59.44
Linux	Intel(R) Core(TM) i7-7700 CPU @ 3.60GHz	NVIDIA GeForce RTX 2060	47.05	Ubuntu 24.04 LTS	0.1.41	llava:13b	5.67
Linux	AMD Ryzen 5 7640HS w/ Radeon 760M Graphics	NVIDIA GeForce RTX 4060 Laptop GPU	30.54	Ubuntu 24.04 LTS	0.1.44	mistral:7b	49.99
Linux	AMD Ryzen 5 7640HS w/ Radeon 760M Graphics	NVIDIA GeForce RTX 4060 Laptop GPU	30.54	Ubuntu 24.04 LTS	0.1.44	llama3:8b	45.43
Linux	AMD Ryzen 5 7640HS w/ Radeon 760M Graphics	NVIDIA GeForce RTX 4060 Laptop GPU	30.54	Ubuntu 24.04 LTS	0.1.44	phi3:3.8b	73.59
Linux	AMD Ryzen 5 7640HS w/ Radeon 760M Graphics	NVIDIA GeForce RTX 4060 Laptop GPU	30.54	Ubuntu 24.04 LTS	0.1.44	gemma:2b	104.89
Linux	AMD Ryzen 5 7640HS w/ Radeon 760M Graphics	NVIDIA GeForce RTX 4060 Laptop GPU	30.54	Ubuntu 24.04 LTS	0.1.44	gemma:7b	40.55
Linux	AMD Ryzen 5 7640HS w/ Radeon 760M Graphics	NVIDIA GeForce RTX 4060 Laptop GPU	30.54	Ubuntu 24.04 LTS	0.1.44	llava:7b	51.62
Linux	AMD Ryzen 5 7640HS w/ Radeon 760M Graphics	NVIDIA GeForce RTX 4060 Laptop GPU	30.54	Ubuntu 24.04 LTS	0.1.44	llava:13b	11.63
Linux	Intel(R) Xeon(R) Silver 4214 CPU @ 2.20GHz	NVIDIA GeForce RTX 2080 Ti	94.28	Pop!_OS 22.04 LTS	0.1.44	mistral:7b	32.39
Linux	Intel(R) Xeon(R) Silver 4214 CPU @ 2.20GHz	NVIDIA GeForce RTX 2080 Ti	94.28	Pop!_OS 22.04 LTS	0.1.44	llama3:8b	33.62
Linux	Intel(R) Xeon(R) Silver 4214 CPU @ 2.20GHz	NVIDIA GeForce RTX 2080 Ti	94.28	Pop!_OS 22.04 LTS	0.1.44	phi3:3.8b	50
Linux	Intel(R) Xeon(R) Silver 4214 CPU @ 2.20GHz	NVIDIA GeForce RTX 2080 Ti	94.28	Pop!_OS 22.04 LTS	0.1.44	gemma:2b	56.44
Linux	Intel(R) Xeon(R) Silver 4214 CPU @ 2.20GHz	NVIDIA GeForce RTX 2080 Ti	94.28	Pop!_OS 22.04 LTS	0.1.44	gemma:7b	34.02
Linux	Intel(R) Xeon(R) Silver 4214 CPU @ 2.20GHz	NVIDIA GeForce RTX 2080 Ti	94.28	Pop!_OS 22.04 LTS	0.1.44	llava:7b	39.37
Linux	Intel(R) Xeon(R) Silver 4214 CPU @ 2.20GHz	NVIDIA GeForce RTX 2080 Ti	94.28	Pop!_OS 22.04 LTS	0.1.44	llava:13b	27.94
Linux	AMD Custom APU 0932	no_gpu	14.47	"Ubuntu 22.04.4 LTS"	0.1.44	mistral:7b	5.05
Linux	AMD Custom APU 0932	no_gpu	14.47	"Ubuntu 22.04.4 LTS"	0.1.44	llama3:8b	4.59
Linux	AMD Custom APU 0932	no_gpu	14.47	"Ubuntu 22.04.4 LTS"	0.1.44	phi3:3.8b	8.23
Linux	AMD Custom APU 0932	no_gpu	14.47	"Ubuntu 22.04.4 LTS"	0.1.44	gemma:2b	10.99
Linux	AMD Custom APU 0932	no_gpu	14.47	"Ubuntu 22.04.4 LTS"	0.1.44	gemma:7b	3.92
Linux	AMD Custom APU 0932	no_gpu	14.47	"Ubuntu 22.04.4 LTS"	0.1.44	llava:7b	5.28
Linux	Intel(R) Core(TM) i5-8400 CPU @ 2.80GHz	NVIDIA GeForce GTX 1080 Ti	15.49	Ubuntu 22.04.3 LTS	0.1.44	mistral:7b	50.51
Linux	Intel(R) Core(TM) i5-8400 CPU @ 2.80GHz	NVIDIA GeForce GTX 1080 Ti	15.49	Ubuntu 22.04.3 LTS	0.1.44	llama3:8b	43.27
Linux	Intel(R) Core(TM) i5-8400 CPU @ 2.80GHz	NVIDIA GeForce GTX 1080 Ti	15.49	Ubuntu 22.04.3 LTS	0.1.44	phi3:3.8b	59.69
Linux	Intel(R) Core(TM) i5-8400 CPU @ 2.80GHz	NVIDIA GeForce GTX 1080 Ti	15.49	Ubuntu 22.04.3 LTS	0.1.44	gemma:2b	85.17
Linux	Intel(R) Core(TM) i5-8400 CPU @ 2.80GHz	NVIDIA GeForce GTX 1080 Ti	15.49	Ubuntu 22.04.3 LTS	0.1.44	gemma:7b	41.19
Linux	Intel(R) Core(TM) i5-8400 CPU @ 2.80GHz	NVIDIA GeForce GTX 1080 Ti	15.49	Ubuntu 22.04.3 LTS	0.1.44	llava:7b	53.3
Linux	Intel(R) Core(TM) i5-8400 CPU @ 2.80GHz	NVIDIA GeForce GTX 1080 Ti	15.49	Ubuntu 22.04.3 LTS	0.1.44	llava:13b	34.11
Linux	Intel(R) Xeon(R) CPU E5-2687W v4 @ 3.00GHz	NVIDIA RTX A4000	125.71	Ubuntu 22.04.4 LTS	0.1.38	llama3:8b	56.32
Linux	Intel(R) Xeon(R) CPU E5-2687W v4 @ 3.00GHz	NVIDIA RTX A4000	125.71	Ubuntu 22.04.4 LTS	0.1.38	gemma:2b	118.1
Linux	Cortex-A76	no_gpu	7.86	Debian GNU/Linux 12 (bookworm)	0.1.44	mistral:7b	1.98
Linux	Cortex-A76	no_gpu	7.86	Debian GNU/Linux 12 (bookworm)	0.1.44	llama3:8b	1.84
Linux	Cortex-A76	no_gpu	7.86	Debian GNU/Linux 12 (bookworm)	0.1.44	phi3:3.8b	3.21
Linux	Cortex-A76	no_gpu	7.86	Debian GNU/Linux 12 (bookworm)	0.1.44	gemma:2b	4.94
Linux	Cortex-A76	no_gpu	7.86	Debian GNU/Linux 12 (bookworm)	0.1.44	gemma:7b	1.59
Linux	Cortex-A76	no_gpu	7.86	Debian GNU/Linux 12 (bookworm)	0.1.44	llava:7b	2.06
Linux	Intel(R) Xeon(R) Gold 6148 CPU @ 2.40GHz	Tesla V100-SXM2-16GB	754.12	Ubuntu 22.04.4 LTS	0.1.44	mistral:7b	106.26
Linux	Intel(R) Xeon(R) Gold 6148 CPU @ 2.40GHz	Tesla V100-SXM2-16GB	754.12	Ubuntu 22.04.4 LTS	0.1.44	llama3:8b	85.75
Linux	11th Gen Intel(R) Core(TM) i5-1135G7 @ 2.40GHz	NVIDIA GeForce RTX 3090	31.34	Ubuntu 22.04.4 LTS	0.1.42	mistral:7b	118.37
Linux	11th Gen Intel(R) Core(TM) i5-1135G7 @ 2.40GHz	NVIDIA GeForce RTX 3090	31.34	Ubuntu 22.04.4 LTS	0.1.42	llama3:8b	98.5
Linux	11th Gen Intel(R) Core(TM) i5-1135G7 @ 2.40GHz	NVIDIA GeForce RTX 3090	31.34	Ubuntu 22.04.4 LTS	0.1.42	phi3:3.8b	152.45
Linux	11th Gen Intel(R) Core(TM) i5-1135G7 @ 2.40GHz	NVIDIA GeForce RTX 3090	31.34	Ubuntu 22.04.4 LTS	0.1.42	gemma:2b	167.44
Linux	11th Gen Intel(R) Core(TM) i5-1135G7 @ 2.40GHz	NVIDIA GeForce RTX 3090	31.34	Ubuntu 22.04.4 LTS	0.1.42	gemma:7b	93.37
Linux	11th Gen Intel(R) Core(TM) i5-1135G7 @ 2.40GHz	NVIDIA GeForce RTX 3090	31.34	Ubuntu 22.04.4 LTS	0.1.42	llava:7b	120.56
Linux	11th Gen Intel(R) Core(TM) i5-1135G7 @ 2.40GHz	NVIDIA GeForce RTX 3090	31.34	Ubuntu 22.04.4 LTS	0.1.42	llava:13b	77.84
Linux	AMD Ryzen Threadripper 7980X 64-Cores	NVIDIA RTX A6000	125.54	Ubuntu 22.04.3 LTS	0.1.45	mistral:7b	83.77
Linux	AMD Ryzen Threadripper 7980X 64-Cores	NVIDIA RTX A6000	125.54	Ubuntu 22.04.3 LTS	0.1.45	llama3:8b	75.3
Linux	AMD Ryzen Threadripper 7980X 64-Cores	NVIDIA RTX A6000	125.54	Ubuntu 22.04.3 LTS	0.1.45	phi3:3.8b	79.35
Linux	AMD Ryzen Threadripper 7980X 64-Cores	NVIDIA RTX A6000	125.54	Ubuntu 22.04.3 LTS	0.1.45	gemma:2b	134.2
Linux	AMD Ryzen Threadripper 7980X 64-Cores	NVIDIA RTX A6000	125.54	Ubuntu 22.04.3 LTS	0.1.45	gemma:7b	73.44
Linux	AMD Ryzen Threadripper 7980X 64-Cores	NVIDIA RTX A6000	125.54	Ubuntu 22.04.3 LTS	0.1.45	llava:7b	85.23
Linux	AMD Ryzen Threadripper 7980X 64-Cores	NVIDIA RTX A6000	125.54	Ubuntu 22.04.3 LTS	0.1.45	llava:13b	57.24
Linux	AMD Ryzen Threadripper 7980X 64-Cores	NVIDIA RTX A6000	125.54	Ubuntu 22.04.3 LTS	0.1.45	mistral:7b	82.42
Linux	AMD Ryzen Threadripper 7980X 64-Cores	NVIDIA RTX A6000	125.54	Ubuntu 22.04.3 LTS	0.1.45	llama3:8b	75.18
Linux	AMD Ryzen Threadripper 7980X 64-Cores	NVIDIA RTX A6000	125.54	Ubuntu 22.04.3 LTS	0.1.45	phi3:3.8b	108.69
Linux	AMD Ryzen Threadripper 7980X 64-Cores	NVIDIA RTX A6000	125.54	Ubuntu 22.04.3 LTS	0.1.45	gemma:2b	132.42
Linux	AMD Ryzen Threadripper 7980X 64-Cores	NVIDIA RTX A6000	125.54	Ubuntu 22.04.3 LTS	0.1.45	gemma:7b	73.52
Linux	AMD Ryzen Threadripper 7980X 64-Cores	NVIDIA RTX A6000	125.54	Ubuntu 22.04.3 LTS	0.1.45	llava:7b	85.04
Linux	AMD Ryzen Threadripper 7980X 64-Cores	NVIDIA RTX A6000	125.54	Ubuntu 22.04.3 LTS	0.1.45	llava:13b	53.81
Linux	AMD Ryzen 5 7600 6-Core Processor	NVIDIA GeForce RTX 3090	187.38	"Ubuntu 22.04.4 LTS"	v0.1.45-rc2	mistral:7b	83.04
Linux	AMD Ryzen 5 7600 6-Core Processor	NVIDIA GeForce RTX 3090	187.38	"Ubuntu 22.04.4 LTS"	v0.1.45-rc2	llama3:8b	74.68
Linux	AMD Ryzen 5 7600 6-Core Processor	NVIDIA GeForce RTX 3090	187.38	"Ubuntu 22.04.4 LTS"	v0.1.45-rc2	phi3:3.8b	103.11
Linux	AMD Ryzen 5 7600 6-Core Processor	NVIDIA GeForce RTX 3090	187.38	"Ubuntu 22.04.4 LTS"	v0.1.45-rc2	gemma:2b	148.61
Linux	AMD Ryzen 5 7600 6-Core Processor	NVIDIA GeForce RTX 3090	187.38	"Ubuntu 22.04.4 LTS"	v0.1.45-rc2	gemma:7b	66.64
Linux	AMD Ryzen 5 7600 6-Core Processor	NVIDIA GeForce RTX 3090	187.38	"Ubuntu 22.04.4 LTS"	v0.1.45-rc2	llava:7b	90.46
Linux	AMD Ryzen 5 7600 6-Core Processor	NVIDIA GeForce RTX 3090	187.38	"Ubuntu 22.04.4 LTS"	v0.1.45-rc2	llava:13b	54.55
Linux	AMD Ryzen 5 7600 6-Core Processor	NVIDIA GeForce RTX 3090	187.38	"Ubuntu 22.04.4 LTS"	v0.1.45-rc2	mistral:7b	141.12
Linux	AMD Ryzen 5 7600 6-Core Processor	NVIDIA GeForce RTX 3090	187.38	"Ubuntu 22.04.4 LTS"	v0.1.45-rc2	llama3:8b	122.65
Linux	AMD Ryzen 5 7600 6-Core Processor	NVIDIA GeForce RTX 3090	187.38	"Ubuntu 22.04.4 LTS"	v0.1.45-rc2	phi3:3.8b	165.9
Linux	AMD Ryzen 5 7600 6-Core Processor	NVIDIA GeForce RTX 3090	187.38	"Ubuntu 22.04.4 LTS"	v0.1.45-rc2	gemma:2b	229.74
Linux	AMD Ryzen 5 7600 6-Core Processor	NVIDIA GeForce RTX 3090	187.38	"Ubuntu 22.04.4 LTS"	v0.1.45-rc2	gemma:7b	109.87
Linux	AMD Ryzen 5 7600 6-Core Processor	NVIDIA GeForce RTX 3090	187.38	"Ubuntu 22.04.4 LTS"	v0.1.45-rc2	llava:7b	153.31
Linux	AMD Ryzen 5 7600 6-Core Processor	NVIDIA GeForce RTX 3090	187.38	"Ubuntu 22.04.4 LTS"	v0.1.45-rc2	llava:13b	92.05
Linux	AMD Ryzen 5 7600 6-Core Processor	NVIDIA GeForce RTX 3090	187.38	"Ubuntu 22.04.4 LTS"	v0.1.45-rc2	mistral:7b	140.87
Linux	AMD Ryzen 5 7600 6-Core Processor	NVIDIA GeForce RTX 3090	187.38	"Ubuntu 22.04.4 LTS"	v0.1.45-rc2	llama3:8b	121.96
Linux	AMD Ryzen 5 7600 6-Core Processor	NVIDIA GeForce RTX 3090	187.38	"Ubuntu 22.04.4 LTS"	v0.1.45-rc2	phi3:3.8b	170.72
Linux	AMD Ryzen 5 7600 6-Core Processor	NVIDIA GeForce RTX 3090	187.38	"Ubuntu 22.04.4 LTS"	v0.1.45-rc2	gemma:2b	225.46
Linux	AMD Ryzen 5 7600 6-Core Processor	NVIDIA GeForce RTX 3090	187.38	"Ubuntu 22.04.4 LTS"	v0.1.45-rc2	gemma:7b	108.78
Linux	AMD Ryzen 5 7600 6-Core Processor	NVIDIA GeForce RTX 3090	187.38	"Ubuntu 22.04.4 LTS"	v0.1.45-rc2	llava:7b	153.57
Linux	AMD Ryzen 5 7600 6-Core Processor	NVIDIA GeForce RTX 3090	187.38	"Ubuntu 22.04.4 LTS"	v0.1.45-rc2	llava:13b	54.25
Linux	Intel(R) Xeon(R) CPU E5-2690 v4 @ 2.60GHz	NVIDIA GeForce GTX 1080	2.68	Ubuntu 24.04 LTS	0.1.45	phi:2.7b	96.34
Linux	Intel(R) Xeon(R) CPU E5-2690 v4 @ 2.60GHz	NVIDIA GeForce GTX 1080	2.68	Ubuntu 24.04 LTS	0.1.45	gemma:2b	84.31
Linux	Intel(R) Xeon(R) CPU E5-2690 v4 @ 2.60GHz	NVIDIA GeForce GTX 1080	2.68	Ubuntu 24.04 LTS	0.1.45	phi:2.7b	98.27
Linux	Intel(R) Xeon(R) CPU E5-2690 v4 @ 2.60GHz	NVIDIA GeForce GTX 1080	2.68	Ubuntu 24.04 LTS	0.1.45	gemma:2b	86.35
Linux	Intel(R) Xeon(R) CPU E5-2690 v4 @ 2.60GHz	NVIDIA GeForce GTX 1080	2.68	Ubuntu 24.04 LTS	0.1.45	phi:2.7b	76.92
Linux	Intel(R) Xeon(R) CPU E5-2690 v4 @ 2.60GHz	NVIDIA GeForce GTX 1080	2.68	Ubuntu 24.04 LTS	0.1.45	gemma:2b	82.46
Linux	Intel(R) Xeon(R) Gold 6442Y	NVIDIA A40	503.39	Ubuntu 22.04.4 LTS	0.1.45	command-r-plus:latest	4.65
Linux	Intel(R) Xeon(R) Gold 6442Y	NVIDIA A40	503.39	Ubuntu 22.04.4 LTS	0.1.45	dbrx:latest	11.2
Linux	Intel(R) Xeon(R) Gold 6442Y	NVIDIA A40	503.39	Ubuntu 22.04.4 LTS	0.1.45	mixtral:8x22b	10.59
Linux	12th Gen Intel(R) Core(TM) i5-1250P	Alder Lake-P Integrated Graphics Controller	30.91	Linux Mint 21.3	0.1.45	mistral:7b	6.68
Linux	12th Gen Intel(R) Core(TM) i5-1250P	Alder Lake-P Integrated Graphics Controller	30.91	Linux Mint 21.3	0.1.45	llama3:8b	6.68
Linux	12th Gen Intel(R) Core(TM) i5-1250P	Alder Lake-P Integrated Graphics Controller	30.91	Linux Mint 21.3	0.1.45	phi3:3.8b	12.88
Linux	12th Gen Intel(R) Core(TM) i5-1250P	Alder Lake-P Integrated Graphics Controller	30.91	Linux Mint 21.3	0.1.45	gemma:2b	16.97
Linux	12th Gen Intel(R) Core(TM) i5-1250P	Alder Lake-P Integrated Graphics Controller	30.91	Linux Mint 21.3	0.1.45	gemma:7b	5.79
Linux	12th Gen Intel(R) Core(TM) i5-1250P	Alder Lake-P Integrated Graphics Controller	30.91	Linux Mint 21.3	0.1.45	llava:7b	7.48
Linux	12th Gen Intel(R) Core(TM) i5-1250P	Alder Lake-P Integrated Graphics Controller	30.91	Linux Mint 21.3	0.1.45	llava:13b	4.07
Linux	Intel(R) Xeon(R) CPU E5-2690 v4 @ 2.60GHz	Quadro M4000	62.24	Ubuntu 24.04 LTS	0.1.41	mistral:7b	5.71
Linux	Intel(R) Xeon(R) CPU E5-2690 v4 @ 2.60GHz	Quadro M4000	62.24	Ubuntu 24.04 LTS	0.1.41	llama3:8b	5.33
Linux	Intel(R) Xeon(R) CPU E5-2690 v4 @ 2.60GHz	Quadro M4000	62.24	Ubuntu 24.04 LTS	0.1.41	phi3:3.8b	11.85
Linux	Intel(R) Xeon(R) CPU E5-2690 v4 @ 2.60GHz	Quadro M4000	62.24	Ubuntu 24.04 LTS	0.1.41	gemma:2b	16.02
Linux	Intel(R) Xeon(R) CPU E5-2690 v4 @ 2.60GHz	Quadro M4000	62.24	Ubuntu 24.04 LTS	0.1.41	gemma:7b	4.79
Linux	Intel(R) Xeon(R) CPU E5-2690 v4 @ 2.60GHz	Quadro M4000	62.24	Ubuntu 24.04 LTS	0.1.41	llava:7b	5.85
Linux	Intel(R) Xeon(R) CPU E5-2690 v4 @ 2.60GHz	Quadro M4000	62.24	Ubuntu 24.04 LTS	0.1.41	llava:13b	2.95
Linux	13th Gen Intel(R) Core(TM) i7-13700	Intel Corporation	15.37	Ubuntu 22.04.3 LTS	0.1.41	mistral:7b	9.67
Linux	13th Gen Intel(R) Core(TM) i7-13700	Intel Corporation	15.37	Ubuntu 22.04.3 LTS	0.1.41	llama3:8b	9.03
Linux	13th Gen Intel(R) Core(TM) i7-13700	Intel Corporation	15.37	Ubuntu 22.04.3 LTS	0.1.41	phi3:3.8b	16.56
Linux	13th Gen Intel(R) Core(TM) i7-13700	Intel Corporation	15.37	Ubuntu 22.04.3 LTS	0.1.41	gemma:2b	22.93
Linux	13th Gen Intel(R) Core(TM) i7-13700	Intel Corporation	15.37	Ubuntu 22.04.3 LTS	0.1.41	gemma:7b	7.84
Linux	13th Gen Intel(R) Core(TM) i7-13700	Intel Corporation	15.37	Ubuntu 22.04.3 LTS	0.1.41	llava:7b	10.11
Linux	13th Gen Intel(R) Core(TM) i7-13700	Intel Corporation	15.37	Ubuntu 22.04.3 LTS	0.1.41	llava:13b	5.66
Linux	Intel(R) Xeon(R) W-2223 CPU @ 3.60GHz	NVIDIA GeForce RTX 3070	31.05	"Debian GNU/Linux 12 (bookworm)"	0.1.43	codellama:latest	70.41
Linux	Intel(R) Xeon(R) CPU E5-2650 v4 @ 2.20GHz	Quadro P4000	62.72	Ubuntu 24.04 LTS	0.1.46	mistral:7b	2.96
Linux	Intel(R) Xeon(R) CPU E5-2650 v4 @ 2.20GHz	Quadro P4000	62.72	Ubuntu 24.04 LTS	0.1.46	llama3:8b	1.88
Linux	Intel(R) Xeon(R) CPU E5-2650 v4 @ 2.20GHz	Quadro P4000	62.72	Ubuntu 24.04 LTS	0.1.46	phi3:3.8b	5.06
Linux	Intel(R) Xeon(R) CPU E5-2650 v4 @ 2.20GHz	Quadro P4000	62.72	Ubuntu 24.04 LTS	0.1.46	gemma:2b	61.44
Linux	Intel(R) Xeon(R) CPU E5-2650 v4 @ 2.20GHz	Quadro P4000	62.72	Ubuntu 24.04 LTS	0.1.46	gemma:7b	2.52
Linux	Intel(R) Xeon(R) CPU E5-2650 v4 @ 2.20GHz	Quadro P4000	62.72	Ubuntu 24.04 LTS	0.1.46	llava:7b	3.05
Linux	Intel(R) Xeon(R) CPU E5-2650 v4 @ 2.20GHz	Quadro P4000	62.72	Ubuntu 24.04 LTS	0.1.46	llava:13b	2.36
Linux	Intel(R) Xeon(R) CPU E5-2650 v4 @ 2.20GHz	Quadro P4000	62.72	Ubuntu 24.04 LTS	0.1.46	mistral:7b	33.45
Linux	Intel(R) Xeon(R) CPU E5-2650 v4 @ 2.20GHz	Quadro P4000	62.72	Ubuntu 24.04 LTS	0.1.46	llama3:8b	25.96
Linux	Intel(R) Xeon(R) CPU E5-2650 v4 @ 2.20GHz	Quadro P4000	62.72	Ubuntu 24.04 LTS	0.1.46	phi3:3.8b	36.81
Linux	Intel(R) Xeon(R) CPU E5-2650 v4 @ 2.20GHz	Quadro P4000	62.72	Ubuntu 24.04 LTS	0.1.46	gemma:2b	58.68
Linux	Intel(R) Xeon(R) CPU E5-2650 v4 @ 2.20GHz	Quadro P4000	62.72	Ubuntu 24.04 LTS	0.1.46	gemma:7b	24.56
Linux	Intel(R) Xeon(R) CPU E5-2650 v4 @ 2.20GHz	Quadro P4000	62.72	Ubuntu 24.04 LTS	0.1.46	llava:7b	33.11
Linux	Intel(R) Xeon(R) CPU E5-2650 v4 @ 2.20GHz	Quadro P4000	62.72	Ubuntu 24.04 LTS	0.1.46	llava:13b	2.23
Linux	Intel(R) Xeon(R) CPU E5-2650 v4 @ 2.20GHz	Quadro P4000	62.72	Ubuntu 24.04 LTS	0.1.46	mistral:7b	32.59
Linux	Intel(R) Xeon(R) CPU E5-2650 v4 @ 2.20GHz	Quadro P4000	62.72	Ubuntu 24.04 LTS	0.1.46	llama3:8b	2.93
Linux	Intel(R) Xeon(R) CPU E5-2650 v4 @ 2.20GHz	Quadro P4000	62.72	Ubuntu 24.04 LTS	0.1.46	phi3:3.8b	38.1
Linux	Intel(R) Xeon(R) CPU E5-2650 v4 @ 2.20GHz	Quadro P4000	62.72	Ubuntu 24.04 LTS	0.1.46	gemma:2b	58.62
Linux	Intel(R) Xeon(R) CPU E5-2650 v4 @ 2.20GHz	Quadro P4000	62.72	Ubuntu 24.04 LTS	0.1.46	gemma:7b	24.51
Linux	Intel(R) Xeon(R) CPU E5-2650 v4 @ 2.20GHz	Quadro P4000	62.72	Ubuntu 24.04 LTS	0.1.46	llava:7b	3.3
Linux	Intel(R) Xeon(R) CPU E5-2650 v4 @ 2.20GHz	Quadro P4000	62.72	Ubuntu 24.04 LTS	0.1.46	llava:13b	1.21
Linux	Intel(R) Xeon(R) CPU E5-2650 v4 @ 2.20GHz	Quadro P4000	62.72	Ubuntu 24.04 LTS	0.1.46	mistral:7b	30.47
Linux	Intel(R) Xeon(R) CPU E5-2650 v4 @ 2.20GHz	Quadro P4000	62.72	Ubuntu 24.04 LTS	0.1.46	llama3:8b	2.74
Linux	Intel(R) Xeon(R) CPU E5-2650 v4 @ 2.20GHz	Quadro P4000	62.72	Ubuntu 24.04 LTS	0.1.46	phi3:3.8b	37.67
Linux	Intel(R) Xeon(R) CPU E5-2650 v4 @ 2.20GHz	Quadro P4000	62.72	Ubuntu 24.04 LTS	0.1.46	gemma:2b	58.61
Linux	Intel(R) Xeon(R) CPU E5-2650 v4 @ 2.20GHz	Quadro P4000	62.72	Ubuntu 24.04 LTS	0.1.46	gemma:7b	24.55
Linux	Intel(R) Xeon(R) CPU E5-2650 v4 @ 2.20GHz	Quadro P4000	62.72	Ubuntu 24.04 LTS	0.1.46	llava:7b	3.9
Linux	Intel(R) Xeon(R) CPU E5-2650 v4 @ 2.20GHz	Quadro P4000	62.72	Ubuntu 24.04 LTS	0.1.46	llava:13b	1.43
Linux	Intel(R) Xeon(R) CPU E5-2650 v4 @ 2.20GHz	Quadro P4000	62.72	Ubuntu 24.04 LTS	0.1.46	mistral:7b	2.26
Linux	Intel(R) Xeon(R) CPU E5-2650 v4 @ 2.20GHz	Quadro P4000	62.72	Ubuntu 24.04 LTS	0.1.46	llama3:8b	2.11
Linux	Intel(R) Xeon(R) CPU E5-2650 v4 @ 2.20GHz	Quadro P4000	62.72	Ubuntu 24.04 LTS	0.1.46	phi3:3.8b	6.01
Linux	Intel(R) Xeon(R) CPU E5-2650 v4 @ 2.20GHz	Quadro P4000	62.72	Ubuntu 24.04 LTS	0.1.46	gemma:2b	8.33
Linux	Intel(R) Xeon(R) CPU E5-2650 v4 @ 2.20GHz	Quadro P4000	62.72	Ubuntu 24.04 LTS	0.1.46	gemma:7b	2.16
Linux	Intel(R) Xeon(R) CPU E5-2650 v4 @ 2.20GHz	Quadro P4000	62.72	Ubuntu 24.04 LTS	0.1.46	llava:7b	2.38
Linux	Intel(R) Xeon(R) CPU E5-2650 v4 @ 2.20GHz	Quadro P4000	62.72	Ubuntu 24.04 LTS	0.1.46	llava:13b	1.61
Linux	Intel(R) Xeon(R) CPU E5-2650 v4 @ 2.20GHz	Quadro P4000	62.72	Ubuntu 24.04 LTS	0.1.46	mistral:7b	32.26
Linux	Intel(R) Xeon(R) CPU E5-2650 v4 @ 2.20GHz	Quadro P4000	62.72	Ubuntu 24.04 LTS	0.1.46	llama3:8b	25.89
Linux	Intel(R) Xeon(R) CPU E5-2650 v4 @ 2.20GHz	Quadro P4000	62.72	Ubuntu 24.04 LTS	0.1.46	phi3:3.8b	36.08
Linux	Intel(R) Xeon(R) CPU E5-2650 v4 @ 2.20GHz	Quadro P4000	62.72	Ubuntu 24.04 LTS	0.1.46	gemma:2b	58.25
Linux	Intel(R) Xeon(R) CPU E5-2650 v4 @ 2.20GHz	Quadro P4000	62.72	Ubuntu 24.04 LTS	0.1.46	gemma:7b	24.46
Linux	Intel(R) Xeon(R) CPU E5-2650 v4 @ 2.20GHz	Quadro P4000	62.72	Ubuntu 24.04 LTS	0.1.46	llava:7b	32.6
Linux	Intel(R) Xeon(R) CPU E5-2650 v4 @ 2.20GHz	Quadro P4000	62.72	Ubuntu 24.04 LTS	0.1.46	llava:13b	2.2
Linux	Intel(R) Xeon(R) CPU E5-2650 v4 @ 2.20GHz	Quadro P4000	62.72	Ubuntu 24.04 LTS	0.1.46	mistral:7b	34.34
Linux	Intel(R) Xeon(R) CPU E5-2650 v4 @ 2.20GHz	Quadro P4000	62.72	Ubuntu 24.04 LTS	0.1.46	llama3:8b	26.86
Linux	Intel(R) Xeon(R) CPU E5-2650 v4 @ 2.20GHz	Quadro P4000	62.72	Ubuntu 24.04 LTS	0.1.46	phi3:3.8b	36.51
Linux	Intel(R) Xeon(R) CPU E5-2650 v4 @ 2.20GHz	Quadro P4000	62.72	Ubuntu 24.04 LTS	0.1.46	gemma:2b	58.15
Linux	Intel(R) Xeon(R) CPU E5-2650 v4 @ 2.20GHz	Quadro P4000	62.72	Ubuntu 24.04 LTS	0.1.46	gemma:7b	24.81
Linux	Intel(R) Xeon(R) CPU E5-2650 v4 @ 2.20GHz	Quadro P4000	62.72	Ubuntu 24.04 LTS	0.1.46	llava:7b	33.07
Linux	Intel(R) Xeon(R) CPU E5-2650 v4 @ 2.20GHz	Quadro P4000	62.72	Ubuntu 24.04 LTS	0.1.46	llava:13b	4.79
Linux	Intel(R) Core(TM) i7-4765T CPU @ 2.00GHz	Xeon E3-1200 v3/4th Gen Core Processor Integrated Graphics Controller	15.51	Ubuntu 22.04.2 LTS	0.1.46	mistral:7b	1.8
Linux	Intel(R) Core(TM) i7-4765T CPU @ 2.00GHz	Xeon E3-1200 v3/4th Gen Core Processor Integrated Graphics Controller	15.51	Ubuntu 22.04.2 LTS	0.1.46	llama3:8b	1.75
Linux	Intel(R) Core(TM) i7-4765T CPU @ 2.00GHz	Xeon E3-1200 v3/4th Gen Core Processor Integrated Graphics Controller	15.51	Ubuntu 22.04.2 LTS	0.1.46	phi3:3.8b	3.82
Linux	Intel(R) Core(TM) i7-4765T CPU @ 2.00GHz	Xeon E3-1200 v3/4th Gen Core Processor Integrated Graphics Controller	15.51	Ubuntu 22.04.2 LTS	0.1.46	gemma:2b	4.82
Linux	Intel(R) Core(TM) i7-4765T CPU @ 2.00GHz	Xeon E3-1200 v3/4th Gen Core Processor Integrated Graphics Controller	15.51	Ubuntu 22.04.2 LTS	0.1.46	gemma:7b	1.52
Linux	Intel(R) Core(TM) i7-4765T CPU @ 2.00GHz	Xeon E3-1200 v3/4th Gen Core Processor Integrated Graphics Controller	15.51	Ubuntu 22.04.2 LTS	0.1.46	llava:7b	1.9
Linux	Intel(R) Core(TM) i7-4765T CPU @ 2.00GHz	Xeon E3-1200 v3/4th Gen Core Processor Integrated Graphics Controller	15.51	Ubuntu 22.04.2 LTS	0.1.46	llava:13b	1.05
Linux	Intel(R) Xeon(R) CPU E5-2650 v4 @ 2.20GHz	Quadro P4000	62.72	Ubuntu 24.04 LTS	0.1.46	mistral:7b	32.39
Linux	Intel(R) Xeon(R) CPU E5-2650 v4 @ 2.20GHz	Quadro P4000	62.72	Ubuntu 24.04 LTS	0.1.46	llama3:8b	28.22
Linux	Intel(R) Xeon(R) CPU E5-2650 v4 @ 2.20GHz	Quadro P4000	62.72	Ubuntu 24.04 LTS	0.1.46	phi3:3.8b	6.17
Linux	Intel(R) Xeon(R) CPU E5-2650 v4 @ 2.20GHz	Quadro P4000	62.72	Ubuntu 24.04 LTS	0.1.46	gemma:2b	67.21
Linux	Intel(R) Xeon(R) CPU E5-2650 v4 @ 2.20GHz	Quadro P4000	62.72	Ubuntu 24.04 LTS	0.1.46	gemma:7b	5.75
Linux	Intel(R) Xeon(R) CPU E5-2650 v4 @ 2.20GHz	Quadro P4000	62.72	Ubuntu 24.04 LTS	0.1.46	llava:7b	36.44
Linux	Intel(R) Xeon(R) CPU E5-2650 v4 @ 2.20GHz	Quadro P4000	62.72	Ubuntu 24.04 LTS	0.1.46	llava:13b	3.93
Linux	AMD A4 PRO-7300B APU with Radeon HD Graphics	Richland [Radeon HD 8470D]	13.57	Ubuntu 24.04 LTS	0.1.44	mistral:7b	0.69
Linux	AMD A4 PRO-7300B APU with Radeon HD Graphics	Richland [Radeon HD 8470D]	13.57	Ubuntu 24.04 LTS	0.1.44	llama3:8b	0.69
Linux	AMD A4 PRO-7300B APU with Radeon HD Graphics	Richland [Radeon HD 8470D]	13.57	Ubuntu 24.04 LTS	0.1.44	phi3:3.8b	1.01
Linux	AMD A4 PRO-7300B APU with Radeon HD Graphics	Richland [Radeon HD 8470D]	13.57	Ubuntu 24.04 LTS	0.1.44	gemma:2b	2.09
Linux	AMD A4 PRO-7300B APU with Radeon HD Graphics	Richland [Radeon HD 8470D]	13.57	Ubuntu 24.04 LTS	0.1.44	gemma:7b	0.61
Linux	AMD A4 PRO-7300B APU with Radeon HD Graphics	Richland [Radeon HD 8470D]	13.57	Ubuntu 24.04 LTS	0.1.44	llava:7b	0.83
Linux	Intel(R) Core(TM) i7-6700 CPU @ 3.40GHz	HD Graphics 530	31.23	Ubuntu 20.04.6 LTS	0.1.38	mistral:7b	5.09
Linux	Intel(R) Core(TM) i7-6700 CPU @ 3.40GHz	HD Graphics 530	31.23	Ubuntu 20.04.6 LTS	0.1.38	llama3:8b	4.71
Linux	Intel(R) Core(TM) i7-6700 CPU @ 3.40GHz	HD Graphics 530	31.23	Ubuntu 20.04.6 LTS	0.1.38	phi3:3.8b	8.32
Linux	Intel(R) Core(TM) i7-6700 CPU @ 3.40GHz	HD Graphics 530	31.23	Ubuntu 20.04.6 LTS	0.1.38	gemma:2b	11.71
Linux	Intel(R) Core(TM) i7-6700 CPU @ 3.40GHz	HD Graphics 530	31.23	Ubuntu 20.04.6 LTS	0.1.38	gemma:7b	4.15
Linux	Intel(R) Core(TM) i7-6700 CPU @ 3.40GHz	HD Graphics 530	31.23	Ubuntu 20.04.6 LTS	0.1.38	llava:7b	5.21
Linux	Intel(R) Core(TM) i7-6700 CPU @ 3.40GHz	HD Graphics 530	31.23	Ubuntu 20.04.6 LTS	0.1.38	llava:13b	2.71
Linux	unknown	NVIDIA A100 80GB PCIe	1021.17	Ubuntu 22.04.4 LTS	0.1.47	mistral:7b	114.5
Linux	unknown	NVIDIA A100 80GB PCIe	1021.17	Ubuntu 22.04.4 LTS	0.1.47	llama3:8b	88.85
Linux	unknown	NVIDIA A100 80GB PCIe	1021.17	Ubuntu 22.04.4 LTS	0.1.47	phi3:3.8b	125.71
Linux	unknown	NVIDIA A100 80GB PCIe	1021.17	Ubuntu 22.04.4 LTS	0.1.47	gemma:2b	105.45
Linux	unknown	NVIDIA A100 80GB PCIe	1021.17	Ubuntu 22.04.4 LTS	0.1.47	gemma:7b	114.87
Linux	unknown	NVIDIA A100 80GB PCIe	1021.17	Ubuntu 22.04.4 LTS	0.1.47	llava:7b	120.83
Linux	unknown	NVIDIA A100 80GB PCIe	1021.17	Ubuntu 22.04.4 LTS	0.1.47	llava:13b	84.61
Linux	unknown	NVIDIA A100 80GB PCIe	1021.17	Ubuntu 22.04.4 LTS	0.1.47	mistral:7b	114.65
Linux	unknown	NVIDIA A100 80GB PCIe	1021.17	Ubuntu 22.04.4 LTS	0.1.47	llama3:8b	87.98
Linux	unknown	NVIDIA A100 80GB PCIe	1021.17	Ubuntu 22.04.4 LTS	0.1.47	phi3:3.8b	139.4
Linux	unknown	NVIDIA A100 80GB PCIe	1021.17	Ubuntu 22.04.4 LTS	0.1.47	llava:7b	118.7
Linux	unknown	NVIDIA A100 80GB PCIe	1021.17	Ubuntu 22.04.4 LTS	0.1.47	llava:13b	89.42
Linux	unknown	NVIDIA A100 80GB PCIe	1021.17	Ubuntu 22.04.4 LTS	0.1.47	mistral:7b	116.69
Linux	unknown	NVIDIA A100 80GB PCIe	1021.17	Ubuntu 22.04.4 LTS	0.1.47	llama3:8b	87.38
Linux	unknown	NVIDIA A100 80GB PCIe	1021.17	Ubuntu 22.04.4 LTS	0.1.47	phi3:3.8b	125.59
Linux	unknown	NVIDIA A100 80GB PCIe	1021.17	Ubuntu 22.04.4 LTS	0.1.47	gemma2:9b	86.37
Linux	unknown	NVIDIA A100 80GB PCIe	1021.17	Ubuntu 22.04.4 LTS	0.1.47	gemma2:27b	51.02
Linux	unknown	NVIDIA A100 80GB PCIe	1021.17	Ubuntu 22.04.4 LTS	0.1.47	llava:7b	119.76
Linux	unknown	NVIDIA A100 80GB PCIe	1021.17	Ubuntu 22.04.4 LTS	0.1.47	llava:13b	83.81
Linux	Intel(R) Xeon(R) Gold 6448Y	NVIDIA A30	503.40	Rocky Linux 9.3 (Blue Onyx)	0.1.47	mistral:7b	96.74
Linux	Intel(R) Xeon(R) Gold 6448Y	NVIDIA A30	503.40	Rocky Linux 9.3 (Blue Onyx)	0.1.47	llama3:8b	82.01
Linux	Intel(R) Xeon(R) Gold 6448Y	NVIDIA A30	503.40	Rocky Linux 9.3 (Blue Onyx)	0.1.47	phi3:3.8b	109.41
Linux	Intel(R) Xeon(R) Gold 6448Y	NVIDIA A30	503.40	Rocky Linux 9.3 (Blue Onyx)	0.1.47	gemma:2b	136.08
Linux	Intel(R) Xeon(R) Gold 6448Y	NVIDIA A30	503.40	Rocky Linux 9.3 (Blue Onyx)	0.1.47	gemma:7b	74.92
Linux	Intel(R) Xeon(R) Gold 6448Y	NVIDIA A30	503.40	Rocky Linux 9.3 (Blue Onyx)	0.1.47	llava:7b	99.42
Linux	Intel(R) Xeon(R) Gold 6448Y	NVIDIA A30	503.40	Rocky Linux 9.3 (Blue Onyx)	0.1.47	llava:13b	61.91
Linux	Intel(R) Core(TM) i7-8550U CPU @ 1.80GHz	UHD Graphics 620	15.39	Ubuntu 24.04 LTS	0.1.47	mistral:7b	3.08
Linux	Intel(R) Core(TM) i7-8550U CPU @ 1.80GHz	UHD Graphics 620	15.39	Ubuntu 24.04 LTS	0.1.47	llama3:8b	3.35
Linux	Intel(R) Core(TM) i7-8550U CPU @ 1.80GHz	UHD Graphics 620	15.39	Ubuntu 24.04 LTS	0.1.47	phi3:3.8b	7.26
Linux	Intel(R) Core(TM) i7-8550U CPU @ 1.80GHz	UHD Graphics 620	15.39	Ubuntu 24.04 LTS	0.1.47	gemma:2b	9.54
Linux	Intel(R) Core(TM) i7-8550U CPU @ 1.80GHz	UHD Graphics 620	15.39	Ubuntu 24.04 LTS	0.1.47	gemma:7b	3.53
Linux	Intel(R) Core(TM) i7-8550U CPU @ 1.80GHz	UHD Graphics 620	15.39	Ubuntu 24.04 LTS	0.1.47	llava:7b	4.74
Linux	Intel(R) Core(TM) i7-8550U CPU @ 1.80GHz	UHD Graphics 620	15.39	Ubuntu 24.04 LTS	0.1.47	llava:13b	2.69
Linux	unknown	NVIDIA A100 80GB PCIe	1021.17	Ubuntu 22.04.4 LTS	0.1.47	mistral:7b	126.15
Linux	unknown	NVIDIA A100 80GB PCIe	1021.17	Ubuntu 22.04.4 LTS	0.1.47	llama3:8b	110.97
Linux	unknown	NVIDIA A100 80GB PCIe	1021.17	Ubuntu 22.04.4 LTS	0.1.47	phi3:3.8b	139.21
Linux	unknown	NVIDIA A100 80GB PCIe	1021.17	Ubuntu 22.04.4 LTS	0.1.47	gemma2:9b	86.1
Linux	unknown	NVIDIA A100 80GB PCIe	1021.17	Ubuntu 22.04.4 LTS	0.1.47	gemma2:27b	38.29
Linux	unknown	NVIDIA A100 80GB PCIe	1021.17	Ubuntu 22.04.4 LTS	0.1.47	llava:7b	131.38
Linux	unknown	NVIDIA A100 80GB PCIe	1021.17	Ubuntu 22.04.4 LTS	0.1.47	llava:13b	89.51
Linux	AMD Ryzen Threadripper PRO 5955WX 16-Cores	Tesla M60	125.63	Debian GNU/Linux 12 (bookworm)	0.1.46	mistral:7b	10.25
Linux	AMD Ryzen Threadripper PRO 5955WX 16-Cores	Tesla M60	125.63	Debian GNU/Linux 12 (bookworm)	0.1.46	llama3:8b	9.69
Linux	AMD Ryzen Threadripper PRO 5955WX 16-Cores	Tesla M60	125.63	Debian GNU/Linux 12 (bookworm)	0.1.46	phi3:3.8b	21.58
Linux	AMD Ryzen Threadripper PRO 5955WX 16-Cores	Tesla M60	125.63	Debian GNU/Linux 12 (bookworm)	0.1.46	gemma:2b	27.14
Linux	AMD Ryzen Threadripper PRO 5955WX 16-Cores	Tesla M60	125.63	Debian GNU/Linux 12 (bookworm)	0.1.46	gemma:7b	8.7
Linux	AMD Ryzen Threadripper PRO 5955WX 16-Cores	Tesla M60	125.63	Debian GNU/Linux 12 (bookworm)	0.1.46	llava:7b	10.62
Linux	AMD Ryzen Threadripper PRO 5955WX 16-Cores	Tesla M60	125.63	Debian GNU/Linux 12 (bookworm)	0.1.46	llava:13b	5.95
Linux	AMD Ryzen 5 5600G with Radeon Graphics	no_gpu	30.71	"Fedora Linux 40 (Workstation Edition)"	0.1.48	mistral:7b	7.94
Linux	AMD Ryzen 5 5600G with Radeon Graphics	no_gpu	30.71	"Fedora Linux 40 (Workstation Edition)"	0.1.48	llama3:8b	7.29
Linux	AMD Ryzen 5 5600G with Radeon Graphics	no_gpu	30.71	"Fedora Linux 40 (Workstation Edition)"	0.1.48	phi3:3.8b	12.86
Linux	AMD Ryzen 5 5600G with Radeon Graphics	no_gpu	30.71	"Fedora Linux 40 (Workstation Edition)"	0.1.48	gemma:2b	17.92
Linux	AMD Ryzen 5 5600G with Radeon Graphics	no_gpu	30.71	"Fedora Linux 40 (Workstation Edition)"	0.1.48	gemma:7b	6.27
Linux	AMD Ryzen 5 5600G with Radeon Graphics	no_gpu	30.71	"Fedora Linux 40 (Workstation Edition)"	0.1.48	llava:7b	7.61
Linux	AMD Ryzen 5 5600G with Radeon Graphics	no_gpu	30.71	"Fedora Linux 40 (Workstation Edition)"	0.1.48	llava:13b	4.5
Linux	Intel(R) Core(TM) i7-3770 CPU @ 3.40GHz	no_gpu	25.41	Debian GNU/Linux 12 (bookworm)	0.1.48	mistral:7b	3.45
Linux	Intel(R) Core(TM) i7-3770 CPU @ 3.40GHz	no_gpu	25.41	Debian GNU/Linux 12 (bookworm)	0.1.48	llama3:8b	3.34
Linux	Intel(R) Core(TM) i7-3770 CPU @ 3.40GHz	no_gpu	25.41	Debian GNU/Linux 12 (bookworm)	0.1.48	phi3:3.8b	6.05
Linux	Intel(R) Core(TM) i7-3770 CPU @ 3.40GHz	no_gpu	25.41	Debian GNU/Linux 12 (bookworm)	0.1.48	gemma:2b	8.91
Linux	Intel(R) Core(TM) i7-3770 CPU @ 3.40GHz	no_gpu	25.41	Debian GNU/Linux 12 (bookworm)	0.1.48	gemma:7b	3.03
Linux	Intel(R) Core(TM) i7-3770 CPU @ 3.40GHz	no_gpu	25.41	Debian GNU/Linux 12 (bookworm)	0.1.48	llava:7b	3.83
Linux	Intel(R) Core(TM) i7-3770 CPU @ 3.40GHz	no_gpu	25.41	Debian GNU/Linux 12 (bookworm)	0.1.48	llava:13b	2.12
Linux	Intel(R) Xeon(R) CPU E5-2690 v4 @ 2.60GHz	Tesla P40	62.24	Ubuntu 24.04 LTS	0.1.45	mistral:7b	48.44
Linux	Intel(R) Xeon(R) CPU E5-2690 v4 @ 2.60GHz	Tesla P40	62.24	Ubuntu 24.04 LTS	0.1.45	llama3:8b	41.3
Linux	Intel(R) Xeon(R) CPU E5-2690 v4 @ 2.60GHz	Tesla P40	62.24	Ubuntu 24.04 LTS	0.1.45	phi3:3.8b	59.04
Linux	Intel(R) Xeon(R) CPU E5-2690 v4 @ 2.60GHz	Tesla P40	62.24	Ubuntu 24.04 LTS	0.1.45	gemma:2b	93.36
Linux	Intel(R) Xeon(R) CPU E5-2690 v4 @ 2.60GHz	Tesla P40	62.24	Ubuntu 24.04 LTS	0.1.45	gemma:7b	36.88
Linux	Intel(R) Xeon(R) CPU E5-2690 v4 @ 2.60GHz	Tesla P40	62.24	Ubuntu 24.04 LTS	0.1.45	llava:7b	49.34
Linux	Intel(R) Xeon(R) CPU E5-2690 v4 @ 2.60GHz	Tesla P40	62.24	Ubuntu 24.04 LTS	0.1.45	llava:13b	19.28
Linux	AMD Ryzen 5 5600G with Radeon Graphics	NVIDIA GeForce RTX 3060	30.72	"Fedora Linux 40 (Workstation Edition)"	0.1.48	mistral:7b	63.33
Linux	AMD Ryzen 5 5600G with Radeon Graphics	NVIDIA GeForce RTX 3060	30.72	"Fedora Linux 40 (Workstation Edition)"	0.1.48	llama3:8b	56.28
Linux	AMD Ryzen 5 5600G with Radeon Graphics	NVIDIA GeForce RTX 3060	30.72	"Fedora Linux 40 (Workstation Edition)"	0.1.48	phi3:3.8b	89.63
Linux	AMD Ryzen 5 5600G with Radeon Graphics	NVIDIA GeForce RTX 3060	30.72	"Fedora Linux 40 (Workstation Edition)"	0.1.48	gemma:2b	125.44
Linux	AMD Ryzen 5 5600G with Radeon Graphics	NVIDIA GeForce RTX 3060	30.72	"Fedora Linux 40 (Workstation Edition)"	0.1.48	gemma:7b	52.16
Linux	AMD Ryzen 5 5600G with Radeon Graphics	NVIDIA GeForce RTX 3060	30.72	"Fedora Linux 40 (Workstation Edition)"	0.1.48	llava:7b	66.66
Linux	AMD Ryzen 5 5600G with Radeon Graphics	NVIDIA GeForce RTX 3060	30.72	"Fedora Linux 40 (Workstation Edition)"	0.1.48	llava:13b	39.15
Linux	Intel(R) Core(TM) i5-4590 CPU @ 3.30GHz	NVIDIA GeForce GTX 960	31.23	Debian GNU/Linux 12 (bookworm)	0.1.48	mistral:7b	5.29
Linux	Intel(R) Core(TM) i5-4590 CPU @ 3.30GHz	NVIDIA GeForce GTX 960	31.23	Debian GNU/Linux 12 (bookworm)	0.1.48	llama3:8b	4.41
Linux	Intel(R) Core(TM) i5-4590 CPU @ 3.30GHz	NVIDIA GeForce GTX 960	31.23	Debian GNU/Linux 12 (bookworm)	0.1.48	phi3:3.8b	6.27
Linux	Intel(R) Core(TM) i5-4590 CPU @ 3.30GHz	NVIDIA GeForce GTX 960	31.23	Debian GNU/Linux 12 (bookworm)	0.1.48	gemma:2b	12.93
Linux	Intel(R) Core(TM) i5-4590 CPU @ 3.30GHz	NVIDIA GeForce GTX 960	31.23	Debian GNU/Linux 12 (bookworm)	0.1.48	gemma:7b	3.32
Linux	Intel(R) Core(TM) i5-4590 CPU @ 3.30GHz	NVIDIA GeForce GTX 960	31.23	Debian GNU/Linux 12 (bookworm)	0.1.48	llava:7b	5.36
Linux	Intel(R) Core(TM) i5-4590 CPU @ 3.30GHz	NVIDIA GeForce GTX 960	31.23	Debian GNU/Linux 12 (bookworm)	0.1.48	llava:13b	2.49
Linux	unknown	NVIDIA A100 80GB PCIe	1021.17	Ubuntu 22.04.4 LTS	0.1.47	mistral:7b	127.57
Linux	unknown	NVIDIA A100 80GB PCIe	1021.17	Ubuntu 22.04.4 LTS	0.1.47	llama3:8b	111.77
Linux	unknown	NVIDIA A100 80GB PCIe	1021.17	Ubuntu 22.04.4 LTS	0.1.47	phi3:3.8b	141.46
Linux	unknown	NVIDIA A100 80GB PCIe	1021.17	Ubuntu 22.04.4 LTS	0.1.47	gemma2:9b	85.9
Linux	unknown	NVIDIA A100 80GB PCIe	1021.17	Ubuntu 22.04.4 LTS	0.1.47	gemma2:27b	46.81
Linux	unknown	NVIDIA A100 80GB PCIe	1021.17	Ubuntu 22.04.4 LTS	0.1.47	llava:7b	132.14
Linux	unknown	NVIDIA A100 80GB PCIe	1021.17	Ubuntu 22.04.4 LTS	0.1.47	llava:13b	90.8
Linux	unknown	NVIDIA A100 80GB PCIe	1021.17	Ubuntu 22.04.4 LTS	0.1.48	mistral:7b	126.09
Linux	unknown	NVIDIA A100 80GB PCIe	1021.17	Ubuntu 22.04.4 LTS	0.1.48	llama3:8b	88.21
Linux	unknown	NVIDIA A100 80GB PCIe	1021.17	Ubuntu 22.04.4 LTS	0.1.48	phi3:3.8b	126.88
Linux	unknown	NVIDIA A100 80GB PCIe	1021.17	Ubuntu 22.04.4 LTS	0.1.48	gemma2:9b	61.37
Linux	unknown	NVIDIA A100 80GB PCIe	1021.17	Ubuntu 22.04.4 LTS	0.1.48	gemma2:27b	38.53
Linux	unknown	NVIDIA A100 80GB PCIe	1021.17	Ubuntu 22.04.4 LTS	0.1.48	llava:7b	131.86
Linux	unknown	NVIDIA A100 80GB PCIe	1021.17	Ubuntu 22.04.4 LTS	0.1.48	llava:13b	84.78
Linux	Intel(R) Core(TM) i7-3770 CPU @ 3.40GHz	NVIDIA GeForce GTX 1080 Ti	31.29	Ubuntu 22.04.4 LTS	0.1.48	llama3:8b	48.08
Linux	Intel(R) Core(TM) i7-3770 CPU @ 3.40GHz	NVIDIA GeForce GTX 1080 Ti	31.29	Ubuntu 22.04.4 LTS	0.1.48	phi3:3.8b	79
Linux	Intel(R) Core(TM) i7-3770 CPU @ 3.40GHz	NVIDIA GeForce GTX 1080 Ti	31.29	Ubuntu 22.04.4 LTS	0.1.48	phi:2.7b	117.98
Linux	Intel(R) Core(TM) i7-3770 CPU @ 3.40GHz	NVIDIA GeForce GTX 1080 Ti	31.29	Ubuntu 22.04.4 LTS	0.1.48	phi:2.7b	144.06
Linux	Intel(R) Xeon(R) CPU E5-2696 v3 @ 2.30GHz	NVIDIA GeForce RTX 4060 Ti	503.77	Ubuntu 22.04.4 LTS	0.1.48	llama3:8b	45.03
Linux	Intel(R) Xeon(R) CPU E5-2696 v3 @ 2.30GHz	NVIDIA GeForce RTX 4060 Ti	503.77	Ubuntu 22.04.4 LTS	0.1.48	phi3:3.8b	81.69
Linux	Intel(R) Xeon(R) CPU E5-2696 v3 @ 2.30GHz	NVIDIA GeForce RTX 4060 Ti	503.77	Ubuntu 22.04.4 LTS	0.1.48	phi:2.7b	119.05
Linux	AMD EPYC 7282 16-Core Processor	NVIDIA GeForce RTX 3090	755.75	Ubuntu 22.04.4 LTS	0.1.48	llama3:8b	90.93
Linux	AMD EPYC 7282 16-Core Processor	NVIDIA GeForce RTX 3090	755.75	Ubuntu 22.04.4 LTS	0.1.48	phi3:3.8b	151.03
Linux	AMD EPYC 7282 16-Core Processor	NVIDIA GeForce RTX 3090	755.75	Ubuntu 22.04.4 LTS	0.1.48	phi:2.7b	162.48
Linux	AMD EPYC 7282 16-Core Processor	NVIDIA GeForce RTX 3090	755.75	Ubuntu 22.04.4 LTS	0.1.48	phi:2.7b	161.55
Linux	AMD Ryzen 5 2600X Six-Core Processor	NVIDIA GeForce RTX 3060 Ti	15.63	Ubuntu 22.04.4 LTS	0.1.48	llama3:8b	63.6
Linux	AMD Ryzen 5 2600X Six-Core Processor	NVIDIA GeForce RTX 3060 Ti	15.63	Ubuntu 22.04.4 LTS	0.1.48	phi3:3.8b	107.6
Linux	AMD Ryzen 5 2600X Six-Core Processor	NVIDIA GeForce RTX 3060 Ti	15.63	Ubuntu 22.04.4 LTS	0.1.48	phi:2.7b	158.54
Linux	AMD Ryzen 5 5600 6-Core Processor	NVIDIA GeForce RTX 4090	62.70	Ubuntu 22.04.4 LTS	0.1.48	llama3:8b	121.92
Linux	AMD Ryzen 5 5600 6-Core Processor	NVIDIA GeForce RTX 4090	62.70	Ubuntu 22.04.4 LTS	0.1.48	phi3:3.8b	194.77
Linux	AMD Ryzen 5 5600 6-Core Processor	NVIDIA GeForce RTX 4090	62.70	Ubuntu 22.04.4 LTS	0.1.48	phi:2.7b	253.73
Linux	Intel(R) Core(TM) i5-8400 CPU @ 2.80GHz	NVIDIA GeForce RTX 3080	31.30	Ubuntu 22.04.4 LTS	0.1.48	llama3:8b	83.91
Linux	Intel(R) Core(TM) i5-8400 CPU @ 2.80GHz	NVIDIA GeForce RTX 3080	31.30	Ubuntu 22.04.4 LTS	0.1.48	phi3:3.8b	140.1
Linux	Intel(R) Core(TM) i5-8400 CPU @ 2.80GHz	NVIDIA GeForce RTX 3080	31.30	Ubuntu 22.04.4 LTS	0.1.48	phi:2.7b	200.66
Linux	AMD EPYC 7302 16-Core Processor	NVIDIA GeForce RTX 3080 Ti	125.67	Ubuntu 22.04.4 LTS	0.1.48	llama3:8b	92.55
Linux	AMD EPYC 7302 16-Core Processor	NVIDIA GeForce RTX 3080 Ti	125.67	Ubuntu 22.04.4 LTS	0.1.48	phi3:3.8b	154.72
Linux	AMD EPYC 7302 16-Core Processor	NVIDIA GeForce RTX 3080 Ti	125.67	Ubuntu 22.04.4 LTS	0.1.48	phi:2.7b	204.39
Linux	AMD EPYC 7302 16-Core Processor	NVIDIA GeForce RTX 3070	125.72	Ubuntu 22.04.4 LTS	0.1.48	llama3:8b	62.19
Linux	AMD EPYC 7302 16-Core Processor	NVIDIA GeForce RTX 3070	125.72	Ubuntu 22.04.4 LTS	0.1.48	phi3:3.8b	107.39
Linux	AMD EPYC 7302 16-Core Processor	NVIDIA GeForce RTX 3070	125.72	Ubuntu 22.04.4 LTS	0.1.48	phi:2.7b	183.42
Linux	AMD EPYC 7542 32-Core Processor	Tesla V100-PCIE-16GB	125.72	Ubuntu 22.04.4 LTS	0.1.48	llama3:8b	86.3
Linux	AMD EPYC 7542 32-Core Processor	Tesla V100-PCIE-16GB	125.72	Ubuntu 22.04.4 LTS	0.1.48	phi3:3.8b	134.74
Linux	AMD EPYC 7542 32-Core Processor	Tesla V100-PCIE-16GB	125.72	Ubuntu 22.04.4 LTS	0.1.48	phi:2.7b	146.26
Linux	Intel(R) Xeon(R) Gold 6134 CPU @ 3.20GHz	Tesla P100-SXM2-16GB	141.28	Ubuntu 22.04.4 LTS	0.1.48	llama3:8b	19.25
Linux	Intel(R) Xeon(R) Gold 6134 CPU @ 3.20GHz	Tesla P100-SXM2-16GB	141.28	Ubuntu 22.04.4 LTS	0.1.48	phi3:3.8b	37.06
Linux	Intel(R) Xeon(R) Gold 6134 CPU @ 3.20GHz	Tesla P100-SXM2-16GB	141.28	Ubuntu 22.04.4 LTS	0.1.48	phi:2.7b	57.61
Linux	AMD Ryzen 5 5600X 6-Core Processor	NVIDIA GeForce RTX 3060	31.26	Ubuntu 22.04.4 LTS	0.1.48	llama3:8b	54.87
Linux	AMD Ryzen 5 5600X 6-Core Processor	NVIDIA GeForce RTX 3060	31.26	Ubuntu 22.04.4 LTS	0.1.48	phi3:3.8b	91.36
Linux	AMD Ryzen 5 5600X 6-Core Processor	NVIDIA GeForce RTX 3060	31.26	Ubuntu 22.04.4 LTS	0.1.48	phi:2.7b	146
Linux	unknown	NVIDIA A100 80GB PCIe	1021.17	Ubuntu 22.04.4 LTS	0.1.48	mistral:7b	116.72
Linux	unknown	NVIDIA A100 80GB PCIe	1021.17	Ubuntu 22.04.4 LTS	0.1.48	llama3:8b	87.83
Linux	unknown	NVIDIA A100 80GB PCIe	1021.17	Ubuntu 22.04.4 LTS	0.1.48	phi3:3.8b	130.92
Linux	unknown	NVIDIA A100 80GB PCIe	1021.17	Ubuntu 22.04.4 LTS	0.1.48	gemma2:9b	61.56
Linux	unknown	NVIDIA A100 80GB PCIe	1021.17	Ubuntu 22.04.4 LTS	0.1.48	gemma2:27b	38.42
Linux	unknown	NVIDIA A100 80GB PCIe	1021.17	Ubuntu 22.04.4 LTS	0.1.48	llava:7b	130.68
Linux	unknown	NVIDIA A100 80GB PCIe	1021.17	Ubuntu 22.04.4 LTS	0.1.48	llava:13b	85.02
Linux	(None)	NVIDIA RTX A6000	15.33	Ubuntu 22.04.3 LTS	0.1.44	mistral:7b	101.22
Linux	(None)	NVIDIA RTX A6000	15.33	Ubuntu 22.04.3 LTS	0.1.44	llama3:8b	81.62
Linux	(None)	NVIDIA RTX A6000	15.33	Ubuntu 22.04.3 LTS	0.1.44	phi3:3.8b	138.51
Linux	(None)	NVIDIA RTX A6000	15.33	Ubuntu 22.04.3 LTS	0.1.44	gemma:2b	137.98
Linux	(None)	NVIDIA RTX A6000	15.33	Ubuntu 22.04.3 LTS	0.1.44	gemma:7b	77.07
Linux	(None)	NVIDIA RTX A6000	15.33	Ubuntu 22.04.3 LTS	0.1.44	llava:7b	104.17
Linux	(None)	NVIDIA RTX A6000	15.33	Ubuntu 22.04.3 LTS	0.1.44	llava:13b	66.55
Linux	Intel(R) Xeon(R) Gold 6226R CPU @ 2.90GHz	Tesla V100S-PCIE-32GB	43.05	Ubuntu 22.04.4 LTS	0.2.1	mistral:7b	113.13
Linux	Intel(R) Xeon(R) Gold 6226R CPU @ 2.90GHz	Tesla V100S-PCIE-32GB	43.05	Ubuntu 22.04.4 LTS	0.2.1	llama3:8b	89.86
Linux	Intel(R) Xeon(R) Gold 6226R CPU @ 2.90GHz	Tesla V100S-PCIE-32GB	43.05	Ubuntu 22.04.4 LTS	0.2.1	phi3:3.8b	137.81
Linux	Intel(R) Xeon(R) Gold 6226R CPU @ 2.90GHz	Tesla V100S-PCIE-32GB	43.05	Ubuntu 22.04.4 LTS	0.2.1	gemma:2b	156.72
Linux	Intel(R) Xeon(R) Gold 6226R CPU @ 2.90GHz	Tesla V100S-PCIE-32GB	43.05	Ubuntu 22.04.4 LTS	0.2.1	gemma:7b	91.01
Linux	Intel(R) Xeon(R) Gold 6226R CPU @ 2.90GHz	Tesla V100S-PCIE-32GB	43.05	Ubuntu 22.04.4 LTS	0.2.1	llava:7b	117.94
Linux	Intel(R) Xeon(R) Gold 6226R CPU @ 2.90GHz	Tesla V100S-PCIE-32GB	43.05	Ubuntu 22.04.4 LTS	0.2.1	llava:13b	79.41
Linux	13th Gen Intel(R) Core(TM) i5-13500	GRID P100-16Q	31.34	Debian GNU/Linux 12 (bookworm)	0.2.1	mistral:7b	46.24
Linux	13th Gen Intel(R) Core(TM) i5-13500	GRID P100-16Q	31.34	Debian GNU/Linux 12 (bookworm)	0.2.1	llama3:8b	41.4
Linux	13th Gen Intel(R) Core(TM) i5-13500	GRID P100-16Q	31.34	Debian GNU/Linux 12 (bookworm)	0.2.1	phi3:3.8b	70.31
Linux	13th Gen Intel(R) Core(TM) i5-13500	GRID P100-16Q	31.34	Debian GNU/Linux 12 (bookworm)	0.2.1	gemma:2b	89.38
Linux	13th Gen Intel(R) Core(TM) i5-13500	GRID P100-16Q	31.34	Debian GNU/Linux 12 (bookworm)	0.2.1	gemma:7b	38.51
Linux	13th Gen Intel(R) Core(TM) i5-13500	GRID P100-16Q	31.34	Debian GNU/Linux 12 (bookworm)	0.2.1	llava:7b	48
Linux	13th Gen Intel(R) Core(TM) i5-13500	GRID P100-16Q	31.34	Debian GNU/Linux 12 (bookworm)	0.2.1	llava:13b	29.45
Linux	AMD Ryzen 7 7840HS w/ Radeon 780M Graphics	Phoenix1	38.55	Ubuntu 24.04 LTS	0.1.48	gemma:2b	19.49
Linux	AMD Ryzen 7 7840HS w/ Radeon 780M Graphics	Phoenix1	38.55	Ubuntu 24.04 LTS	0.1.48	gemma:7b	6.69
Linux	AMD Ryzen 7 7840HS w/ Radeon 780M Graphics	Phoenix1	38.55	Ubuntu 24.04 LTS	0.1.48	llava:7b	8.85
Linux	AMD Ryzen 7 7840HS w/ Radeon 780M Graphics	Phoenix1	38.55	Ubuntu 24.04 LTS	0.1.48	llava:13b	4.9
Linux	Intel(R) Xeon(R) Gold 5320 CPU @ 2.20GHz	SVGA II Adapter	31.34	Ubuntu 22.04.4 LTS	0.1.42	mistral:7b	10.16
Linux	Intel(R) Xeon(R) Gold 5320 CPU @ 2.20GHz	SVGA II Adapter	31.34	Ubuntu 22.04.4 LTS	0.1.42	llama3:8b	9.15
Linux	Intel(R) Xeon(R) Gold 5320 CPU @ 2.20GHz	SVGA II Adapter	31.34	Ubuntu 22.04.4 LTS	0.1.42	phi3:3.8b	16.96
Linux	Intel(R) Xeon(R) Gold 5320 CPU @ 2.20GHz	SVGA II Adapter	31.34	Ubuntu 22.04.4 LTS	0.1.42	gemma:2b	25.1
Linux	Intel(R) Xeon(R) Gold 5320 CPU @ 2.20GHz	SVGA II Adapter	31.34	Ubuntu 22.04.4 LTS	0.1.42	gemma:7b	8.8
Linux	Intel(R) Xeon(R) Gold 5320 CPU @ 2.20GHz	SVGA II Adapter	31.34	Ubuntu 22.04.4 LTS	0.1.42	llava:7b	11.24
Linux	Intel(R) Xeon(R) Gold 5320 CPU @ 2.20GHz	SVGA II Adapter	31.34	Ubuntu 22.04.4 LTS	0.1.42	llava:13b	6.38
Linux	AMD Ryzen 9 3950X 16-Core Processor	Advanced Micro Devices, Inc. [AMD/ATI]	62.70	Linux Mint 21.3	0.1.42	mistral:7b	93.2
Linux	AMD Ryzen 9 3950X 16-Core Processor	Advanced Micro Devices, Inc. [AMD/ATI]	62.70	Linux Mint 21.3	0.1.42	llama3:8b	83.43
Linux	AMD Ryzen 9 3950X 16-Core Processor	Advanced Micro Devices, Inc. [AMD/ATI]	62.70	Linux Mint 21.3	0.1.42	phi3:3.8b	99.71
Linux	AMD Ryzen 9 3950X 16-Core Processor	Advanced Micro Devices, Inc. [AMD/ATI]	62.70	Linux Mint 21.3	0.1.42	gemma:2b	170.03
Linux	AMD Ryzen 9 3950X 16-Core Processor	Advanced Micro Devices, Inc. [AMD/ATI]	62.70	Linux Mint 21.3	0.1.42	gemma:7b	80.85
Linux	AMD Ryzen 9 3950X 16-Core Processor	Advanced Micro Devices, Inc. [AMD/ATI]	62.70	Linux Mint 21.3	0.1.42	llava:7b	95.98
Linux	AMD Ryzen 9 3950X 16-Core Processor	Advanced Micro Devices, Inc. [AMD/ATI]	62.70	Linux Mint 21.3	0.1.42	llava:13b	64.1
Linux	AMD Ryzen 7 5700G with Radeon Graphics	Cezanne	29.16	Ubuntu 22.04.4 LTS	0.2.1	mistral:7b	10.12
Linux	AMD Ryzen 7 5700G with Radeon Graphics	Cezanne	29.16	Ubuntu 22.04.4 LTS	0.2.1	llama3:8b	9.25
Linux	AMD Ryzen 7 5700G with Radeon Graphics	Cezanne	29.16	Ubuntu 22.04.4 LTS	0.2.1	phi3:3.8b	16.2
Linux	AMD Ryzen 7 5700G with Radeon Graphics	Cezanne	29.16	Ubuntu 22.04.4 LTS	0.2.1	gemma:2b	22.31
Linux	AMD Ryzen 7 5700G with Radeon Graphics	Cezanne	29.16	Ubuntu 22.04.4 LTS	0.2.1	gemma:7b	7.5
Linux	AMD Ryzen 7 5700G with Radeon Graphics	Cezanne	29.16	Ubuntu 22.04.4 LTS	0.2.1	llava:7b	10.43
Linux	AMD Ryzen 7 5700G with Radeon Graphics	Cezanne	29.16	Ubuntu 22.04.4 LTS	0.2.1	llava:13b	5.59
Linux	Intel(R) Core(TM) i5-8500T CPU @ 2.10GHz	NVIDIA Quadro P1000	28.20	Debian GNU/Linux 12 (bookworm)	0.1.45	llava:13b	3.69
Linux	Intel(R) Core(TM) i5-14500	NVIDIA GeForce RTX 4060 Ti	62.54	Ubuntu 24.04 LTS	0.1.44	mistral:7b	53.81
Linux	Intel(R) Core(TM) i5-14500	NVIDIA GeForce RTX 4060 Ti	62.54	Ubuntu 24.04 LTS	0.1.44	llama3:8b	48.5
Linux	Intel(R) Core(TM) i5-14500	NVIDIA GeForce RTX 4060 Ti	62.54	Ubuntu 24.04 LTS	0.1.44	phi3:3.8b	67.7
Linux	Intel(R) Core(TM) i5-14500	NVIDIA GeForce RTX 4060 Ti	62.54	Ubuntu 24.04 LTS	0.1.44	gemma:2b	99.66
Linux	Intel(R) Core(TM) i5-14500	NVIDIA GeForce RTX 4060 Ti	62.54	Ubuntu 24.04 LTS	0.1.44	gemma:7b	42.14
Linux	Intel(R) Core(TM) i5-14500	NVIDIA GeForce RTX 4060 Ti	62.54	Ubuntu 24.04 LTS	0.1.44	llava:7b	58.02
Linux	Intel(R) Core(TM) i5-14500	NVIDIA GeForce RTX 4060 Ti	62.54	Ubuntu 24.04 LTS	0.1.44	llava:13b	9.34
Linux	AMD Ryzen 7 5700G with Radeon Graphics	NVIDIA CMP 50HX	30.63	Ubuntu 22.04.4 LTS	0.2.1	llava:13b	25.24
Linux	AMD Ryzen 5 5600X 6-Core Processor	NVIDIA GeForce GTX 1080 Ti	15.54	Pop!_OS 22.04 LTS	0.2.1	mistral:7b	60.95
Linux	AMD Ryzen 5 5600X 6-Core Processor	NVIDIA GeForce GTX 1080 Ti	15.54	Pop!_OS 22.04 LTS	0.2.1	llama3:8b	54.39
Linux	13th Gen Intel(R) Core(TM) i9-13900K	NVIDIA GeForce RTX 4090	125.56	Ubuntu 20.04.6 LTS	0.1.48	llama3:8b-instruct-q2_K	150.36
Linux	13th Gen Intel(R) Core(TM) i9-13900K	NVIDIA GeForce RTX 4090	125.56	Ubuntu 20.04.6 LTS	0.1.48	llama3:8b-instruct-q4_0	124.87
Linux	13th Gen Intel(R) Core(TM) i9-13900K	NVIDIA GeForce RTX 4090	125.56	Ubuntu 20.04.6 LTS	0.1.48	gemma2:9b-instruct-q2_K	113.99
Linux	13th Gen Intel(R) Core(TM) i9-13900K	NVIDIA GeForce RTX 4090	125.56	Ubuntu 20.04.6 LTS	0.1.48	gemma2:9b-instruct-q4_0	96.58
Linux	13th Gen Intel(R) Core(TM) i9-13900K	NVIDIA GeForce RTX 4090	125.56	Ubuntu 20.04.6 LTS	0.1.48	llama3:8b-instruct-q2_K	146.27
Linux	13th Gen Intel(R) Core(TM) i9-13900K	NVIDIA GeForce RTX 4090	125.56	Ubuntu 20.04.6 LTS	0.1.48	llama3:8b-instruct-q4_0	122.07
Linux	13th Gen Intel(R) Core(TM) i9-13900K	NVIDIA GeForce RTX 4090	125.56	Ubuntu 20.04.6 LTS	0.1.48	gemma2:9b-instruct-q2_K	111.14
Linux	13th Gen Intel(R) Core(TM) i9-13900K	NVIDIA GeForce RTX 4090	125.56	Ubuntu 20.04.6 LTS	0.1.48	gemma2:9b-instruct-q4_0	93.14
Linux	13th Gen Intel(R) Core(TM) i9-13900K	NVIDIA GeForce RTX 4090	125.56	Ubuntu 20.04.6 LTS	0.1.48	llama3:8b-instruct-fp16	52.69
Linux	13th Gen Intel(R) Core(TM) i9-13900K	NVIDIA GeForce RTX 4090	125.56	Ubuntu 20.04.6 LTS	0.1.48	gemma2:9b-instruct-fp16	41.15
Linux	AMD EPYC 7532 32-Core Processor	NVIDIA GeForce RTX 3070 Ti	125.71	Ubuntu 22.04.4 LTS	0.1.48	llama3:8b	74.91
Linux	AMD EPYC 7532 32-Core Processor	NVIDIA GeForce RTX 3070 Ti	125.71	Ubuntu 22.04.4 LTS	0.1.48	phi3:3.8b	122.9
Linux	AMD EPYC 7532 32-Core Processor	NVIDIA GeForce RTX 3070 Ti	125.71	Ubuntu 22.04.4 LTS	0.1.48	phi:2.7b	174.09
Linux	AMD Ryzen 9 5900X 12-Core Processor	NVIDIA GeForce RTX 3080	31.28	Ubuntu 20.04.6 LTS	0.1.48	mistral:7b	108.22
Linux	AMD Ryzen 9 5900X 12-Core Processor	NVIDIA GeForce RTX 3080	31.28	Ubuntu 20.04.6 LTS	0.1.48	llama3:8b	96.28
Linux	AMD Ryzen 9 5900X 12-Core Processor	NVIDIA GeForce RTX 3080	31.28	Ubuntu 20.04.6 LTS	0.1.48	phi3:3.8b	149.48
Linux	AMD Ryzen 9 5900X 12-Core Processor	NVIDIA GeForce RTX 3080	31.28	Ubuntu 20.04.6 LTS	0.1.48	gemma:2b	185.18
Linux	AMD Ryzen 9 5900X 12-Core Processor	NVIDIA GeForce RTX 3080	31.28	Ubuntu 20.04.6 LTS	0.1.48	gemma:7b	91.17
Linux	AMD Ryzen 9 5900X 12-Core Processor	NVIDIA GeForce RTX 3080	31.28	Ubuntu 20.04.6 LTS	0.1.48	llava:7b	113.07
Linux	AMD Ryzen 9 5900X 12-Core Processor	NVIDIA GeForce RTX 3080	31.28	Ubuntu 20.04.6 LTS	0.1.48	llava:13b	39.57
Linux	(None)	NVIDIA RTX A6000	15.33	Ubuntu 22.04.3 LTS	0.1.44	mistral:7b	100.8
Linux	(None)	NVIDIA RTX A6000	15.33	Ubuntu 22.04.3 LTS	0.1.44	llama3:8b	81.52
Linux	(None)	NVIDIA RTX A6000	15.33	Ubuntu 22.04.3 LTS	0.1.44	phi3:3.8b	139.91
Linux	(None)	NVIDIA RTX A6000	15.33	Ubuntu 22.04.3 LTS	0.1.44	gemma:2b	136.79
Linux	(None)	NVIDIA RTX A6000	15.33	Ubuntu 22.04.3 LTS	0.1.44	gemma:7b	77.08
Linux	(None)	NVIDIA RTX A6000	15.33	Ubuntu 22.04.3 LTS	0.1.44	llava:7b	104.08
Linux	(None)	NVIDIA RTX A6000	15.33	Ubuntu 22.04.3 LTS	0.1.44	llava:13b	66.78
Linux	(None)	NVIDIA RTX A6000	15.33	Ubuntu 22.04.3 LTS	0.1.44	mistral:7b	100.74
Linux	(None)	NVIDIA RTX A6000	15.33	Ubuntu 22.04.3 LTS	0.1.44	llama3:8b	81.63
Linux	(None)	NVIDIA RTX A6000	15.33	Ubuntu 22.04.3 LTS	0.1.44	phi3:3.8b	139.33
Linux	(None)	NVIDIA RTX A6000	15.33	Ubuntu 22.04.3 LTS	0.1.44	gemma:2b	136.74
Linux	(None)	NVIDIA RTX A6000	15.33	Ubuntu 22.04.3 LTS	0.1.44	gemma:7b	76.8
Linux	(None)	NVIDIA RTX A6000	15.33	Ubuntu 22.04.3 LTS	0.1.44	llava:7b	104.39
Linux	(None)	NVIDIA RTX A6000	15.33	Ubuntu 22.04.3 LTS	0.1.44	llava:13b	66.82
Linux	(None)	NVIDIA RTX A6000	15.33	Ubuntu 22.04.3 LTS	0.1.44	mistral:7b	100.6
Linux	(None)	NVIDIA RTX A6000	15.33	Ubuntu 22.04.3 LTS	0.1.44	llama3:8b	81.41
Linux	(None)	NVIDIA RTX A6000	15.33	Ubuntu 22.04.3 LTS	0.1.44	phi3:3.8b	134.74
Linux	(None)	NVIDIA RTX A6000	15.33	Ubuntu 22.04.3 LTS	0.1.44	gemma:2b	137.84
Linux	(None)	NVIDIA RTX A6000	15.33	Ubuntu 22.04.3 LTS	0.1.44	gemma:7b	76.95
Linux	(None)	NVIDIA RTX A6000	15.33	Ubuntu 22.04.3 LTS	0.1.44	llava:7b	104.29
Linux	(None)	NVIDIA RTX A6000	15.33	Ubuntu 22.04.3 LTS	0.1.44	llava:13b	66.38
Linux	AMD Ryzen 9 5900X 12-Core Processor	NVIDIA GeForce RTX 3080	31.28	Ubuntu 20.04.6 LTS	0.1.48	mistral:7b	108.54
Linux	AMD Ryzen 9 5900X 12-Core Processor	NVIDIA GeForce RTX 3080	31.28	Ubuntu 20.04.6 LTS	0.1.48	llama3:8b	96.44
Linux	AMD Ryzen 9 5900X 12-Core Processor	NVIDIA GeForce RTX 3080	31.28	Ubuntu 20.04.6 LTS	0.1.48	phi3:3.8b	149.33
Linux	AMD Ryzen 9 5900X 12-Core Processor	NVIDIA GeForce RTX 3080	31.28	Ubuntu 20.04.6 LTS	0.1.48	gemma:2b	182.16
Linux	AMD Ryzen 9 5900X 12-Core Processor	NVIDIA GeForce RTX 3080	31.28	Ubuntu 20.04.6 LTS	0.1.48	gemma:7b	90.7
Linux	AMD Ryzen 9 5900X 12-Core Processor	NVIDIA GeForce RTX 3080	31.28	Ubuntu 20.04.6 LTS	0.1.48	llava:7b	112.21
Linux	AMD Ryzen 9 5900X 12-Core Processor	NVIDIA GeForce RTX 3080	31.28	Ubuntu 20.04.6 LTS	0.1.48	llava:13b	39.61
Linux	AMD Ryzen 9 5900X 12-Core Processor	NVIDIA GeForce RTX 3080	31.28	Ubuntu 20.04.6 LTS	0.1.48	mistral:7b	108.31
Linux	AMD Ryzen 9 5900X 12-Core Processor	NVIDIA GeForce RTX 3080	31.28	Ubuntu 20.04.6 LTS	0.1.48	llama3:8b	95.53
Linux	AMD Ryzen 9 5900X 12-Core Processor	NVIDIA GeForce RTX 3080	31.28	Ubuntu 20.04.6 LTS	0.1.48	phi3:3.8b	149.12
Linux	AMD Ryzen 9 5900X 12-Core Processor	NVIDIA GeForce RTX 3080	31.28	Ubuntu 20.04.6 LTS	0.1.48	gemma:2b	183.05
Linux	AMD Ryzen 9 5900X 12-Core Processor	NVIDIA GeForce RTX 3080	31.28	Ubuntu 20.04.6 LTS	0.1.48	gemma:7b	90.64
Linux	AMD Ryzen 9 5900X 12-Core Processor	NVIDIA GeForce RTX 3080	31.28	Ubuntu 20.04.6 LTS	0.1.48	llava:7b	113.5
Linux	AMD Ryzen 9 5900X 12-Core Processor	NVIDIA GeForce RTX 3080	31.28	Ubuntu 20.04.6 LTS	0.1.48	llava:13b	39.62
Linux	AMD EPYC 7742 64-Core Processor	NVIDIA GeForce RTX 4090	503.75	Ubuntu 22.04.3 LTS	0.1.48	mistral:7b	130.41
Linux	AMD EPYC 7742 64-Core Processor	NVIDIA GeForce RTX 4090	503.75	Ubuntu 22.04.3 LTS	0.1.48	llama3:8b	103.25
Linux	AMD EPYC 7742 64-Core Processor	NVIDIA GeForce RTX 4090	503.75	Ubuntu 22.04.3 LTS	0.1.48	phi3:3.8b	175.14
Linux	AMD EPYC 7742 64-Core Processor	NVIDIA GeForce RTX 4090	503.75	Ubuntu 22.04.3 LTS	0.1.48	gemma:2b	181.41
Linux	AMD EPYC 7742 64-Core Processor	NVIDIA GeForce RTX 4090	503.75	Ubuntu 22.04.3 LTS	0.1.48	gemma:7b	101.37
Linux	AMD EPYC 7742 64-Core Processor	NVIDIA GeForce RTX 4090	503.75	Ubuntu 22.04.3 LTS	0.1.48	llava:7b	134.13
Linux	AMD EPYC 7742 64-Core Processor	NVIDIA GeForce RTX 4090	503.75	Ubuntu 22.04.3 LTS	0.1.48	llava:13b	87.24
Linux	AMD EPYC 7742 64-Core Processor	NVIDIA GeForce RTX 4090	503.75	Ubuntu 22.04.3 LTS	0.1.48	mistral:7b	130.07
Linux	AMD EPYC 7742 64-Core Processor	NVIDIA GeForce RTX 4090	503.75	Ubuntu 22.04.3 LTS	0.1.48	llama3:8b	103.68
Linux	AMD EPYC 7742 64-Core Processor	NVIDIA GeForce RTX 4090	503.75	Ubuntu 22.04.3 LTS	0.1.48	phi3:3.8b	175.11
Linux	AMD EPYC 7742 64-Core Processor	NVIDIA GeForce RTX 4090	503.75	Ubuntu 22.04.3 LTS	0.1.48	gemma:2b	184.2
Linux	AMD EPYC 7742 64-Core Processor	NVIDIA GeForce RTX 4090	503.75	Ubuntu 22.04.3 LTS	0.1.48	gemma:7b	101.62
Linux	AMD EPYC 7742 64-Core Processor	NVIDIA GeForce RTX 4090	503.75	Ubuntu 22.04.3 LTS	0.1.48	llava:7b	133.73
Linux	AMD EPYC 7742 64-Core Processor	NVIDIA GeForce RTX 4090	503.75	Ubuntu 22.04.3 LTS	0.1.48	llava:13b	87.24
Linux	AMD EPYC 7742 64-Core Processor	NVIDIA GeForce RTX 4090	503.75	Ubuntu 22.04.3 LTS	0.1.48	gemma:2b	179.74
Linux	Intel(R) Xeon(R) Gold 6448Y	NVIDIA H100 80GB HBM3	1007.38	Ubuntu 22.04.3 LTS	0.1.48	mistral:7b	136.05
Linux	Intel(R) Xeon(R) Gold 6448Y	NVIDIA H100 80GB HBM3	1007.38	Ubuntu 22.04.3 LTS	0.1.48	llama3:8b	116.52
Linux	Intel(R) Xeon(R) Gold 6448Y	NVIDIA H100 80GB HBM3	1007.38	Ubuntu 22.04.3 LTS	0.1.48	phi3:3.8b	152.08
Linux	Intel(R) Xeon(R) Gold 6448Y	NVIDIA H100 80GB HBM3	1007.38	Ubuntu 22.04.3 LTS	0.1.48	gemma:2b	170.86
Linux	Intel(R) Xeon(R) Gold 6448Y	NVIDIA H100 80GB HBM3	1007.38	Ubuntu 22.04.3 LTS	0.1.48	gemma:7b	115.51
Linux	Intel(R) Xeon(R) Gold 6448Y	NVIDIA H100 80GB HBM3	1007.38	Ubuntu 22.04.3 LTS	0.1.48	llava:7b	155.14
Linux	Intel(R) Xeon(R) Gold 6448Y	NVIDIA H100 80GB HBM3	1007.38	Ubuntu 22.04.3 LTS	0.1.48	llava:13b	112.37
Linux	Intel(R) Xeon(R) Gold 6226R CPU @ 2.90GHz	Tesla V100S-PCIE-32GB	43.05	Ubuntu 22.04.4 LTS	0.1.48	mistral:7b	115.51
Linux	Intel(R) Xeon(R) Gold 6226R CPU @ 2.90GHz	Tesla V100S-PCIE-32GB	43.05	Ubuntu 22.04.4 LTS	0.1.48	llama3:8b	91.7
Linux	Intel(R) Xeon(R) Gold 6226R CPU @ 2.90GHz	Tesla V100S-PCIE-32GB	43.05	Ubuntu 22.04.4 LTS	0.1.48	phi3:3.8b	146.29
Linux	Intel(R) Xeon(R) Gold 6226R CPU @ 2.90GHz	Tesla V100S-PCIE-32GB	43.05	Ubuntu 22.04.4 LTS	0.1.48	gemma:2b	171.82
Linux	Intel(R) Xeon(R) Gold 6226R CPU @ 2.90GHz	Tesla V100S-PCIE-32GB	43.05	Ubuntu 22.04.4 LTS	0.1.48	gemma:7b	94.95
Linux	Intel(R) Xeon(R) Gold 6226R CPU @ 2.90GHz	Tesla V100S-PCIE-32GB	43.05	Ubuntu 22.04.4 LTS	0.1.48	llava:7b	120.33
Linux	Intel(R) Xeon(R) Gold 6226R CPU @ 2.90GHz	Tesla V100S-PCIE-32GB	43.05	Ubuntu 22.04.4 LTS	0.1.48	llava:13b	78.58
Linux	Intel(R) Xeon(R) Silver 4114 CPU @ 2.20GHz	NVIDIA GeForce GTX 1080 Ti	187.56	"Rocky Linux 8.9 (Green Obsidian)"	0.1.48	mistral:7b	56.65
Linux	Intel(R) Xeon(R) Silver 4114 CPU @ 2.20GHz	NVIDIA GeForce GTX 1080 Ti	187.56	"Rocky Linux 8.9 (Green Obsidian)"	0.1.48	llama3:8b	47.3
Linux	Intel(R) Xeon(R) Silver 4114 CPU @ 2.20GHz	NVIDIA GeForce GTX 1080 Ti	187.56	"Rocky Linux 8.9 (Green Obsidian)"	0.1.48	phi3:3.8b	79.16
Linux	Intel(R) Xeon(R) Silver 4114 CPU @ 2.20GHz	NVIDIA GeForce GTX 1080 Ti	187.56	"Rocky Linux 8.9 (Green Obsidian)"	0.1.48	gemma:2b	104.51
Linux	Intel(R) Xeon(R) Silver 4114 CPU @ 2.20GHz	NVIDIA GeForce GTX 1080 Ti	187.56	"Rocky Linux 8.9 (Green Obsidian)"	0.1.48	gemma:7b	45.24
Linux	Intel(R) Xeon(R) Silver 4114 CPU @ 2.20GHz	NVIDIA GeForce GTX 1080 Ti	187.56	"Rocky Linux 8.9 (Green Obsidian)"	0.1.48	llava:7b	59.26
Linux	Intel(R) Xeon(R) Silver 4114 CPU @ 2.20GHz	NVIDIA GeForce GTX 1080 Ti	187.56	"Rocky Linux 8.9 (Green Obsidian)"	0.1.48	llava:13b	34.89
Linux	Intel(R) Core(TM) i9-9960X CPU @ 3.10GHz	NVIDIA GeForce RTX 2080 Ti	62.47	Ubuntu 22.04.4 LTS	0.1.48	mistral:7b	85.7
Linux	Intel(R) Core(TM) i9-9960X CPU @ 3.10GHz	NVIDIA GeForce RTX 2080 Ti	62.47	Ubuntu 22.04.4 LTS	0.1.48	llama3:8b	74.4
Linux	Intel(R) Core(TM) i9-9960X CPU @ 3.10GHz	NVIDIA GeForce RTX 2080 Ti	62.47	Ubuntu 22.04.4 LTS	0.1.48	phi3:3.8b	126.32
Linux	Intel(R) Core(TM) i9-9960X CPU @ 3.10GHz	NVIDIA GeForce RTX 2080 Ti	62.47	Ubuntu 22.04.4 LTS	0.1.48	gemma:2b	154.87
Linux	Intel(R) Core(TM) i9-9960X CPU @ 3.10GHz	NVIDIA GeForce RTX 2080 Ti	62.47	Ubuntu 22.04.4 LTS	0.1.48	gemma:7b	73.1
Linux	Intel(R) Core(TM) i9-9960X CPU @ 3.10GHz	NVIDIA GeForce RTX 2080 Ti	62.47	Ubuntu 22.04.4 LTS	0.1.48	llava:7b	91.02
Linux	Intel(R) Core(TM) i9-9960X CPU @ 3.10GHz	NVIDIA GeForce RTX 2080 Ti	62.47	Ubuntu 22.04.4 LTS	0.1.48	llava:13b	54.84
Linux	Intel(R) Core(TM) i7-8565U CPU @ 1.80GHz	NVIDIA GeForce MX110	19.41	Ubuntu 20.04.6 LTS	0.1.48	mistral:7b	1.73
Linux	Intel(R) Core(TM) i7-8565U CPU @ 1.80GHz	NVIDIA GeForce MX110	19.41	Ubuntu 20.04.6 LTS	0.1.48	llama3:8b	1.77
Linux	Intel(R) Core(TM) i7-8565U CPU @ 1.80GHz	NVIDIA GeForce MX110	19.41	Ubuntu 20.04.6 LTS	0.1.48	phi3:3.8b	2.97
Linux	Intel(R) Core(TM) i7-8565U CPU @ 1.80GHz	NVIDIA GeForce MX110	19.41	Ubuntu 20.04.6 LTS	0.1.48	gemma:2b	4.91
Linux	Intel(R) Core(TM) i7-8565U CPU @ 1.80GHz	NVIDIA GeForce MX110	19.41	Ubuntu 20.04.6 LTS	0.1.48	gemma:7b	2.05
Linux	Intel(R) Core(TM) i7-8565U CPU @ 1.80GHz	NVIDIA GeForce MX110	19.41	Ubuntu 20.04.6 LTS	0.1.48	llava:7b	2.42
Linux	Intel(R) Core(TM) i7-8565U CPU @ 1.80GHz	NVIDIA GeForce MX110	19.41	Ubuntu 20.04.6 LTS	0.1.48	llava:13b	1.25
Linux	(None)	NVIDIA RTX A6000	15.33	Ubuntu 22.04.3 LTS	0.1.44	mistral:7b	100.4
Linux	(None)	NVIDIA RTX A6000	15.33	Ubuntu 22.04.3 LTS	0.1.44	llama3:8b	81.59
Linux	(None)	NVIDIA RTX A6000	15.33	Ubuntu 22.04.3 LTS	0.1.44	phi3:3.8b	140.34
Linux	(None)	NVIDIA RTX A6000	15.33	Ubuntu 22.04.3 LTS	0.1.44	gemma:2b	137.92
Linux	(None)	NVIDIA RTX A6000	15.33	Ubuntu 22.04.3 LTS	0.1.44	gemma:7b	76.97
Linux	(None)	NVIDIA RTX A6000	15.33	Ubuntu 22.04.3 LTS	0.1.44	llava:7b	104.54
Linux	(None)	NVIDIA RTX A6000	15.33	Ubuntu 22.04.3 LTS	0.1.44	llava:13b	66.5
Linux	(None)	NVIDIA RTX A6000	15.33	Ubuntu 22.04.3 LTS	0.1.44	mistral:7b	100.89
Linux	(None)	NVIDIA RTX A6000	15.33	Ubuntu 22.04.3 LTS	0.1.44	llama3:8b	81.45
Linux	(None)	NVIDIA RTX A6000	15.33	Ubuntu 22.04.3 LTS	0.1.44	phi3:3.8b	137.58
Linux	(None)	NVIDIA RTX A6000	15.33	Ubuntu 22.04.3 LTS	0.1.44	gemma:2b	136.24
Linux	(None)	NVIDIA RTX A6000	15.33	Ubuntu 22.04.3 LTS	0.1.44	gemma:7b	77.09
Linux	(None)	NVIDIA RTX A6000	15.33	Ubuntu 22.04.3 LTS	0.1.44	llava:7b	103.85
Linux	(None)	NVIDIA RTX A6000	15.33	Ubuntu 22.04.3 LTS	0.1.44	llava:13b	66.7
Linux	AMD Ryzen 7 5800X 8-Core Processor	NVIDIA GeForce RTX 3090	62.68	Pop!_OS 22.04 LTS	0.1.48	mistral:7b	6.04
Linux	AMD Ryzen 7 5800X 8-Core Processor	NVIDIA GeForce RTX 3090	62.68	Pop!_OS 22.04 LTS	0.1.48	llama3:8b	5.47
Linux	AMD Ryzen 7 5800X 8-Core Processor	NVIDIA GeForce RTX 3090	62.68	Pop!_OS 22.04 LTS	0.1.48	phi3:3.8b	9.32
Linux	AMD Ryzen 7 5800X 8-Core Processor	NVIDIA GeForce RTX 3090	62.68	Pop!_OS 22.04 LTS	0.1.48	gemma:2b	13.88
Linux	AMD Ryzen 7 5800X 8-Core Processor	NVIDIA GeForce RTX 3090	62.68	Pop!_OS 22.04 LTS	0.1.48	gemma:7b	4.9
Linux	AMD Ryzen 7 5800X 8-Core Processor	NVIDIA GeForce RTX 3090	62.68	Pop!_OS 22.04 LTS	0.1.48	llava:7b	6.35
Linux	AMD Ryzen 7 5800X 8-Core Processor	NVIDIA GeForce RTX 3090	62.68	Pop!_OS 22.04 LTS	0.1.48	llava:13b	3.51
Linux	AMD Ryzen 7 5800X 8-Core Processor	NVIDIA GeForce RTX 3090	62.68	Pop!_OS 22.04 LTS	0.1.48	mistral:7b	115.2
Linux	AMD Ryzen 7 5800X 8-Core Processor	NVIDIA GeForce RTX 3090	62.68	Pop!_OS 22.04 LTS	0.1.48	llama3:8b	106.02
Linux	AMD Ryzen 7 5800X 8-Core Processor	NVIDIA GeForce RTX 3090	62.68	Pop!_OS 22.04 LTS	0.1.48	phi3:3.8b	166.22
Linux	AMD Ryzen 7 5800X 8-Core Processor	NVIDIA GeForce RTX 3090	62.68	Pop!_OS 22.04 LTS	0.1.48	gemma:2b	201.57
Linux	AMD Ryzen 7 5800X 8-Core Processor	NVIDIA GeForce RTX 3090	62.68	Pop!_OS 22.04 LTS	0.1.48	gemma:7b	99.86
Linux	AMD Ryzen 7 5800X 8-Core Processor	NVIDIA GeForce RTX 3090	62.68	Pop!_OS 22.04 LTS	0.1.48	llava:7b	123.66
Linux	AMD Ryzen 7 5800X 8-Core Processor	NVIDIA GeForce RTX 3090	62.68	Pop!_OS 22.04 LTS	0.1.48	llava:13b	78.09
Linux	Intel(R) Xeon(R) CPU E5-2698 v3 @ 2.30GHz	NVIDIA RTX A2000 12GB	31.27	Ubuntu 22.04.3 LTS	0.1.48	mistral:7b	43.58
Linux	Intel(R) Xeon(R) CPU E5-2698 v3 @ 2.30GHz	NVIDIA RTX A2000 12GB	31.27	Ubuntu 22.04.3 LTS	0.1.48	llama3:8b	37.86
Linux	Intel(R) Xeon(R) CPU E5-2698 v3 @ 2.30GHz	NVIDIA RTX A2000 12GB	31.27	Ubuntu 22.04.3 LTS	0.1.48	phi3:3.8b	64.95
Linux	Intel(R) Xeon(R) CPU E5-2698 v3 @ 2.30GHz	NVIDIA RTX A2000 12GB	31.27	Ubuntu 22.04.3 LTS	0.1.48	gemma:2b	78.23
Linux	Intel(R) Xeon(R) CPU E5-2698 v3 @ 2.30GHz	NVIDIA RTX A2000 12GB	31.27	Ubuntu 22.04.3 LTS	0.1.48	gemma:7b	36.11
Linux	Intel(R) Xeon(R) CPU E5-2698 v3 @ 2.30GHz	NVIDIA RTX A2000 12GB	31.27	Ubuntu 22.04.3 LTS	0.1.48	llava:7b	36.18
Linux	Intel(R) Xeon(R) CPU E5-2698 v3 @ 2.30GHz	NVIDIA RTX A2000 12GB	31.27	Ubuntu 22.04.3 LTS	0.1.48	llava:13b	23.12
Linux	Intel(R) Xeon(R) CPU E5-2650L v4 @ 1.70GHz	NVIDIA RTX 4000 SFF Ada Generation	62.79	Ubuntu 22.04.4 LTS	0.1.45	mistral:7b	48.31
Linux	Intel(R) Xeon(R) CPU E5-2650L v4 @ 1.70GHz	NVIDIA RTX 4000 SFF Ada Generation	62.79	Ubuntu 22.04.4 LTS	0.1.45	llama3:8b	39.38
Linux	Intel(R) Xeon(R) CPU E5-2650L v4 @ 1.70GHz	NVIDIA RTX 4000 SFF Ada Generation	62.79	Ubuntu 22.04.4 LTS	0.1.45	phi3:3.8b	72.44
Linux	Intel(R) Xeon(R) CPU E5-2650L v4 @ 1.70GHz	NVIDIA RTX 4000 SFF Ada Generation	62.79	Ubuntu 22.04.4 LTS	0.1.45	gemma:2b	82.93
Linux	Intel(R) Xeon(R) CPU E5-2650L v4 @ 1.70GHz	NVIDIA RTX 4000 SFF Ada Generation	62.79	Ubuntu 22.04.4 LTS	0.1.45	gemma:7b	38.11
Linux	Intel(R) Xeon(R) CPU E5-2650L v4 @ 1.70GHz	NVIDIA RTX 4000 SFF Ada Generation	62.79	Ubuntu 22.04.4 LTS	0.1.45	llava:7b	49.3
Linux	Intel(R) Xeon(R) CPU E5-2650L v4 @ 1.70GHz	NVIDIA RTX 4000 SFF Ada Generation	62.79	Ubuntu 22.04.4 LTS	0.1.45	llava:13b	29.56
Linux	unknown	NVIDIA A100 80GB PCIe	1021.17	Ubuntu 22.04.4 LTS	0.2.1	mistral:7b	115.37
Linux	unknown	NVIDIA A100 80GB PCIe	1021.17	Ubuntu 22.04.4 LTS	0.2.1	llama3:8b	88.93
Linux	unknown	NVIDIA A100 80GB PCIe	1021.17	Ubuntu 22.04.4 LTS	0.2.1	phi3:3.8b	137.04
Linux	unknown	NVIDIA A100 80GB PCIe	1021.17	Ubuntu 22.04.4 LTS	0.2.1	gemma2:9b	54.96
Linux	unknown	NVIDIA A100 80GB PCIe	1021.17	Ubuntu 22.04.4 LTS	0.2.1	gemma2:27b	46.29
Linux	unknown	NVIDIA A100 80GB PCIe	1021.17	Ubuntu 22.04.4 LTS	0.2.1	llava:7b	120.03
Linux	unknown	NVIDIA A100 80GB PCIe	1021.17	Ubuntu 22.04.4 LTS	0.2.1	llava:13b	90.06
Linux	unknown	NVIDIA A100 80GB PCIe	1021.17	Ubuntu 22.04.4 LTS	0.2.1	mistral:7b	117.66
Linux	unknown	NVIDIA A100 80GB PCIe	1021.17	Ubuntu 22.04.4 LTS	0.2.1	llama3:8b	88.99
Linux	unknown	NVIDIA A100 80GB PCIe	1021.17	Ubuntu 22.04.4 LTS	0.2.1	phi3:3.8b	136.55
Linux	unknown	NVIDIA A100 80GB PCIe	1021.17	Ubuntu 22.04.4 LTS	0.2.1	gemma2:9b	55.31
Linux	unknown	NVIDIA A100 80GB PCIe	1021.17	Ubuntu 22.04.4 LTS	0.2.1	gemma2:27b	37.97
Linux	unknown	NVIDIA A100 80GB PCIe	1021.17	Ubuntu 22.04.4 LTS	0.2.1	llava:7b	128.92
Linux	unknown	NVIDIA A100 80GB PCIe	1021.17	Ubuntu 22.04.4 LTS	0.2.1	llava:13b	85.05
Linux	AMD EPYC 7R13 Processor	NVIDIA L4	181.77	"Amazon Linux 2"	0.2.1	mistral:7b	51.54
Linux	AMD EPYC 7R13 Processor	NVIDIA L4	181.77	"Amazon Linux 2"	0.2.1	llama3:8b	46.57
Linux	AMD EPYC 7R13 Processor	NVIDIA L4	181.77	"Amazon Linux 2"	0.2.1	phi3:3.8b	81.3
Linux	AMD EPYC 7R13 Processor	NVIDIA L4	181.77	"Amazon Linux 2"	0.2.1	gemma:2b	103.27
Linux	AMD EPYC 7R13 Processor	NVIDIA L4	181.77	"Amazon Linux 2"	0.2.1	gemma:7b	41.86
Linux	AMD EPYC 7R13 Processor	NVIDIA L4	181.77	"Amazon Linux 2"	0.2.1	llava:7b	53.21
Linux	AMD EPYC 7R13 Processor	NVIDIA L4	181.77	"Amazon Linux 2"	0.2.1	llava:13b	30.46
Linux	AMD Ryzen 5 5600X 6-Core Processor	NVIDIA GeForce GTX 1080 Ti	15.54	Pop!_OS 22.04 LTS	0.2.1	mistral:7b	56.9
Linux	AMD Ryzen 5 5600X 6-Core Processor	NVIDIA GeForce GTX 1080 Ti	15.54	Pop!_OS 22.04 LTS	0.2.1	llama3:8b	51.12
Linux	AMD Ryzen 5 5600X 6-Core Processor	NVIDIA GeForce GTX 1080 Ti	15.54	Pop!_OS 22.04 LTS	0.2.1	phi3:3.8b	82.85
Linux	AMD Ryzen 5 5600X 6-Core Processor	NVIDIA GeForce GTX 1080 Ti	15.54	Pop!_OS 22.04 LTS	0.2.1	gemma:2b	113.54
Linux	AMD Ryzen 5 5600X 6-Core Processor	NVIDIA GeForce GTX 1080 Ti	15.54	Pop!_OS 22.04 LTS	0.2.1	gemma:7b	45.8
Linux	AMD Ryzen 5 5600X 6-Core Processor	NVIDIA GeForce GTX 1080 Ti	15.54	Pop!_OS 22.04 LTS	0.2.1	llava:7b	60.99
Linux	AMD Ryzen 5 5600X 6-Core Processor	NVIDIA GeForce GTX 1080 Ti	15.54	Pop!_OS 22.04 LTS	0.2.1	llava:13b	35.47
Linux	AMD Ryzen 9 3950X 16-Core Processor	Advanced Micro Devices, Inc. [AMD/ATI]	62.70	Linux Mint 21.3	0.1.42	mistral:7b	93.17
Linux	AMD Ryzen 9 3950X 16-Core Processor	Advanced Micro Devices, Inc. [AMD/ATI]	62.70	Linux Mint 21.3	0.1.42	llama3:8b	83.21
Linux	AMD Ryzen 9 3950X 16-Core Processor	Advanced Micro Devices, Inc. [AMD/ATI]	62.70	Linux Mint 21.3	0.1.42	phi3:3.8b	101.14
Linux	AMD Ryzen 9 3950X 16-Core Processor	Advanced Micro Devices, Inc. [AMD/ATI]	62.70	Linux Mint 21.3	0.1.42	gemma:2b	168.75
Linux	AMD Ryzen 9 3950X 16-Core Processor	Advanced Micro Devices, Inc. [AMD/ATI]	62.70	Linux Mint 21.3	0.1.42	gemma:7b	80.78
Linux	AMD Ryzen 9 3950X 16-Core Processor	Advanced Micro Devices, Inc. [AMD/ATI]	62.70	Linux Mint 21.3	0.1.42	llava:7b	95.44
Linux	AMD Ryzen 9 3950X 16-Core Processor	Advanced Micro Devices, Inc. [AMD/ATI]	62.70	Linux Mint 21.3	0.1.42	llava:13b	64.12
Linux	13th Gen Intel(R) Core(TM) i5-13500	NVIDIA RTX 4000 SFF Ada Generation	62.60	Ubuntu 22.04.4 LTS	0.2.1	mistral:7b	52.8
Linux	13th Gen Intel(R) Core(TM) i5-13500	NVIDIA RTX 4000 SFF Ada Generation	62.60	Ubuntu 22.04.4 LTS	0.2.1	llama3:8b	47.45
Linux	13th Gen Intel(R) Core(TM) i5-13500	NVIDIA RTX 4000 SFF Ada Generation	62.60	Ubuntu 22.04.4 LTS	0.2.1	phi3:3.8b	82.6
Linux	13th Gen Intel(R) Core(TM) i5-13500	NVIDIA RTX 4000 SFF Ada Generation	62.60	Ubuntu 22.04.4 LTS	0.2.1	gemma:2b	105.75
Linux	13th Gen Intel(R) Core(TM) i5-13500	NVIDIA RTX 4000 SFF Ada Generation	62.60	Ubuntu 22.04.4 LTS	0.2.1	gemma:7b	42.56
Linux	13th Gen Intel(R) Core(TM) i5-13500	NVIDIA RTX 4000 SFF Ada Generation	62.60	Ubuntu 22.04.4 LTS	0.2.1	llava:7b	54.03
Linux	13th Gen Intel(R) Core(TM) i5-13500	NVIDIA RTX 4000 SFF Ada Generation	62.60	Ubuntu 22.04.4 LTS	0.2.1	llava:13b	31.39
Linux	AMD Ryzen 9 7950X3D 16-Core Processor	NVIDIA RTX A6000	62.50	Ubuntu 24.04 LTS	0.2.1	mistral:7b	109.77
Linux	AMD Ryzen 9 7950X3D 16-Core Processor	NVIDIA RTX A6000	62.50	Ubuntu 24.04 LTS	0.2.1	llama3:8b	96.65
Linux	AMD Ryzen 9 7950X3D 16-Core Processor	NVIDIA RTX A6000	62.50	Ubuntu 24.04 LTS	0.2.1	phi3:3.8b	156.56
Linux	AMD Ryzen 9 7950X3D 16-Core Processor	NVIDIA RTX A6000	62.50	Ubuntu 24.04 LTS	0.2.1	gemma:2b	188.69
Linux	AMD Ryzen 9 7950X3D 16-Core Processor	NVIDIA RTX A6000	62.50	Ubuntu 24.04 LTS	0.2.1	gemma:7b	90.81
Linux	AMD Ryzen 9 7950X3D 16-Core Processor	NVIDIA RTX A6000	62.50	Ubuntu 24.04 LTS	0.2.1	llava:7b	112.26
Linux	AMD Ryzen 9 7950X3D 16-Core Processor	NVIDIA RTX A6000	62.50	Ubuntu 24.04 LTS	0.2.1	llava:13b	69.9
Linux	AMD Ryzen 7 5700G with Radeon Graphics	NVIDIA CMP 50HX	29.16	Ubuntu 22.04.4 LTS	0.2.1	mistral:7b	37.24
Linux	AMD Ryzen 7 5700G with Radeon Graphics	NVIDIA CMP 50HX	29.16	Ubuntu 22.04.4 LTS	0.2.1	llama3:8b	34.43
Linux	AMD Ryzen 7 5700G with Radeon Graphics	NVIDIA CMP 50HX	29.16	Ubuntu 22.04.4 LTS	0.2.1	phi3:3.8b	57.57
Linux	AMD Ryzen 7 5700G with Radeon Graphics	NVIDIA CMP 50HX	29.16	Ubuntu 22.04.4 LTS	0.2.1	gemma:2b	72.1
Linux	AMD Ryzen 7 5700G with Radeon Graphics	NVIDIA CMP 50HX	29.16	Ubuntu 22.04.4 LTS	0.2.1	gemma:7b	35.17
Linux	AMD Ryzen 7 5700G with Radeon Graphics	NVIDIA CMP 50HX	29.16	Ubuntu 22.04.4 LTS	0.2.1	llava:7b	49.32
Linux	AMD Ryzen 7 5700G with Radeon Graphics	NVIDIA CMP 50HX	29.16	Ubuntu 22.04.4 LTS	0.2.1	llava:13b	25.48
Linux	AMD Ryzen 7 5700G with Radeon Graphics	NVIDIA CMP 50HX	30.63	Ubuntu 22.04.4 LTS	0.2.1	mistral:7b	36.9
Linux	AMD Ryzen 7 5700G with Radeon Graphics	NVIDIA CMP 50HX	30.63	Ubuntu 22.04.4 LTS	0.2.1	llama3:8b	34.42
Linux	AMD Ryzen 7 5700G with Radeon Graphics	NVIDIA CMP 50HX	30.63	Ubuntu 22.04.4 LTS	0.2.1	phi3:3.8b	51.05
Linux	AMD Ryzen 7 5700G with Radeon Graphics	NVIDIA CMP 50HX	30.63	Ubuntu 22.04.4 LTS	0.2.1	gemma:2b	73.87
Linux	AMD Ryzen 7 5700G with Radeon Graphics	NVIDIA CMP 50HX	30.63	Ubuntu 22.04.4 LTS	0.2.1	gemma:7b	34.82
Linux	AMD Ryzen 7 5700G with Radeon Graphics	NVIDIA CMP 50HX	30.63	Ubuntu 22.04.4 LTS	0.2.1	llava:7b	50.51
Linux	AMD Ryzen 5 5600X 6-Core Processor	NVIDIA GeForce GTX 1080 Ti	15.54	Pop!_OS 22.04 LTS	0.2.1	phi3:3.8b	89.23
Linux	AMD Ryzen 5 5600X 6-Core Processor	NVIDIA GeForce GTX 1080 Ti	15.54	Pop!_OS 22.04 LTS	0.2.1	gemma:2b	120.94
Linux	AMD Ryzen 5 5600X 6-Core Processor	NVIDIA GeForce GTX 1080 Ti	15.54	Pop!_OS 22.04 LTS	0.2.1	gemma:7b	49.1
Linux	AMD Ryzen 5 5600X 6-Core Processor	NVIDIA GeForce GTX 1080 Ti	15.54	Pop!_OS 22.04 LTS	0.2.1	llava:7b	64.9
Linux	AMD Ryzen 5 5600X 6-Core Processor	NVIDIA GeForce GTX 1080 Ti	15.54	Pop!_OS 22.04 LTS	0.2.1	llava:13b	37.6
Linux	AMD Ryzen 5 5600X 6-Core Processor	NVIDIA GeForce GTX 1080 Ti	15.54	Pop!_OS 22.04 LTS	0.2.1	mistral:7b	63.76
Linux	AMD Ryzen 5 5600X 6-Core Processor	NVIDIA GeForce GTX 1080 Ti	15.54	Pop!_OS 22.04 LTS	0.2.1	llama3:8b	53.84
Linux	AMD Ryzen 5 5600X 6-Core Processor	NVIDIA GeForce GTX 1080 Ti	15.54	Pop!_OS 22.04 LTS	0.2.1	phi3:3.8b	86.94
Linux	AMD Ryzen 5 5600X 6-Core Processor	NVIDIA GeForce GTX 1080 Ti	15.54	Pop!_OS 22.04 LTS	0.2.1	gemma:2b	121.26
Linux	AMD Ryzen 5 5600X 6-Core Processor	NVIDIA GeForce GTX 1080 Ti	15.54	Pop!_OS 22.04 LTS	0.2.1	gemma:7b	46.79
Linux	AMD Ryzen 5 5600X 6-Core Processor	NVIDIA GeForce GTX 1080 Ti	15.54	Pop!_OS 22.04 LTS	0.2.1	llava:7b	64.01
Linux	AMD Ryzen 5 5600X 6-Core Processor	NVIDIA GeForce GTX 1080 Ti	15.54	Pop!_OS 22.04 LTS	0.2.1	llava:13b	37.39
Linux	Intel(R) Core(TM) i5-8500T CPU @ 2.10GHz	unknown	47.05	Debian GNU/Linux 12 (bookworm)	0.1.45	mistral:7b	6.7
Linux	Intel(R) Core(TM) i5-8500T CPU @ 2.10GHz	unknown	47.05	Debian GNU/Linux 12 (bookworm)	0.1.45	llama3:8b	6.22
Linux	Intel(R) Core(TM) i5-8500T CPU @ 2.10GHz	unknown	47.05	Debian GNU/Linux 12 (bookworm)	0.1.45	phi3:3.8b	11.6
Linux	Intel(R) Core(TM) i5-8500T CPU @ 2.10GHz	unknown	47.05	Debian GNU/Linux 12 (bookworm)	0.1.45	gemma:2b	16.14
Linux	Intel(R) Core(TM) i5-8500T CPU @ 2.10GHz	unknown	47.05	Debian GNU/Linux 12 (bookworm)	0.1.45	gemma:7b	5.48
Linux	Intel(R) Core(TM) i5-8500T CPU @ 2.10GHz	unknown	47.05	Debian GNU/Linux 12 (bookworm)	0.1.45	llava:7b	7.08
Linux	Intel(R) Core(TM) i5-8500T CPU @ 2.10GHz	unknown	47.05	Debian GNU/Linux 12 (bookworm)	0.1.45	llava:13b	3.89
Linux	Intel(R) Core(TM) i5-8500T CPU @ 2.10GHz	unknown	47.05	Debian GNU/Linux 12 (bookworm)	0.1.45	mistral:7b	6.75
Linux	Intel(R) Core(TM) i5-8500T CPU @ 2.10GHz	unknown	47.05	Debian GNU/Linux 12 (bookworm)	0.1.45	llama3:8b	5.89
Linux	Intel(R) Core(TM) i5-8500T CPU @ 2.10GHz	unknown	47.05	Debian GNU/Linux 12 (bookworm)	0.1.45	phi3:3.8b	11.13
Linux	Intel(R) Core(TM) i5-8500T CPU @ 2.10GHz	unknown	47.05	Debian GNU/Linux 12 (bookworm)	0.1.45	gemma:2b	15.78
Linux	Intel(R) Core(TM) i5-8500T CPU @ 2.10GHz	unknown	47.05	Debian GNU/Linux 12 (bookworm)	0.1.45	gemma:7b	5.48
Linux	Intel(R) Core(TM) i5-8500T CPU @ 2.10GHz	unknown	47.05	Debian GNU/Linux 12 (bookworm)	0.1.45	llava:7b	7
Linux	Intel(R) Core(TM) i5-8500T CPU @ 2.10GHz	unknown	47.05	Debian GNU/Linux 12 (bookworm)	0.1.45	llava:13b	3.86
Linux	AMD Ryzen 7 7840U w/ Radeon 780M Graphics	no_gpu	30.66	"Arch Linux"	0.1.48	mistral:7b	6.06
Linux	AMD Ryzen 7 7840U w/ Radeon 780M Graphics	no_gpu	30.66	"Arch Linux"	0.1.48	llama3:8b	5.65
Linux	AMD Ryzen 7 7840U w/ Radeon 780M Graphics	no_gpu	30.66	"Arch Linux"	0.1.48	phi3:3.8b	10.02
Linux	AMD Ryzen 7 7840U w/ Radeon 780M Graphics	no_gpu	30.66	"Arch Linux"	0.1.48	gemma:2b	13.65
Linux	AMD Ryzen 7 7840U w/ Radeon 780M Graphics	no_gpu	30.66	"Arch Linux"	0.1.48	gemma:7b	4.88
Linux	AMD Ryzen 7 7840U w/ Radeon 780M Graphics	no_gpu	30.66	"Arch Linux"	0.1.48	llava:7b	6.22
Linux	AMD Ryzen 7 7840U w/ Radeon 780M Graphics	no_gpu	30.66	"Arch Linux"	0.1.48	llava:13b	3.45
Linux	AMD Ryzen 5 7640U w/ Radeon 760M Graphics	Phoenix1	60.63	Fedora Linux 40.20240712.0 (Silverblue)	0.2.1	mistral:7b	10.66
Linux	AMD Ryzen 5 7640U w/ Radeon 760M Graphics	Phoenix1	60.63	Fedora Linux 40.20240712.0 (Silverblue)	0.2.1	llama3:8b	9.79
Linux	AMD Ryzen 5 7640U w/ Radeon 760M Graphics	Phoenix1	60.63	Fedora Linux 40.20240712.0 (Silverblue)	0.2.1	phi3:3.8b	15.66
Linux	AMD Ryzen 5 7640U w/ Radeon 760M Graphics	Phoenix1	60.63	Fedora Linux 40.20240712.0 (Silverblue)	0.2.1	gemma:2b	22.24
Linux	AMD Ryzen 5 7640U w/ Radeon 760M Graphics	Phoenix1	60.63	Fedora Linux 40.20240712.0 (Silverblue)	0.2.1	gemma:7b	7.89
Linux	AMD Ryzen 5 7640U w/ Radeon 760M Graphics	Phoenix1	60.63	Fedora Linux 40.20240712.0 (Silverblue)	0.2.1	llava:7b	10.65
Linux	AMD Ryzen 5 7640U w/ Radeon 760M Graphics	Phoenix1	60.63	Fedora Linux 40.20240712.0 (Silverblue)	0.2.1	llava:13b	6.04
Linux	AMD EPYC 7313P 16-Core Processor	NVIDIA RTX 6000 Ada Generation	251.56	Ubuntu 24.04 LTS	0.0.0	llama3:8b	108.09
Linux	AMD EPYC 7313P 16-Core Processor	NVIDIA RTX 6000 Ada Generation	251.56	Ubuntu 24.04 LTS	0.0.0	llama3:8b	108.89
Linux	Intel(R) Xeon(R) CPU E5-2698 v3 @ 2.30GHz	NVIDIA RTX A2000 12GB	31.27	Ubuntu 22.04.3 LTS	0.1.48	mistral:7b	43.56
Linux	Intel(R) Xeon(R) CPU E5-2698 v3 @ 2.30GHz	NVIDIA RTX A2000 12GB	31.27	Ubuntu 22.04.3 LTS	0.1.48	llama3:8b	37.09
Linux	Intel(R) Xeon(R) CPU E5-2698 v3 @ 2.30GHz	NVIDIA RTX A2000 12GB	31.27	Ubuntu 22.04.3 LTS	0.1.48	phi3:3.8b	63.79
Linux	Intel(R) Xeon(R) CPU E5-2698 v3 @ 2.30GHz	NVIDIA RTX A2000 12GB	31.27	Ubuntu 22.04.3 LTS	0.1.48	gemma:2b	79.66
Linux	Intel(R) Xeon(R) CPU E5-2698 v3 @ 2.30GHz	NVIDIA RTX A2000 12GB	31.27	Ubuntu 22.04.3 LTS	0.1.48	gemma:7b	37.02
Linux	Intel(R) Xeon(R) CPU E5-2698 v3 @ 2.30GHz	NVIDIA RTX A2000 12GB	31.27	Ubuntu 22.04.3 LTS	0.1.48	llava:7b	44.3
Linux	Intel(R) Xeon(R) CPU E5-2698 v3 @ 2.30GHz	NVIDIA RTX A2000 12GB	31.27	Ubuntu 22.04.3 LTS	0.1.48	llava:13b	27.33
Linux	Intel(R) Core(TM) i5-9400 CPU @ 2.90GHz	NVIDIA GeForce GTX 1050 Ti	15.56	Ubuntu 24.04 LTS	0.2.5	mistral:7b	12.74
Linux	Intel(R) Core(TM) i5-9400 CPU @ 2.90GHz	NVIDIA GeForce GTX 1050 Ti	15.56	Ubuntu 24.04 LTS	0.2.5	llama3:8b	9.88
Linux	Intel(R) Core(TM) i5-9400 CPU @ 2.90GHz	NVIDIA GeForce GTX 1050 Ti	15.56	Ubuntu 24.04 LTS	0.2.5	phi3:3.8b	26.15
Linux	Intel(R) Core(TM) i5-9400 CPU @ 2.90GHz	NVIDIA GeForce GTX 1050 Ti	15.56	Ubuntu 24.04 LTS	0.2.5	gemma:2b	41.27
Linux	Intel(R) Core(TM) i5-9400 CPU @ 2.90GHz	NVIDIA GeForce GTX 1050 Ti	15.56	Ubuntu 24.04 LTS	0.2.5	gemma:7b	7.06
Linux	Intel(R) Core(TM) i5-9400 CPU @ 2.90GHz	NVIDIA GeForce GTX 1050 Ti	15.56	Ubuntu 24.04 LTS	0.2.5	llava:7b	11.57
Linux	Intel(R) Core(TM) i5-9400 CPU @ 2.90GHz	NVIDIA GeForce GTX 1050 Ti	15.56	Ubuntu 24.04 LTS	0.2.5	llava:13b	4.71
Linux	AMD Ryzen 9 7950X3D 16-Core Processor	NVIDIA GeForce GTX 1660 SUPER	61.92	Ubuntu 24.04 LTS	0.2.5	mistral:7b	25.59
Linux	AMD Ryzen 9 7950X3D 16-Core Processor	NVIDIA GeForce GTX 1660 SUPER	61.92	Ubuntu 24.04 LTS	0.2.5	llama3:8b	25.11
Linux	AMD Ryzen 9 7950X3D 16-Core Processor	NVIDIA GeForce GTX 1660 SUPER	61.92	Ubuntu 24.04 LTS	0.2.5	phi3:3.8b	38.69
Linux	AMD Ryzen 9 7950X3D 16-Core Processor	NVIDIA GeForce GTX 1660 SUPER	61.92	Ubuntu 24.04 LTS	0.2.5	gemma:2b	70.69
Linux	AMD Ryzen 9 7950X3D 16-Core Processor	NVIDIA GeForce GTX 1660 SUPER	61.92	Ubuntu 24.04 LTS	0.2.5	gemma:7b	18.24
Linux	AMD Ryzen 9 7950X3D 16-Core Processor	NVIDIA GeForce GTX 1660 SUPER	61.92	Ubuntu 24.04 LTS	0.2.5	llava:7b	40.43
Linux	AMD Ryzen 9 7950X3D 16-Core Processor	NVIDIA GeForce GTX 1660 SUPER	61.92	Ubuntu 24.04 LTS	0.2.5	llava:13b	11.05
Linux	Intel(R) Xeon(R) CPU E5-2650 v3 @ 2.30GHz	G200eR2	32.00	"Fedora Linux 39 (Container Image)"	0.2.5	mistral:7b	5.78
Linux	Intel(R) Xeon(R) CPU E5-2650 v3 @ 2.30GHz	G200eR2	32.00	"Fedora Linux 39 (Container Image)"	0.2.5	llama3:8b	4.27
Linux	Intel(R) Xeon(R) CPU E5-2650 v3 @ 2.30GHz	G200eR2	32.00	"Fedora Linux 39 (Container Image)"	0.2.5	phi3:3.8b	6.89
Linux	Intel(R) Xeon(R) CPU E5-2650 v3 @ 2.30GHz	G200eR2	32.00	"Fedora Linux 39 (Container Image)"	0.2.5	gemma:2b	7.36
Linux	Intel(R) Xeon(R) CPU E5-2650 v3 @ 2.30GHz	G200eR2	32.00	"Fedora Linux 39 (Container Image)"	0.2.5	gemma:7b	4.37
Linux	Intel(R) Xeon(R) CPU E5-2650 v3 @ 2.30GHz	G200eR2	32.00	"Fedora Linux 39 (Container Image)"	0.2.5	llava:7b	5.22
Linux	Intel(R) Xeon(R) CPU E5-2650 v3 @ 2.30GHz	G200eR2	32.00	"Fedora Linux 39 (Container Image)"	0.2.5	llava:13b	3.41
Linux	13th Gen Intel(R) Core(TM) i9-13900K	NVIDIA GeForce RTX 4090	31.23	Ubuntu 23.10	0.2.5	mistral:7b	98.22
Linux	13th Gen Intel(R) Core(TM) i9-13900K	NVIDIA GeForce RTX 4090	31.23	Ubuntu 23.10	0.2.5	llama3:8b	89.69
Linux	13th Gen Intel(R) Core(TM) i9-13900K	NVIDIA GeForce RTX 4090	31.23	Ubuntu 23.10	0.2.5	phi3:3.8b	135.8
Linux	13th Gen Intel(R) Core(TM) i9-13900K	NVIDIA GeForce RTX 4090	31.23	Ubuntu 23.10	0.2.5	gemma:2b	151.74
Linux	13th Gen Intel(R) Core(TM) i9-13900K	NVIDIA GeForce RTX 4090	31.23	Ubuntu 23.10	0.2.5	gemma:7b	83.67
Linux	13th Gen Intel(R) Core(TM) i9-13900K	NVIDIA GeForce RTX 4090	31.23	Ubuntu 23.10	0.2.5	llava:7b	101.68
Linux	13th Gen Intel(R) Core(TM) i9-13900K	NVIDIA GeForce RTX 4090	31.23	Ubuntu 23.10	0.2.5	llava:13b	74.41
Linux	AMD Ryzen 7 3700X 8-Core Processor	NVIDIA GeForce GTX 1660 SUPER	125.72	"Fedora Linux 40 (Workstation Edition)"	0.2.5	mistral:7b	14.96
Linux	AMD Ryzen 7 3700X 8-Core Processor	NVIDIA GeForce GTX 1660 SUPER	125.72	"Fedora Linux 40 (Workstation Edition)"	0.2.5	llama3:8b	10.54
Linux	AMD Ryzen 7 3700X 8-Core Processor	NVIDIA GeForce GTX 1660 SUPER	125.72	"Fedora Linux 40 (Workstation Edition)"	0.2.5	phi3:3.8b	22.07
Linux	AMD Ryzen 7 3700X 8-Core Processor	NVIDIA GeForce GTX 1660 SUPER	125.72	"Fedora Linux 40 (Workstation Edition)"	0.2.5	gemma:2b	53.62
Linux	AMD Ryzen 7 3700X 8-Core Processor	NVIDIA GeForce GTX 1660 SUPER	125.72	"Fedora Linux 40 (Workstation Edition)"	0.2.5	gemma:7b	8.36
Linux	AMD Ryzen 7 3700X 8-Core Processor	NVIDIA GeForce GTX 1660 SUPER	125.72	"Fedora Linux 40 (Workstation Edition)"	0.2.5	llava:7b	17.68
Linux	AMD Ryzen 7 3700X 8-Core Processor	NVIDIA GeForce GTX 1660 SUPER	125.72	"Fedora Linux 40 (Workstation Edition)"	0.2.5	llava:13b	5.05
Linux	Intel(R) Core(TM) i7-9700K CPU @ 3.60GHz	NVIDIA GeForce RTX 2080 Ti	62.73	Ubuntu 18.04.4 LTS	0.2.5	mistral:7b	93.33
Linux	Intel(R) Core(TM) i7-9700K CPU @ 3.60GHz	NVIDIA GeForce RTX 2080 Ti	62.73	Ubuntu 18.04.4 LTS	0.2.5	llama3:8b	50.85
Linux	Intel(R) Core(TM) i7-9700K CPU @ 3.60GHz	NVIDIA GeForce RTX 2080 Ti	62.73	Ubuntu 18.04.4 LTS	0.2.5	phi3:3.8b	124.9
Linux	Intel(R) Core(TM) i7-9700K CPU @ 3.60GHz	NVIDIA GeForce RTX 2080 Ti	62.73	Ubuntu 18.04.4 LTS	0.2.5	gemma:2b	120.92
Linux	Intel(R) Core(TM) i7-9700K CPU @ 3.60GHz	NVIDIA GeForce RTX 2080 Ti	62.73	Ubuntu 18.04.4 LTS	0.2.5	gemma:7b	76.4
Linux	Intel(R) Core(TM) i7-9700K CPU @ 3.60GHz	NVIDIA GeForce RTX 2080 Ti	62.73	Ubuntu 18.04.4 LTS	0.2.5	llava:7b	96.01
Linux	Intel(R) Core(TM) i7-9700K CPU @ 3.60GHz	NVIDIA GeForce RTX 2080 Ti	62.73	Ubuntu 18.04.4 LTS	0.2.5	llava:13b	58.77
Linux	AMD Ryzen 7 5700G with Radeon Graphics	NVIDIA GeForce RTX 3060	30.64	Ubuntu 24.04 LTS	0.2.5	mistral:7b	63.72
Linux	AMD Ryzen 7 5700G with Radeon Graphics	NVIDIA GeForce RTX 3060	30.64	Ubuntu 24.04 LTS	0.2.5	llama3:8b	58.22
Linux	AMD Ryzen 7 5700G with Radeon Graphics	NVIDIA GeForce RTX 3060	30.64	Ubuntu 24.04 LTS	0.2.5	phi3:3.8b	99.81
Linux	AMD Ryzen 7 5700G with Radeon Graphics	NVIDIA GeForce RTX 3060	30.64	Ubuntu 24.04 LTS	0.2.5	gemma:2b	125.3
Linux	AMD Ryzen 7 5700G with Radeon Graphics	NVIDIA GeForce RTX 3060	30.64	Ubuntu 24.04 LTS	0.2.5	gemma:7b	52.5
Linux	AMD Ryzen 7 5700G with Radeon Graphics	NVIDIA GeForce RTX 3060	30.64	Ubuntu 24.04 LTS	0.2.5	llava:7b	66.73
Linux	AMD Ryzen 7 5700G with Radeon Graphics	NVIDIA GeForce RTX 3060	30.64	Ubuntu 24.04 LTS	0.2.5	llava:13b	39.22
Linux	AMD EPYC 7313P 16-Core Processor	NVIDIA RTX 6000 Ada Generation	251.56	Ubuntu 24.04 LTS	0.0.0	llama3:8b	109.1
Linux	Intel(R) Core(TM) i7-14700KF	NVIDIA GeForce RTX 4090 D	31.15	Ubuntu 24.04 LTS	0.2.1	mistral:7b	141.64
Linux	Intel(R) Core(TM) i7-14700KF	NVIDIA GeForce RTX 4090 D	31.15	Ubuntu 24.04 LTS	0.2.1	llama3:8b	123.22
Linux	Intel(R) Core(TM) i7-14700KF	NVIDIA GeForce RTX 4090 D	31.15	Ubuntu 24.04 LTS	0.2.1	phi3:3.8b	195.24
Linux	Intel(R) Core(TM) i7-14700KF	NVIDIA GeForce RTX 4090 D	31.15	Ubuntu 24.04 LTS	0.2.1	gemma:2b	227.56
Linux	Intel(R) Core(TM) i7-14700KF	NVIDIA GeForce RTX 4090 D	31.15	Ubuntu 24.04 LTS	0.2.1	gemma:7b	116.24
Linux	Intel(R) Core(TM) i7-14700KF	NVIDIA GeForce RTX 4090 D	31.15	Ubuntu 24.04 LTS	0.2.1	llava:7b	146.03
Linux	Intel(R) Core(TM) i7-14700KF	NVIDIA GeForce RTX 4090 D	31.15	Ubuntu 24.04 LTS	0.2.1	llava:13b	93.11
Linux	Intel(R) Core(TM) i7-14700KF	NVIDIA GeForce RTX 4090 D	31.15	Ubuntu 24.04 LTS	0.2.1	mistral:7b	154.48
Linux	Intel(R) Core(TM) i7-14700KF	NVIDIA GeForce RTX 4090 D	31.15	Ubuntu 24.04 LTS	0.2.1	llama3:8b	132.32
Linux	Intel(R) Core(TM) i7-14700KF	NVIDIA GeForce RTX 4090 D	31.15	Ubuntu 24.04 LTS	0.2.1	phi3:3.8b	199.4
Linux	Intel(R) Core(TM) i7-14700KF	NVIDIA GeForce RTX 4090 D	31.15	Ubuntu 24.04 LTS	0.2.1	gemma:2b	242.19
Linux	Intel(R) Core(TM) i7-14700KF	NVIDIA GeForce RTX 4090 D	31.15	Ubuntu 24.04 LTS	0.2.1	gemma:7b	120.85
Linux	Intel(R) Core(TM) i7-14700KF	NVIDIA GeForce RTX 4090 D	31.15	Ubuntu 24.04 LTS	0.2.1	llava:7b	157.7
Linux	Intel(R) Core(TM) i7-14700KF	NVIDIA GeForce RTX 4090 D	31.15	Ubuntu 24.04 LTS	0.2.1	llava:13b	98.16
Linux	Intel(R) Core(TM) i7-14700KF	NVIDIA GeForce RTX 4090 D	31.15	Ubuntu 24.04 LTS	0.2.1	mistral:7b	141.59
Linux	Intel(R) Core(TM) i7-14700KF	NVIDIA GeForce RTX 4090 D	31.15	Ubuntu 24.04 LTS	0.2.1	llama3:8b	123.21
Linux	Intel(R) Core(TM) i7-14700KF	NVIDIA GeForce RTX 4090 D	31.15	Ubuntu 24.04 LTS	0.2.1	phi3:3.8b	199.9
Linux	Intel(R) Core(TM) i7-14700KF	NVIDIA GeForce RTX 4090 D	31.15	Ubuntu 24.04 LTS	0.2.1	gemma:2b	227.96
Linux	Intel(R) Core(TM) i7-14700KF	NVIDIA GeForce RTX 4090 D	31.15	Ubuntu 24.04 LTS	0.2.1	gemma:7b	115.8
Linux	Intel(R) Core(TM) i7-14700KF	NVIDIA GeForce RTX 4090 D	31.15	Ubuntu 24.04 LTS	0.2.1	llava:7b	147.05
Linux	Intel(R) Core(TM) i7-14700KF	NVIDIA GeForce RTX 4090 D	31.15	Ubuntu 24.04 LTS	0.2.1	llava:13b	93.21
Linux	AMD EPYC 7313P 16-Core Processor	NVIDIA RTX 6000 Ada Generation	251.56	Ubuntu 24.04 LTS	0.0.0	llama3:8b	108.67
Linux	AMD Ryzen 5 5600G with Radeon Graphics	Cezanne	27.29	Ubuntu 22.04.4 LTS	0.2.5	mistral:7b	8.57
Linux	AMD Ryzen 5 5600G with Radeon Graphics	Cezanne	27.29	Ubuntu 22.04.4 LTS	0.2.5	llama3:8b	7.88
Linux	AMD Ryzen 5 5600G with Radeon Graphics	Cezanne	27.29	Ubuntu 22.04.4 LTS	0.2.5	phi3:3.8b	12.83
Linux	AMD Ryzen 5 5600G with Radeon Graphics	Cezanne	27.29	Ubuntu 22.04.4 LTS	0.2.5	gemma:2b	18.96
Linux	AMD Ryzen 5 5600G with Radeon Graphics	Cezanne	27.29	Ubuntu 22.04.4 LTS	0.2.5	gemma:7b	6.62
Linux	AMD Ryzen 5 5600G with Radeon Graphics	Cezanne	27.29	Ubuntu 22.04.4 LTS	0.2.5	llava:7b	8.79
Linux	AMD Ryzen 5 5600G with Radeon Graphics	Cezanne	27.29	Ubuntu 22.04.4 LTS	0.2.5	llava:13b	4.88
Linux	AMD Ryzen 9 7900X 12-Core Processor	NVIDIA GeForce RTX 4090	15.19	Ubuntu 22.04.4 LTS	0.2.7	mistral:7b	99.58
Linux	AMD Ryzen 9 7900X 12-Core Processor	NVIDIA GeForce RTX 4090	15.19	Ubuntu 22.04.4 LTS	0.2.7	llama3:8b	90.73
Linux	AMD Ryzen 9 7900X 12-Core Processor	NVIDIA GeForce RTX 4090	15.19	Ubuntu 22.04.4 LTS	0.2.7	phi3:3.8b	137.15
Linux	AMD Ryzen 9 7900X 12-Core Processor	NVIDIA GeForce RTX 4090	15.19	Ubuntu 22.04.4 LTS	0.2.7	gemma:2b	156.77
Linux	AMD Ryzen 9 7900X 12-Core Processor	NVIDIA GeForce RTX 4090	15.19	Ubuntu 22.04.4 LTS	0.2.7	gemma:7b	89.62
Linux	AMD Ryzen 9 7900X 12-Core Processor	NVIDIA GeForce RTX 4090	15.19	Ubuntu 22.04.4 LTS	0.2.7	llava:7b	103.04
Linux	AMD Ryzen 9 7900X 12-Core Processor	NVIDIA GeForce RTX 4090	15.19	Ubuntu 22.04.4 LTS	0.2.7	llava:13b	74.13
Linux	AMD EPYC Processor	NVIDIA RTX A4000	235.93	Ubuntu 22.04.4 LTS	0.2.6	mistral:7b	69.53
Linux	AMD EPYC Processor	NVIDIA RTX A4000	235.93	Ubuntu 22.04.4 LTS	0.2.6	llama3:8b	59.58
Linux	AMD EPYC Processor	NVIDIA RTX A4000	235.93	Ubuntu 22.04.4 LTS	0.2.6	phi3:3.8b	104.62
Linux	AMD EPYC Processor	NVIDIA RTX A4000	235.93	Ubuntu 22.04.4 LTS	0.2.6	gemma:2b	121.62
Linux	AMD EPYC Processor	NVIDIA RTX A4000	235.93	Ubuntu 22.04.4 LTS	0.2.6	gemma:7b	56.27
Linux	AMD EPYC Processor	NVIDIA RTX A4000	235.93	Ubuntu 22.04.4 LTS	0.2.6	llava:7b	73.22
Linux	AMD EPYC Processor	NVIDIA RTX A4000	235.93	Ubuntu 22.04.4 LTS	0.2.6	llava:13b	43.24
Linux	Intel(R) Core(TM) i5-10400F CPU @ 2.90GHz	Navi 10 [Radeon RX 5600 OEM/5600 XT / 5700/5700 XT]	15.42	Ubuntu 22.04.4 LTS	0.0.0	llama3:latest	17.4
Linux	Intel(R) Core(TM) i5-10400F CPU @ 2.90GHz	Navi 10 [Radeon RX 5600 OEM/5600 XT / 5700/5700 XT]	15.42	Ubuntu 22.04.4 LTS	0.0.0	llama3:latest	15.41
Linux	13th Gen Intel(R) Core(TM) i7-13700	NVIDIA GeForce RTX 4090	31.23	Ubuntu 22.04.4 LTS	0.2.8	mistral:7b	106.38
Linux	13th Gen Intel(R) Core(TM) i7-13700	NVIDIA GeForce RTX 4090	31.23	Ubuntu 22.04.4 LTS	0.2.8	llama3:8b	99.6
Linux	13th Gen Intel(R) Core(TM) i7-13700	NVIDIA GeForce RTX 4090	31.23	Ubuntu 22.04.4 LTS	0.2.8	phi3:3.8b	143.04
Linux	13th Gen Intel(R) Core(TM) i7-13700	NVIDIA GeForce RTX 4090	31.23	Ubuntu 22.04.4 LTS	0.2.8	gemma:2b	180.42
Linux	13th Gen Intel(R) Core(TM) i7-13700	NVIDIA GeForce RTX 4090	31.23	Ubuntu 22.04.4 LTS	0.2.8	gemma:7b	111.66
Linux	13th Gen Intel(R) Core(TM) i7-13700	NVIDIA GeForce RTX 4090	31.23	Ubuntu 22.04.4 LTS	0.2.8	llava:7b	110.71
Linux	13th Gen Intel(R) Core(TM) i7-13700	NVIDIA GeForce RTX 4090	31.23	Ubuntu 22.04.4 LTS	0.2.8	llava:13b	81.44
Linux	Intel(R) Core(TM) i9-7960X CPU @ 2.80GHz	NVIDIA GeForce RTX 3090	93.97	Ubuntu 22.04.4 LTS	0.2.8	mistral:7b	105.2
Linux	Intel(R) Core(TM) i9-7960X CPU @ 2.80GHz	NVIDIA GeForce RTX 3090	93.97	Ubuntu 22.04.4 LTS	0.2.8	llama3:8b	87.01
Linux	Intel(R) Core(TM) i9-7960X CPU @ 2.80GHz	NVIDIA GeForce RTX 3090	93.97	Ubuntu 22.04.4 LTS	0.2.8	phi3:3.8b	137.25
Linux	Intel(R) Core(TM) i9-7960X CPU @ 2.80GHz	NVIDIA GeForce RTX 3090	93.97	Ubuntu 22.04.4 LTS	0.2.8	gemma:2b	182.28
Linux	Intel(R) Core(TM) i9-7960X CPU @ 2.80GHz	NVIDIA GeForce RTX 3090	93.97	Ubuntu 22.04.4 LTS	0.2.8	gemma:7b	99.31
Linux	Intel(R) Core(TM) i9-7960X CPU @ 2.80GHz	NVIDIA GeForce RTX 3090	93.97	Ubuntu 22.04.4 LTS	0.2.8	llava:7b	112.73
Linux	Intel(R) Core(TM) i9-7960X CPU @ 2.80GHz	NVIDIA GeForce RTX 3090	93.97	Ubuntu 22.04.4 LTS	0.2.8	llava:13b	69.67
Linux	Intel(R) Core(TM) i9-7960X CPU @ 2.80GHz	NVIDIA GeForce RTX 3090	93.97	Ubuntu 22.04.4 LTS	0.2.8	mistral:7b	104.6
Linux	Intel(R) Core(TM) i9-7960X CPU @ 2.80GHz	NVIDIA GeForce RTX 3090	93.97	Ubuntu 22.04.4 LTS	0.2.8	llama3:8b	83.11
Linux	Intel(R) Core(TM) i9-7960X CPU @ 2.80GHz	NVIDIA GeForce RTX 3090	93.97	Ubuntu 22.04.4 LTS	0.2.8	phi3:3.8b	136.61
Linux	Intel(R) Core(TM) i9-7960X CPU @ 2.80GHz	NVIDIA GeForce RTX 3090	93.97	Ubuntu 22.04.4 LTS	0.2.8	gemma:2b	186.71
Linux	Intel(R) Core(TM) i9-7960X CPU @ 2.80GHz	NVIDIA GeForce RTX 3090	93.97	Ubuntu 22.04.4 LTS	0.2.8	gemma:7b	98.82
Linux	Intel(R) Core(TM) i9-7960X CPU @ 2.80GHz	NVIDIA GeForce RTX 3090	93.97	Ubuntu 22.04.4 LTS	0.2.8	llava:7b	110.79
Linux	Intel(R) Core(TM) i9-7960X CPU @ 2.80GHz	NVIDIA GeForce RTX 3090	93.97	Ubuntu 22.04.4 LTS	0.2.8	llava:13b	73.52
Linux	Intel(R) Core(TM) i9-7960X CPU @ 2.80GHz	NVIDIA GeForce RTX 3090	93.97	Ubuntu 22.04.4 LTS	0.2.8	mistral:7b	117.27
Linux	Intel(R) Core(TM) i9-7960X CPU @ 2.80GHz	NVIDIA GeForce RTX 3090	93.97	Ubuntu 22.04.4 LTS	0.2.8	llama3:8b	98.34
Linux	Intel(R) Core(TM) i9-7960X CPU @ 2.80GHz	NVIDIA GeForce RTX 3090	93.97	Ubuntu 22.04.4 LTS	0.2.8	phi3:3.8b	145.58
Linux	Intel(R) Core(TM) i9-7960X CPU @ 2.80GHz	NVIDIA GeForce RTX 3090	93.97	Ubuntu 22.04.4 LTS	0.2.8	gemma:2b	224.86
Linux	Intel(R) Core(TM) i9-7960X CPU @ 2.80GHz	NVIDIA GeForce RTX 3090	93.97	Ubuntu 22.04.4 LTS	0.2.8	gemma:7b	102.94
Linux	Intel(R) Core(TM) i9-7960X CPU @ 2.80GHz	NVIDIA GeForce RTX 3090	93.97	Ubuntu 22.04.4 LTS	0.2.8	llava:7b	113.55
Linux	Intel(R) Core(TM) i9-7960X CPU @ 2.80GHz	NVIDIA GeForce RTX 3090	93.97	Ubuntu 22.04.4 LTS	0.2.8	llava:13b	75.42
Linux	Intel(R) Xeon(R) CPU E5-2690 v4 @ 2.60GHz	NVIDIA GeForce RTX 3090	62.79	Ubuntu 22.04.4 LTS	0.2.7	mistral:7b	102.61
Linux	Intel(R) Xeon(R) CPU E5-2690 v4 @ 2.60GHz	NVIDIA GeForce RTX 3090	62.79	Ubuntu 22.04.4 LTS	0.2.7	llama3.1:8b	87.24
Linux	Intel(R) Xeon(R) CPU E5-2690 v4 @ 2.60GHz	NVIDIA GeForce RTX 3090	62.79	Ubuntu 22.04.4 LTS	0.2.7	phi3:3.8b	144.72
Linux	Intel(R) Xeon(R) CPU E5-2690 v4 @ 2.60GHz	NVIDIA GeForce RTX 3090	62.79	Ubuntu 22.04.4 LTS	0.2.7	qwen2:7b	87.31
Linux	Intel(R) Xeon(R) CPU E5-2690 v4 @ 2.60GHz	NVIDIA GeForce RTX 3090	62.79	Ubuntu 22.04.4 LTS	0.2.7	gemma2:9b	59.76
Linux	Intel(R) Xeon(R) CPU E5-2690 v4 @ 2.60GHz	NVIDIA GeForce RTX 3090	62.79	Ubuntu 22.04.4 LTS	0.2.7	llava:7b	108.88
Linux	Intel(R) Xeon(R) CPU E5-2690 v4 @ 2.60GHz	NVIDIA GeForce RTX 3090	62.79	Ubuntu 22.04.4 LTS	0.2.7	llava:13b	60.78
Linux	Intel(R) Xeon(R) CPU E5-2690 v4 @ 2.60GHz	NVIDIA GeForce RTX 3090	62.79	Ubuntu 22.04.4 LTS	0.2.7	mistral:7b	102
Linux	Intel(R) Xeon(R) CPU E5-2690 v4 @ 2.60GHz	NVIDIA GeForce RTX 3090	62.79	Ubuntu 22.04.4 LTS	0.2.7	llama3.1:8b	86.26
Linux	Intel(R) Xeon(R) CPU E5-2690 v4 @ 2.60GHz	NVIDIA GeForce RTX 3090	62.79	Ubuntu 22.04.4 LTS	0.2.7	phi3:3.8b	145.84
Linux	Intel(R) Xeon(R) CPU E5-2690 v4 @ 2.60GHz	NVIDIA GeForce RTX 3090	62.79	Ubuntu 22.04.4 LTS	0.2.7	qwen2:7b	87.02
Linux	Intel(R) Xeon(R) CPU E5-2690 v4 @ 2.60GHz	NVIDIA GeForce RTX 3090	62.79	Ubuntu 22.04.4 LTS	0.2.7	gemma2:9b	60.99
Linux	Intel(R) Xeon(R) CPU E5-2690 v4 @ 2.60GHz	NVIDIA GeForce RTX 3090	62.79	Ubuntu 22.04.4 LTS	0.2.7	llava:7b	110
Linux	Intel(R) Xeon(R) CPU E5-2690 v4 @ 2.60GHz	NVIDIA GeForce RTX 3090	62.79	Ubuntu 22.04.4 LTS	0.2.7	llava:13b	60.99
Linux	Intel(R) Xeon(R) Platinum 8470	NVIDIA H100 80GB HBM3	1007.34	Ubuntu 24.04 LTS	0.2.8	mistral:7b	145.13
Linux	Intel(R) Xeon(R) Platinum 8470	NVIDIA H100 80GB HBM3	1007.34	Ubuntu 24.04 LTS	0.2.8	llama3.1:8b	125.99
Linux	Intel(R) Xeon(R) Platinum 8470	NVIDIA H100 80GB HBM3	1007.34	Ubuntu 24.04 LTS	0.2.8	phi3:3.8b	148.81
Linux	Intel(R) Xeon(R) Platinum 8470	NVIDIA H100 80GB HBM3	1007.34	Ubuntu 24.04 LTS	0.2.8	qwen2:7b	135.18
Linux	Intel(R) Xeon(R) Platinum 8470	NVIDIA H100 80GB HBM3	1007.34	Ubuntu 24.04 LTS	0.2.8	gemma2:9b	81.82
Linux	Intel(R) Xeon(R) Platinum 8470	NVIDIA H100 80GB HBM3	1007.34	Ubuntu 24.04 LTS	0.2.8	llava:7b	167.12
Linux	Intel(R) Xeon(R) Platinum 8470	NVIDIA H100 80GB HBM3	1007.34	Ubuntu 24.04 LTS	0.2.8	llava:13b	115.41
Linux	Intel(R) Xeon(R) Gold 6138 CPU @ 2.00GHz	NVIDIA RTXA5000-24Q	78.54	Debian GNU/Linux 12 (bookworm)	0.2.7	mistral:7b	91.58
Linux	Intel(R) Xeon(R) Gold 6138 CPU @ 2.00GHz	NVIDIA RTXA5000-24Q	78.54	Debian GNU/Linux 12 (bookworm)	0.2.7	llama3.1:8b	74.36
Linux	Intel(R) Xeon(R) Gold 6138 CPU @ 2.00GHz	NVIDIA RTXA5000-24Q	78.54	Debian GNU/Linux 12 (bookworm)	0.2.7	phi3:3.8b	125.87
Linux	Intel(R) Xeon(R) Gold 6138 CPU @ 2.00GHz	NVIDIA RTXA5000-24Q	78.54	Debian GNU/Linux 12 (bookworm)	0.2.7	qwen2:7b	74.55
Linux	Intel(R) Xeon(R) Gold 6138 CPU @ 2.00GHz	NVIDIA RTXA5000-24Q	78.54	Debian GNU/Linux 12 (bookworm)	0.2.7	gemma2:9b	51.13
Linux	Intel(R) Xeon(R) Gold 6138 CPU @ 2.00GHz	NVIDIA RTXA5000-24Q	78.54	Debian GNU/Linux 12 (bookworm)	0.2.7	llava:7b	94.89
Linux	Intel(R) Xeon(R) Gold 6138 CPU @ 2.00GHz	NVIDIA RTXA5000-24Q	78.54	Debian GNU/Linux 12 (bookworm)	0.2.7	llava:13b	58.33
Linux	QEMU Virtual CPU version 2.5+	Navi 22 [Radeon RX 6700/6700 XT/6750 XT / 6800M/6850M XT]	23.43	Ubuntu 22.04.4 LTS	0.2.8	mistral:7b	55.06
Linux	QEMU Virtual CPU version 2.5+	Navi 22 [Radeon RX 6700/6700 XT/6750 XT / 6800M/6850M XT]	23.43	Ubuntu 22.04.4 LTS	0.2.8	llama3.1:8b	50.52
Linux	QEMU Virtual CPU version 2.5+	Navi 22 [Radeon RX 6700/6700 XT/6750 XT / 6800M/6850M XT]	23.43	Ubuntu 22.04.4 LTS	0.2.8	phi3:3.8b	68.34
Linux	QEMU Virtual CPU version 2.5+	Navi 22 [Radeon RX 6700/6700 XT/6750 XT / 6800M/6850M XT]	23.43	Ubuntu 22.04.4 LTS	0.2.8	qwen2:7b	53.03
Linux	QEMU Virtual CPU version 2.5+	Navi 22 [Radeon RX 6700/6700 XT/6750 XT / 6800M/6850M XT]	23.43	Ubuntu 22.04.4 LTS	0.2.8	gemma2:9b	35.62
Linux	QEMU Virtual CPU version 2.5+