Intel Xeon Max Sees Some Performance Gains For OpenVINO & ONNX With Linux 6.9

Written by Michael Larabel in Software on 4 April 2024 at 10:30 AM EDT. Page 5 of 5. 2 Comments.

ONNX Runtime benchmark with settings of Model: GPT-2, Device: CPU, Executor: Parallel. Linux 6.9-rc2 was the fastest.

ONNX Runtime benchmark with settings of Model: yolov4, Device: CPU, Executor: Parallel. Linux 6.9-rc2 was the fastest.

ONNX Runtime benchmark with settings of Model: yolov4, Device: CPU, Executor: Standard. Linux 6.9-rc2 was the fastest.

ONNX Runtime benchmark with settings of Model: T5 Encoder, Device: CPU, Executor: Parallel. Linux 6.9-rc2 was the fastest.

ONNX Runtime benchmark with settings of Model: T5 Encoder, Device: CPU, Executor: Standard. Linux 6.9-rc2 was the fastest.

ONNX Runtime benchmark with settings of Model: bertsquad-12, Device: CPU, Executor: Standard. Linux 6.9-rc2 was the fastest.

ONNX Runtime benchmark with settings of Model: CaffeNet 12-int8, Device: CPU, Executor: Standard. Linux 6.9-rc2 was the fastest.

ONNX Runtime benchmark with settings of Model: fcn-resnet101-11, Device: CPU, Executor: Parallel. Linux 6.9-rc2 was the fastest.

ONNX Runtime benchmark with settings of Model: ArcFace ResNet-100, Device: CPU, Executor: Parallel. Linux 6.9-rc2 was the fastest.

ONNX Runtime benchmark with settings of Model: ResNet50 v1-12-int8, Device: CPU, Executor: Standard. Linux 6.9-rc2 was the fastest.

ONNX Runtime benchmark with settings of Model: super-resolution-10, Device: CPU, Executor: Standard. Linux 6.9-rc2 was the fastest.

With Microsoft's ONNX Runtime there were some wins picked up on Linux 6.9.

Llama.cpp benchmark with settings of Model: llama-2-13b.Q4_0.gguf. Linux 6.9-rc2 was the fastest.

Llama.cpp benchmark with settings of Model: llama-2-70b-chat.Q5_0.gguf. Linux 6.9-rc2 was the fastest.

Llamafile benchmark with settings of Test: mistral-7b-instruct-v0.2.Q8_0, Acceleration: CPU. Linux 6.9-rc2 was the fastest.

Llamafile benchmark with settings of Test: wizardcoder-python-34b-v1.0.Q6_K, Acceleration: CPU. Linux 6.8 was the fastest.

With Llamafile and Llama.cpp there were some minor improvements too on the Linux 6.9 kernel.

BRL-CAD benchmark with settings of VGR Performance Metric. Linux 6.9-rc2 was the fastest.

Chaos Group V-RAY benchmark with settings of Mode: CPU. Linux 6.8 was the fastest.

Other workloads were commonly rather flat.

Geometric Mean Of All Test Results benchmark with settings of Result Composite, Xeon Max Linux 6.9 Kernel Benchmarks. Linux 6.9-rc2 was the fastest.

Of 236 benchmarks carried out in looking for any performance differences of Linux 6.8 vs. 6.9-rc2 for the Intel Xeon Max 2P Supermicro server, the 6.9-rc2 performance was level overall. The geo mean showed overall the performance was flat for Xeon Max on Linux 6.9 but in a few cases as noted there were some minor improvements to enjoy with this forthcoming kernel. The main workloads that seemed to have some benefits for Xeon Max on Linux 6.9 were ONNX Runtime and OpenVINO. Besides the performance, Linux 6.9 does bring many new features for server administrators and others to enjoy for this kernel that will be out as stable in May. These results weren't as exciting as the AMD EPYC Genoa-X benchmarks last week, but at least there weren't any major performance regressions besides an isolated oneDNN test case.

2 Comments

If you enjoyed this article consider joining Phoronix Premium to view this site ad-free, multi-page articles on a single page, and other benefits. PayPal or Stripe tips are also graciously accepted. Thanks for your support.

Page: 1 2 3 4 5