Transaction Processing Performance Council (TPC) Launches an Artificial Intelligence Benchmark (TPCx-AI) The first industry-standard benchmark for measuring real-world, end-to-end AI and ML scenarios – as well as data science use cases – is now available via downloadable kit
San Francisco, CA
|
September 15, 2021 09:00 AM Pacific Daylight Time
The Transaction Processing Performance Council (TPC) today announced the immediate availability of TPCx-AI, the first industry-standard, vendor-neutral benchmark for measuring real-world, end-to-end AI and ML scenarios and data science use cases. TPCx-AI uses a diverse dataset and was specifically designed to be adaptable across a wide range of scale factors.
“The TPCx-AI benchmark is the result of collaboration between talented engineers and researchers at some of today’s leading AI organizations,” said Hamesh Patel, chair of the TPCx-AI committee and principal engineer at Intel Corporation. “It is designed to emulate real-world examples of organizations that use a variety of production ready data science pipelines – including both AI and ML approaches – and is now widely available to anyone who would like to download and run it. We look forward to feedback as industry experts, academics and others interested in benchmarking system performance begin to use it.”
TPCx-AI provides a means to evaluate performance for the System Under Test (SUT) as a general-purpose data science system that:
- Generates and processes large volumes of data
- Trains preprocessed data to produce realistic machine learning models
- Conducts accurate insights for real-world customer scenarios based on the generated models
- Can scale to large scale distributed configurations
- Allows for flexibility in configuration changes to meet the demands of the dynamic AI landscape
Additionally, the benchmark measures end-to-end time to provide insights for individual use cases, as well as throughput metrics to simulate multiuser environments for a given hardware, operating system, and data processing system configuration under a controlled, complex, multi-user AI or machine learning data science workload.
And as an “Express” class benchmark, TPCx-AI is an executable kit that can be rapidly deployed and measured. It is designed to provide relevant, objective performance data to industry users and is available for download via the TPC’s Web site via the following URL: http://tpc.org/tpcx-ai/default5.asp
Organizations interested in contributing to the TPC’s benchmark development process are also encouraged to become members, and additional information is available via the following URL: http://tpc.org/information/about/join5.asp
TPC members that contributed to the development of TPCx-AI include: Cisco, Dell, HPE, IBM, Intel, Microsoft, Red Hat, TTA, and VMware.
About the TPC:
The TPC is a non-profit corporation founded to define transaction processing and database benchmarks and to disseminate objective, verifiable TPC performance data to the industry. The TPC currently has 21 full members: Actian, Alibaba, AMD, Cisco, Dell, Fujitsu, HP Enterprise, Hitachi, Huawei, IBM, Inspur, Intel, Lenovo, Microsoft, Nutanix, Nvidia, Oracle, Red Hat, Transwarp, TTA and VMware; and four associate members: China Academy of Information and Communications Technology (CAICT), Gartner, Imec, and the University of Coimbra. Further information is available at http://www.tpc.org.
Contact Details
Forrest Carman
+1 206-859-3118
Company Website
