Unitary Fund

Because evolution is unitary.

Automatically running QED-C benchmarks on Metriq

2023-02-28
Metriq Team

We are excited to announce the integration of an automated pipeline to add state-of-the-art benchmark from the Quantum Economic Development Consortium (QED-C) into Metriq, the open platform that makes transparent, accessible benchmarks available to everyone in the quantum computing community.

The Technical Advisory Committee (TAC) for Standards and Performance Metrics from QED-C provided an open-source suite of quantum benchmarks (GitHub) to measure the effectiveness of quantum computing hardware for specific applications (further information provided in this preprint). This provided some of the first application-oriented benchmarks that were made public with supporting data along with a preprint, an incredibly valuable asset to the benchmarking community. Recently, the QED-C benchmarks suite was updated and new optimization benchmarks (preprint).

The results of running these benchmarks can be found via this Zenodo posting. On Metriq, we have integrated these results into a submission that captures these results. While this submission on Metriq captures the results obtained by QED-C it also is a “living” result in that it is automatically updated with benchmarks obtained by running the QED-C code on a variety of hardware offerings.

On Metriq, we have established an automated pipeline that is able to run and update our submission via a fork of the QED-C benchmarking suite. This pipeline enables anyone to automatically run benchmarks for a specific algorithm, hardware provider, and quantum processor. This allows the Metriq community to continually extend the benchmarking foundation provided by QED-C to expand their results by running on further hardware providers to obtain up-to-date results.

The backbone of the pipeline is powered by the metriq-client project. This client allows the user to interface with the Metriq platform via a Python API. The user can perform queries or upload batched results via a Python script for submissions that may entail a sizeable amount of results that would be cumbersome to upload manually via the website.

Many new results have been added to the QED-C submission for the Bernstein-Vazarani task on Metriq thanks to the automatic QED-C to Metriq pipeline.

The automated QED-C to Metriq pipeline will be continuing to run on more algorithms and platforms to generate further benchmarking results to be displayed on Metriq!

This website is hosted on github, and our community follows this code of conduct.

If you have suggestions for changes then please open up a pull request!

©2020 Unitary Fund