Debunking algorithmic qubits
Executive Summary: Quantinuum’s H-Series computers have the highest performance in the industry, verified by multiple widely adopted benchmarks including quantum volume We demonstrate that an alternative benchmark called algorithmic qubits is deeply flawed, hiding computer performance behind a plurality voting trick and gate compilations that are not widely useful.
Recently a new benchmark called algorithmic qubits (AQ) has started to be confused with quantum volume measurements. Quantum volume (QV) was specifically designed to be hard to “game,” however the algorithmic qubits test turns out to be very susceptible to tricks that can make a quantum computer look much better than it actually is. While it is not clear what can be done to fix the algorithmic qubits test, it is already clear that it is much easier to pass than QV and is a poor substitute for measuring performance. It is also important to note that algorithmic qubits are not the same as logical qubits, which are necessary for full fault-tolerant quantum computing.
To make this point clear, we simulated what algorithmic qubits data would look like for two machines, one clearly much higher performing than the other. We applied two tricks that are typically used when sharing algorithmic qubits results: gate compilation and error mitigation with plurality voting. From the data above, you can see how these tricks are misleading without further information. For example, if you compare data from the higher fidelity machine without any compilation or plurality voting (bottom left) to data from the inferior machine with both tricks (top right) you may incorrectly believe the inferior machine is performing better. Unfortunately, this inaccurate and misleading comparison has been made in the past. It is important to note that algorithmic qubits uses a subset of algorithms from a QED-C paper that introduced a suite of application oriented tests and created a repository to test available quantum computers. Importantly, that work explicitly forbids the compilation and error mitigation techniques that are causing the issue here.
As a demonstration of the perils of AQ as a benchmark, we look at data obtained on both Quantinuum’s H2-1 system as well as publicly available data from IonQ’s Forte system.
We reproduce data without any error mitigation from IonQ’s publicly released data in association with a preprint posted to the arXiv, and compare it to data taken on our H2-1 device. Without error mitigation, IonQ Forte achieves an AQ score of 9, whereas Quantinuum H2-1 achieves AQ of 26. Here you can clearly see improved circuit fidelities on the H2-1 device, as one would expect from the higher reported 2Q gate fidelities (average 99.816(5)% for Quantinuum’s H2-1 vs 99.35% for IonQ’s Forte). However, after you apply error mitigation, in this case plurality voting, to both sets of data the picture changes substantially, hiding each underlying computer’s true capabilities.
Here the H2-1 algorithmic performance still exceeds Forte (from the publicly released data), but the perceived gap has been reduced by error mitigation.
“Error mitigation, including plurality voting, may be a useful tool for some near-term quantum computing but it doesn’t work for every problem and it’s unlikely to be scalable to larger systems. In order to achieve the lofty goals of quantum computing we’ll need serious device performance upgrades. If we allow error mitigation in benchmarking it will conflate the error mitigation with the underlying device performance. This will make it hard for users to appreciate actual device improvements that translate to all applications and larger problems,” explained Dr. Charlie Baldwin, a leader in Quantinuum’s benchmarking efforts.
There are other issues with the algorithmic qubits test. The circuits used in the test can be reduced to very easy-to-run circuits with basic quantum circuit compilation that are freely available in packages like pytket. For example, the largest phase estimation and amplitude estimation tests required to pass AQ=32 are specified with 992 and 868 entangling gates respectively but applying pytket optimization reduces the circuits to 141 and 72 entangling gates. This is only possible due to choices in constructing the benchmarks and will not be universally available when using the algorithms in applications. Since AQ reports the precompiled gate counts this also may lead users to expect a machine to be able to run many more entangling gates than what is actually possible on the benchmarked hardware.
What makes a good quantum benchmark? Quantum benchmarking is extremely useful in charting the hardware progress and providing roadmaps for future development. However, quantum benchmarking is an evolving field that is still an open area of research. At Quantinuum we believe in testing the limits of our machine with a variety of different benchmarks to learn as much as possible about the errors present in our system and how they affect different circuits. We are open to working with the larger community on refining benchmarks and creating new ones as the field evolves.
To learn more about the Algorithmic Qubits benchmark and the issues with it, please watch this video where Dr. Charlie Baldwin walks us through the details, starting at 32:40.
About Quantinuum
Quantinuum, the world’s largest integrated quantum company, pioneers powerful quantum computers and advanced software solutions. Quantinuum’s technology drives breakthroughs in materials discovery, cybersecurity, and next-gen quantum AI. With over 500 employees, including 370+ scientists and engineers, Quantinuum leads the quantum computing revolution across continents.
Quantinuum is excited to announce the release of InQuanto™ v4.0, the latest version of our advanced quantum computational chemistry software. This update introduces new features and significant performance improvements, designed to help both industry and academic researchers accelerate their computational chemistry work.
If you're new to InQuanto or want to learn more about how to use it, we encourage you to explore our documentation.
InQuanto v4.0 is being released alongside Quantinuum Nexus, our cloud-based platform for quantum software. Users with Nexus access can leverage the `inquanto-nexus` extension to, for example, take advantage of multiple available backends and seamless cloud storage.
In addition, InQuanto v4.0 introduces enhancements that allow users to run larger chemical simulations on quantum computers. Systems can be easily imported from classical codes using the widely supported FCIDUMP file format. These fermionic representations are then efficiently mapped to qubit representations, benefiting from performance improvements in InQuanto operators. For systems too large for quantum hardware experiments, users can now utilize the new `inquanto-cutensornet` extension to run simulations via tensor networks.
These updates enable users to compile and execute larger quantum circuits with greater ease, while accessing powerful compute resources through Nexus.
Quantinuum Nexus
InQuanto v4.0 is fully integrated with Quantinuum Nexus via the `inquanto-nexus` extension. This integration allows users to easily run experiments across a range of quantum backends, from simulators to hardware, and access results stored in Nexus cloud storage.
Results can be annotated for better searchability and seamlessly shared with others. Nexus also offers the Nexus Lab, which provides a preconfigured Jupyter environment for compiling circuits and executing jobs. The Lab is set up with InQuanto v4.0 and a full suite of related software, enabling users to get started quickly.
Enhanced Operator Performance
The `inquanto.mappings` submodule has received a significant performance enhancement in InQuanto v4.0. By integrating a set of operator classes written in C++, the team has increased the performance of the module past that of other open-source packages’ equivalent methods.
Like any other Python package, InQuanto can benefit from delegating tasks with high computational overhead to compiled languages such as C++. This prescription has been applied to the qubit encoding functions of the `inquanto.mappings` submodule, in which fermionic operators are mapped to their qubit operator equivalents. One such qubit encoding scheme is the Jordan-Wigner (JW) transformation. With respect to JW encoding as a benchmarking task, the integration of C++ operator classes in InQuanto v4.0 has yielded an execution time speed-up of two and a half times that of open-source competitors (Figure 1).
Figure 1. Performance comparison of Jordan Wigner (JW) operator mappings for LiH molecule in several basis sets of increasing size.
This is a substantial increase in performance that all users will benefit from. InQuanto users will still interact with the familiar Python classes such as `FermionOperator` and `QubitOperator` in v4.0. However, when the `mappings` module is called, the Python operator objects are converted to C++ equivalents and vice versa before and after the qubit encoding procedure (Figure 2). With future total integration of C++ operator classes, we can remove the conversion step and push the performance of the `mappings` module further. Tests, once again using the JW mappings scheme, show a 40 times execution time speed-up as compared to open-source competitors (Figure 1).
Figure 2. Representation of the conversion step from Python objects to C++ objects in the qubit encoding processes handled by the `inquanto.mappings` submodule in InQuanto v4.0.
Efficient classical pre-processing implementations such as this are a crucial step on the path to quantum advantage. As the number of physical qubits available on quantum computers increases, so will the size and complexity of the physical systems that can be simulated. To support this hardware upscaling, computational bottlenecks including those associated with the classical manipulation of operator objects must be alleviated. Aside from keeping pace with hardware advancements, it is important to enlarge the tractable system size in situations that do not involve quantum circuit execution, such as tensor network circuit simulation and resource estimation.
Leveraging Tensor Networks
Users with access to GPU capabilities can now take advantage of tensor networks to accelerate simulations in InQuanto v4.0. This is made possible by the `inquanto-cutensornet` extension, which interfaces InQuanto with the NVIDIA® cuTensorNet library. The `inquanto-cutensornet` extension leverages the `pytket-cutensornet` library, which facilitates the conversion of `pytket` circuits into tensor networks to be evaluated using the NVIDIA® cuTensorNet library. This extension increases the size limit of circuits that can be simulated for chemistry applications. Future work will seek to integrate this functionality with our Nexus platform, allowing InQuanto users to employ the extension without requiring access to their own local GPU resources.
Here we demonstrate the use of the `CuTensorNetProtocol` passed to a VQE experiment. For the sake of brevity, we use the `get_system` method of `inquanto.express` to swiftly define the system, in this case H2 using the STO-3G basis-set.
from inquanto.algorithms import AlgorithmVQE
from inquanto.ansatzes import FermionSpaceAnsatzUCCD
from inquanto.computables import ExpectationValue, ExpectationValueDerivative
from inquanto.express import get_system
from inquanto.mappings import QubitMappingJordanWigner
from inquanto.minimizers import MinimizerScipy
from inquanto.extensions.cutensornet import CuTensorNetProtocol
fermion_hamiltonian, space, state = get_system("h2_sto3g.h5")
qubit_hamiltonian = fermion_hamiltonian.qubit_encode()
ansatz = FermionSpaceAnsatzUCCD(space, state, QubitMappingJordanWigner())
expectation_value = ExpectationValue(ansatz, qubit_hamiltonian)
gradient_expression = ExpectationValueDerivative(
ansatz, qubit_hamiltonian, ansatz.free_symbols_ordered()
)
protocol_tn = CuTensorNetProtocol()
vqe_tn = (
AlgorithmVQE(
objective_expression=expectation_value,
gradient_expression=gradient_expression,
minimizer=MinimizerScipy(),
initial_parameters=ansatz.state_symbols.construct_zeros(),
)
.build(protocol_objective=protocol_tn, protocol_gradient=protocol_tn)
.run()
)
print(vqe_tn.generate_report()["final_value"])
# -1.136846575472054
The inherently modular design of InQuanto allows for the seamless integration of new extensions and functionality. For instance, a user can simply modify existing code using `SparseStatevectorProtocol` to enable GPU acceleration through `inquanto-cutensornet`. It is worth noting that the extension is also compatible with shot-based simulation via the `CuTensorNetShotsBackend` provided by `pytket-cutensornet`.
“Hybrid quantum-classical supercomputing is accelerating quantum computational chemistry research,” said Tim Costa, Senior Director at NVIDIA®. “With Quantinuum’s InQuanto v4.0 platform and NVIDIA’s cuQuantum SDK, InQuanto users now have access to unique tensor-network-based methods, enabling large-scale and high-precision quantum chemistry simulations.”
Classical Code Interface
As demonstrated by our `inquanto-pyscf` extension, we want InQuanto to easily interface with classical codes. In InQuanto v4.0, we have clarified integration with other classical codes such as Gaussian and Psi4. All that is required is an FCIDUMP file, which is a common output file for classical codes. An FCIDUMP file encodes all the one and two electron integrals required to set up a CI Hamiltonian. Users can bring their system from classical codes by passing an FCIDUMP file to the `FCIDumpRestricted` class and calling the `to_ChemistryRestrictedIntegralOperator` method or its unrestricted counterpart, depending on how they wish to treat spin. The resulting InQuanto operator object can be used within their workflow as they usually would.
Exposing TKET Compilation
Users can experiment with TKET’s latest circuit compilation tools in a straightforward manner with InQuanto v4.0. Circuit compilation now only occurs within the `inquanto.protocols` module. This allows users to define which optimization passes to run before and/or after the backend specific defaults, all in one line of code. Circuit compilation is a crucial step in all InQuanto workflows. As such, this structural change allows us to cleanly integrate new functionality through extensions such as `inquanto-nexus` and `inquanto-cutensornet`. Looking forward, beyond InQuanto v4.0, this change is a positive step towards bringing quantum error correction to InQuanto.
Conclusion
InQuanto v4.0 pushes the size of the chemical systems that a user can simulate on quantum computers. Users can import larger, carefully constructed systems from classical codes and encode them to optimized quantum circuits. They can then evaluate these circuits on quantum backends with `inquanto-nexus` or execute them as tensor networks using `inquanto-cutensornet`. We look forward to seeing how our users leverage InQuanto v4.0 to demonstrate the increasing power of quantum computational chemistry. If you are curious about InQuanto and want to read further, our initial release blogpost is very informative or visit the InQuanto website.
How to Access InQuanto
If you are interested in trying InQuanto, please request access or a demo at inquanto@quantinuum.com
In July, we proudly introduced the Beta version of Quantinuum Nexus, our comprehensive quantum computing platform. Designed to provide an exceptional experience for managing, storing, and executing quantum workflows, Nexus offers unparalleled integration with Quantinuum’s software and hardware.
What’s New?
Before July, Nexus was primarily available to our internal researchers and software developers, who leveraged it to drive groundbreaking work leading to notable publications such as:
- Protecting expressive circuits with a quantum error detection code
- Volumetric Benchmarking of Error Mitigation with Qermit
- Platinum-based Catalysts for Oxygen Reduction Reaction simulated with a Quantum Computer
Following our initial announcement, we invited external users to experience Nexus for the first time.
We selected quantum computing researchers and developers from both industry and academia to help accelerate their work and advance scientific discovery. Participants included teams from diverse sectors such as automotive and energy technology, as well as research groups from universities and national laboratories worldwide. We also welcomed scientists and software developers from other quantum computing companies to explore areas ranging from physical system simulation to the foundations of quantum mechanics.
The feedback and results from our trial users have been exceptional. But don’t just take our word for it—read on to hear directly from some of them:
At Unitary Fund, we leveraged Nexus to study a foundational question about quantum mechanics. The quantum platform allowed us to scale experimental violations of Local Friendliness to a more significant regime than had been previously tested. Using Nexus, we encoded Extended Wigner’s Friend Scenarios (EWFS) into quantum circuits, running them on state-of-the-art simulators and quantum processors. Nexus enabled us to scale the complexity of these circuits efficiently, helping us validate LF violations at larger and larger scales. The platform's reliability and advanced capabilities were crucial to extending our results, from simulating smaller systems to experimentally demonstrating LF violations on quantum hardware. Nexus has empowered us to deepen our research and contribute to foundational quantum science.
Read the publication here: Towards violations of Local Friendliness with quantum computers.
At Phasecraft we are designing algorithms for near term quantum devices, identifying the most impactful experiments to run on the best available hardware. We recently implemented a series of circuits to simulate the time dynamics of a materials model with a novel layout, exploiting the all-to-all connectivity of the H series. Nexus integrated easily with our software stack, allowing us to easily deploy our circuits and collect data, with impressive results. We first tested that our in-house software could interface with Nexus smoothly using the syntax checker as well as the suite of functionality available through the Nexus API. We then tested our circuits on the H1 emulator, and it was straightforward to switch from the emulator to the hardware when we were ready. Overall, we found nexus a straightforward interface, especially when compared with alternative quantum hardware access models.
Quantum Software Lab, University of Edinburgh
In this project, we performed the largest verified measurement-based quantum computation to date, up to the size of 52 vertices, which was made possible by the Nexus system. The protocol requires complex operations intermingling classical and quantum information. In particular, Nexus allows us to demonstrate our protocol that requires complex decisions for every measurement shot on every node in the graph: circuit branching, mid-circuit measurement and reset, and incorporating fresh randomness. Such requirements are difficult to deliver on most quantum computer frameworks as they are far from conventional gate-based BQP computations; however, Nexus can!
Read the publication here: On-Chip Verified Quantum Computation with an Ion-Trap Quantum Processing Unit
Onward and Upward
We are thrilled to announce that after these successes, Nexus is coming out of beta access for full launch. We can’t wait to offer Nexus to our customers to enable ground-breaking scientific work, powered by Quantinuum.
Register your interest in gaining access to the best full-stack quantum computing platform, today!
Our team is making progress on the path towards “non-Abelian” quantum computing, which promises both fault tolerance and significant resource savings.
Computing with non-Abelian anyons, which are a type of quasiparticle, is sought after as it offers an enticing alternative to some of the biggest challenges in mainstream quantum computing. Estimates vary, but some scientists have calculated that some of the trickiest parts, like T gates and magic states distillation, can take up to 90% of the computer’s resources (when running something such as Shor’s algorithm). The non-abelian approach to quantum computing could mitigate this issue.
In a new paper in collaboration with Harvard and CalTech, our team is one step closer to realizing fault-tolerant non-Abelian quantum computing. This paper is a follow-up to our recent work published in Nature, where we demonstrated control of non-Abelian anyons. This marks a key step toward non-Abelian computing, and we are the only company who has achieved this. Additionally, we are the only company offering commercially-available mid-circuit measurement and feed-forward capabilities, which will be vital as we advance our research in this area.
In this paper, our team prepared the ground state of the “Z3” toric code – meaning this special state of matter was prepared in qutrit (3 states) Hilbert space. Before now, topological order has only been prepared in qubit (2 states) Hilbert spaces. This allowed them to explore the effect of defects in the lattice (for the experts, this was the “parafermion” defect as well as the “charge-conjugation” defect. They then entangled two pairs of charge conjugation defects, making a Bell pair.
All these accomplishments are critical steppingstones towards the non-Abelian anyons of the “S3” toric code, which is the non-Abelian approach that promises both huge resource savings previously discussed because it (unlike most quantum error correction codes) provides a universal gate set. The high-fidelity preparation our team accomplished in this paper suggests that we are very close to achieving a universal topological gate set, which will be an incredible “first” in the quantum computing community.
This work is another feather in our cap in terms of quantum error correction (QEC) research, a field we are leaders in. We recently demonstrated a significant reduction in circuit error rates in collaboration with Microsoft, we performed high fidelity and fault-tolerant teleportation of logical qubits, and we independently demonstrated the first implementation of the Quantum Fourier Transform with error correction. We’ve surpassed the “break-even” point multiple times, recently doing so entangling 4 logical qubits in a non-local code. This latest work in non-Abelian QEC is yet another crucial milestone for the community that we have rigorously passed before anyone else.
This world-class work is enabled by the native flexibility of our Quantum Charge Coupled Device (QCCD) architecture and its best-in-class fidelity. Our world-leading hardware combined with our team of over 350 PhD scientists means that we have the capacity to efficiently investigate a large variety of error correcting codes and fault-tolerant methods, while supporting our partners to do the same. Fault tolerance is one of the most critical challenges our industry faces, and we are proud to be leading the way towards large scale, fault-tolerant quantum computing.