This article originally appeared on The Modern Scientist. Written by Brian N. Siegelwax.

Able to leap deep circuits in a single bound!

So, Infleqtion’s Superstaq is supposed to be able to solve all of my “current quantum needs.” Yeah, yeah, yeah, I’ll believe it when I see it. For my money, there’s only one benchmark that matters: Quantum Phase Estimation (QPE). It’s the algorithm we’re all waiting to run, but it generates sheer noise on real hardware. It’s just way too deep, and not in a thought-provoking philosophical way.

So, you think you can solve MY quantum needs, Infleqtion?

Game on.

2-qubit Quantum Phase Estimation (QPE)

Let’s start nice and gentle at 2 qubits. We’ve got 3 contenders racing neck-and-neck-and-neck on the 127-qubit ibm_brisbane with an ideal simulator showing us what right looks like.

Let’s meet our contestants, shall we?

• Superstaq

• Qiskit optimization_level=3

• Qiskit optimization_level=3 with M3 measurement error mitigation

But 2 qubits aren’t going to impress anyone. Are you impressed? Really? Okay, whew… you had me going there for a moment. Let’s add a qubit and see if we can generate some real excitement….

Now’s the time to make popcorn. You can thank me later.

3-qubit Quantum Phase Estimation (QPE)

You can add up these numbers and do some calculations and try to pick an early leader, but it’s honestly too close to call. I ran these on the Open Plan, so they marinated in the queue for a few hours each while ibm_brisbane’s calibrations continuously changed. Besides, no one really cares about 3-qubit QPE, anyway.

At this time, all 3 contenders have the same top 2 results, which is what we’re looking at.

4-qubit Quantum Phase Estimation (QPE)

Physicists like to tell us to “imagine” stuff, so imagine I’ve annotated this histogram in a way that’s really useful. Technically, all 3 have the same top 2 results. However, we’re looking at 011 and 010, and Superstaq is following the simulator noticeably more closely than Qiskit and Qiskit+M3. That might be an omen as we add more qubits.

I must pause briefly to write that I’m a fan of M3, and I’m surprised it’s not boosting Qiskit more. Maybe there are some unbeknownst-to-me factors contributing to that, but I used it in the same way I’ve always used it and how the documentation still says to use it. I looked up an old experiment, and M3 didn’t boost QPE back then either.

5-qubit Quantum Phase Estimation (QPE)

Uh-oh.

Qiskit seems to be lost. Both with and without M3, its top peak is 0011, which is wrong. The estimated phase would be wrong.

But Superstaq is hanging in there. Without the simulator as a guide, 0100 could be an issue. But 0110 is slightly higher. So, if we took the top 2 results, 0101 and 0110, we’re not too far from the simulator.

6-qubit Quantum Phase Estimation (QPE)

What the…?

Qiskit and Qiskit+M3 disappeared without even saying “goodbye.” There’s not even a trace of them where they’re supposed to be.

How rude.

But Superstaq is still hanging on. The top 2 results are 01011 and 01010, just like the simulator.

7-qubit Quantum Phase Estimation (QPE)

And Superstaq is still clinging on. The top result is 010101, like the simulator, and 010110 moves us in the right direction.

It looks like we received a text message from Qiskit, but now it’s heading toward 110101, which is still wrong.

See, I told you to make popcorn, didn’t I?

8-qubit Quantum Phase Estimation (QPE)

And Superstaq is still hanging on. The top peak is 0101011, just like the simulator, and 0101010 moves us in the correct direction.

I tried calling Qiskit to see where it disappeared to again, but it seems to be out of the coverage area. Unacceptable.

9-qubit Quantum Phase Estimation (QPE)

While this is admittedly becoming less convincing, the peaks are still correct. Using the top peak, 01010101, and the top neighboring peak, 01010110, allows us to at least be in the correct ballpark while estimating the phase.

We haven’t seen Qiskit in so long I’m starting to forget what it looks like.

10-qubit Quantum Phase Estimation (QPE)

I stopped here, but the top peak, 010101011, and the top neighboring peak, 010101010, are still following the simulator. It’s not convincing, but Superstaq is still correct.

Meanwhile, Qiskit and Qiskit+M3 dropped out of the race so long ago they must be halfway across China by now.

Hellinger Fidelity

Along with ecr count and depth, I was originally going to add an X multiplier for Superstaq’s Hellinger Fidelity over Qiskit’s Hellinger Fidelity. The Hellinger Fidelity compares the results to the ideal simulator’s results. However, the multiplier loses its meaning after the 4th qubit. Starting at 5 qubits, Qiskit is just wrong. A Hellinger Fidelity comparison suggests Superstaq is only somewhat better than Qiskit, but you can continue to estimate the phase with Superstaq up to 10 qubits. Therefore, Superstaq’s fidelity is on the bitstrings that matter, whereas Qiskit’s fidelity is on the bitstrings that don’t.

Conclusion

I ran the exact same circuits with Fire Opal almost a year ago. Back then I used the 16-qubit ibm_guadalupe, starting at 16 qubits and working backward from there. I would’ve done a head-to-head battle at this time, but Fire Opal doesn’t support ibm_brisbane yet, so that battle has to wait. I could’ve used a 27 supported by both, but I strenuously object in principle to paying to use anything that’s free. Both Superstaq and Fire Opal are currently free, but the 27s are not. The 7s are too small, so my testing had to be with ibm_brisbane. This way, if you think I’m full of [EXPLETIVE DELETED], you can run QPE yourself with Superstaq on the 127-qubit ibm_brisbane on an IBM Quantum Open Plan absolutely for free. It’ll consume a significant percentage of your allocation and test your patience, but I paid $0.00 to run this experiment and you can, too. If you choose to run this PayGo, I consumed 142 seconds of runtime, which at $1.60 per second would total $227.20.

If you follow the Fire Opal link and read my conclusion, you’ll find that Superstaq fared slightly better. In fairness to Fire Opal, ibm_brisbane is qualitatively better than ibm_guadalupe was. To Superstaq’s credit, however, it supports a qualitatively better device. Therefore, the best QPE results I can generate for free at this time are with Superstaq on ibm_brisbane.

It’s worth noting that Superstaq supported ibm_brisbane within days of the device switching to the Open Plan on or around September 26. I know because I was actively testing Superstaq on ibm_perth at the time, saw ibm_brisbane listed as a supported device, and submitted a test job immediately. As of publication on November 29, due to the retirement of the remaining 7s while I was writing this, Fire Opal is not actually available on any free IBM backends at the moment. Infleqtion, meanwhile, added support for the newly-open 127-qubit ibm_kyoto in less than 24 hours, so hours before the publication of this article Superstaq now supports 2 free 127s, although ibm_brisbane still seems to be the qualitatively better device.

This article originally appeared on The Modern Scientist. Written by Brian N. Siegelwax.