Get 20M+ Full-Text Papers For Less Than $1.50/day. Start a 14-Day Trial for You or Your Team.

Learn More →

Fair Sampling Error Analysis on NISQ Devices

Fair Sampling Error Analysis on NISQ Devices JOHN GOLDEN, Information Sciences (CCS-3) and Computational Earth Sciences (EES-16), Los Alamos National Laboratory, USA ANDREAS BÄRTSCHI, Information Sciences (CCS-3), Los Alamos National Laboratory, USA DANIEL O’MALLEY, Computational Earth Sciences (EES-16), Los Alamos National Laboratory, USA STEPHAN EIDENBENZ, Information Sciences (CCS-3), Los Alamos National Laboratory, USA We study the status of fair sampling on Noisy Intermediate Scale Quantum (NISQ) devices, in particular the IBM Q family of backends. Using the recently introduced Grover Mixer-QAOA algorithm for discrete optimization, we generate fair sampling circuits to solve six problems of varying difficulty, each with several optimal solutions, which we then run on twenty backends across the IBM Q system. For a given circuit evaluated on a specific set of qubits, we evaluate: how frequently the qubits return an optimal solution to the problem, the fairness with which the qubits sample from all optimal solutions, and the reported hardware error rate of the qubits. To quantify fairness, we define a novel metric based on Pearson’s χ test. We find that fairness is relatively high for circuits with small and large error rates, but drops for circuits with medium error rates. This indicates that structured errors dominate in this regime, while unstructured errors, which are random and thus inherently fair, dominate in noisier qubits and longer circuits. Our results show that fairness can be a powerful tool for understanding the intricate web of errors affecting current NISQ hardware. CCS Concepts: • Hardware → Quantum error correction and fault tolerance;• Computer systems organization → Quantum computing Additional Key Words and Phrases: Quantum alternating operator ansatz, fair sampling ACM Reference format: John Golden, Andreas Bärtschi, Daniel O’Malley, and Stephan Eidenbenz. 2022. Fair Sampling Error Analysis on NISQ Devices. ACM Trans. Quantum Comput. 3, 2, Article 8 (May 2022), 23 pages. https://doi.org/10.1145/3510857 1 INTRODUCTION Combinatorial optimization problems, such as the Minimum Traveling Salesperson problem, the Maximum Satisfiability problem, or the Spin Glass Ising model, are widely viewed to be one of the most promising application domains for quantum computing. Such problems often have multiple optimal solutions, and depending on the algorithm being used to solve the problem, some optima may be much easier to find than others. This is problematic if one is interested not in finding a single solution, but instead determining the overall landscape of optimal solutions. Algorithms Authors’ addresses: J. Golden, Information Sciences (CCS-3) and Computational Earth Sciences (EES-16), Bldg. 1489, Casa Grande Drive, Los Alamos National Laboratory, Los Alamos, New Mexico, USA, 87545; email: golden@lanl.gov; A. Bärtschi and S. Eidenbenz, Information Sciences (CCS-3), Bldg. 508, Diamond Drive, Los Alamos National Laboratory, Los Alamos, New Mexico, USA, 87545; emails: {baertschi, eidenben}@lanl.gov; D. O’Malley, Computational Earth Sciences (EES-16), Bldg. 1400, Casa Grande Drive, Los Alamos National Laboratory, Los Alamos,New Mexico, USA, 87545; email: omalled@lanl.gov. This work is licensed under a Creative Commons Attribution International 4.0 License. © 2022 Copyright held by the owner/author(s). 2643-6817/2022/05-ART8 https://doi.org/10.1145/3510857 ACM Transactions on Quantum Computing, Vol. 3, No. 2, Article 8. Publication date: May 2022. 8:2 J. Golden et al. which fairly sample from the space of optimal solutions avoid this problem and have practical value in physics and engineering. For instance, it is often difficult in practice to encode all the engineering design goals in the function to be optimized. By allowing an engineer to explore an unbiased assortment of optimal solutions, fair sampling provides an opportunity to apply these other design goals to a more diverse set of options (especially useful when a customer adds one more requirement after the optimization has been performed). It can also be used to produce an ensemble of predictions from calibrated physical models that can be used to make a forecast in- cluding a range of possible outcomes (e.g., in the context of fluid flow in the earth’s subsurface, which can be formulated as a discrete optimization problem [14, 24]). In this paper, we determine the current status of fair sampling in the context of quantum comput- ing. Specifically, we are interested both in algorithms which theoretically guarantee fair sampling (i.e., generate quantum states where all optimal solutions have the same amplitudes squared and thus are measured with equal probability), as well as actual hardware implementations of such algorithms. Previous studies have shown that quantum annealing cannot be used as a fair sampler for all problems [18]. We therefore focus our work on gate-based quantum computing algorithms and hardware. The most widely-used quantum optimization framework is the Quantum Alternating Oper- ator Ansatz (QAOA), which alternates between a mixer operator that mixes the amplitudes of all feasible solutions and a phase separator operator that separates the phases of the feasible so- lutions [13]. Most mixers do not exhibit fair sampling, however, the recently discovered Grover Mixer [4] has the property that all feasible solutions with the same objective value have identical amplitudes. This novel Grover Mixer implementation of QAOA thus solves the theoretical side of fair sampling for gate-based quantum computing. The majority of this work is dedicated to a practical evaluation of how fair sampling circuits are affected by the errors inherent in Noisy Intermediate Scale Quantum (NISQ) devices. In this study, a paradox quickly becomes apparent: while an error-corrected quantum device could obviously be used as a perfectly fair sampler, so could a quantum device completely dominated by random noise. Of course the noisy machine would be no different than a random number generator, and would sample from all feasible solutions rather than just the optimal solutions, but it would do so fairly. Therefore, our study seeks to clarify where current NISQ devices lie on the spectrum between these two extremes. Specifically, are the hardware errors predominantly random and unstructured, thus leading to relatively fair sampling of optimal solutions? Or do hardware biases and other structured errors play an important role, thus leading to unfair behavior? Our methodology for answering these questions consists of the following steps: (1) We define six example problems, which are a slightly expanded set of examples studied in the context of fair sampling in quantum annealing [18]. (2) We design quantum circuits for our problems and hand-tune them to minimize circuit depth and gate counts, taking into account different qubit connectivity topologies as encountered on IBM Q. Our circuits involve two to five qubits and our medium size examples result in circuits with gate counts in the 30–60 range, which is similar to the size of the (random) circuits used to establish the Quantum Volume metric on the IBM Q backends. (3) We execute our circuits in a large number of shots on twenty different IBM Q backends that have qubit counts of up to 65 qubits. In addition, we embed our circuits in all possible topologies on the circuits resulting in a total of 7,440 experiments at 40,960 shots each. (4) We define a novel metric for fairness in quantum computing based on Pearson’s χ test, a common statistical tool to evaluate how likely it is that a set of observational data came from a null hypothesis distribution. Since all NISQ hardware will necessarily involve some biases ACM Transactions on Quantum Computing, Vol. 3, No. 2, Article 8. Publication date: May 2022. Fair Sampling Error Analysis on NISQ Devices 8:3 and imperfections that inhibit fair sampling, we are interested in how much effort must be expended to prove that a piece of hardware is biased, i.e., reject the null hypothesis of fair sampling. We term this metric ‘number of shots to reject fair sampling,’ and it depends on both a specific set of qubits as well as a fair sampling circuit. This fairness metric is a separate contribution to the main results of the paper. (5) We analyze our experiments using our fairness metric and we calculate the aggregate error of each experiment from IBM’s reported gate-specific error rates. We find that the NISQ devices under evaluation provide noticeably unfair sampling. However, fairness increases as aggregate error approaches both 0 and 1, matching with the theoretical ex- tremes of error-corrected and noise-dominated devices discussed previously. In the case of small aggregate error, this is to be expected, as for 0 error we would retrieve the theoretical guarantees of Grover Mixer QAOA. Likewise, for large aggregate error, one should get samples uniformly at random from all computational basis states. Nonetheless, the result is perhaps counter-intuitive, as one might expect a linear interpolation between these two extremes. For example, the hardware- efficient random circuits in Google’s recent quantum supremacy experiment [ 2] give a sample dis- tribution consisting of a convex combination of an ideal (error-free) Porter-Thomas distribution and a Uniform distribution (caused by errors). Our differing results indicate that we should not ex- pect errors themselves to be evenly distributed (i.e., in a fair manner) across qubits for highly struc- tured circuits such as ours. Indeed, a previous study found that structured errors (e.g., crosstalk, coherent noise) play a large role in the performance of structured circuits [25]. Our result holds for our selected set of circuits and across the twenty NISQ backends on IBM Q. We propose that fair sampling be an addition in the emerging set of benchmarks for NISQ de- vices, which so far have largely focused on circuit correctness. As we show in this work, biased errors can artificially inflate circuit accuracy, and fair sampling on its own can be fooled by devices dominated by random noise. However, by studying both accuracy and fairness, we gain a more complete picture of the type and magnitude of errors affecting NISQ hardware. In summary, the main contributions of our work are: (1) We propose a novel methodology for quantifying the fairness of sampling on NISQ hardware. (2) We elucidate two noise regimes (high noise and low noise) that result in fair sampling, and a third regime (intermediate noise) that is dominated by biased errors. (3) In practice, this approach provides a way to determine whether near-term improvements that reduce noise will improve or degrade the performance of fair sampling for a given problem. (4) As an example, we study how a practical error mitigation technique (namely, measurement error mitigation) impacts the quality of sampling on IBM Q hardware. 2REVIEW In this section we describe previous studies of fair sampling in the context of quantum computing. A note on nomenclature: since the optimization problems we study can all be phrased in terms of Ising models, we adopt the more physics-oriented language and refer to ‘optimal solutions’ as ground states. Problems with multiple optimal solutions are thus said to have degenerate ground states. 2.1 Fair Sampling in Quantum Annealing After quantum annealing was proposed as a possibly useful optimization heuristic [16], it was dis- covered that naïve approaches to quantum annealing do not always sample degenerate ground states fairly [20]. In fact, certain Hamiltonians feature ground states that are sampled with ACM Transactions on Quantum Computing, Vol. 3, No. 2, Article 8. Publication date: May 2022. 8:4 J. Golden et al. probability approaching zero as annealing time increases. This effect is inherent to the quantum an- nealing approaches, and is not the consequence of hardware noise or other technical limitations. This behavior was subsequently verified experimentally [ 19]. Since unfair sampling (especially when certain ground states are strongly suppressed) can have negative consequences for many applications, techniques were proposed to improve upon the naïve quantum annealing approach [17, 18, 27, 34]. None of these techniques have been tested on quantum annealing hardware, in- dicating some of the challenges with these approaches. It should also be noted that it has been proposed that there are some advantages of biased samplers, that is, they can be used effectively in an ensemble of samplers each of which has different biases [ 36]. One technique that has been proposed to improve the fairness of quantum annealing is the introduction of more complex Hamiltionians to drive the annealing process [19]. This generally improves fairness, but is hard to implement and does not resolve the underlying problem. Even for small problems, biases sometimes remain despite using fairly high-order Hamiltonians [18]. In studying these higher-order Hamiltonians, Könz et al. [18] introduced four Ising models that are suitable for our QAOA analyses. The problems are described in Table 1 as Problems (a)–(d). These problems all exhibit biased sampling with quantum annealing, so they present a non-trivial challenge for fair sampling algorithms. Two of them involve 4 qubits, one involves 5 qubits, and another involves 6 qubits. This small number of qubits make it plausible that these problems can be solved with reasonably high fidelity on current NISQ hardware. Impact of noise: Recent work [29] achieved high-quality thermal Gibbs Sampling with the D- Wave 2000Q system for a restricted class of Ising Hamiltonians that are native to the hardware connectivity and have coupling strengths J ∈{−1, 0, 1}. This category includes Hamiltonians ij with degenerate ground states, but not the more general Hamiltonians studied in this paper (see Table 1). Fair Sampling of those ground states is achieved by re-scaling the input energy scale to a sweet spot that is large enough to be noise-resilient but small enough to overcome the biases from the transverse field. In the identified sweet spot energy scale, changing the annealing time influences the effective temperature of the Gibbs samples generated by the hardware, and thus the probability to find ground states. 2.2 Fair Sampling in the antum Qu Alternating Operator Ansatz The Quantum Alternating Operator Ansatz [13] (QAOA) is a promising heuristic quantum algorithm for (combinatorial) optimization problems. In its essence, for a problem instance I with feasible states F and cost Hamiltonian H on n qubits, a p-round QAOA prepares a parametrized state from which one would like to sample low-energy states with respect to H : β γ ββ,γγ := U (β )U (γ )···U (β )U (γ )U ↑ . (1) M p P p M 1 P 1 S The circuit consists of an initial state preparation unitary operator U that creates some super- position of all feasible solutions in I, followed by p applications of alternating parametrized phase separating and mixing unitaries U (γ ), U (β ) with real angle parameters γγγ = (γ ,...,γ ) and P k M k 1 p βββ = (β ,..., β ) , and a final measurement in the computational basis, see e.g., Figure 1.The βββ,γγγ 1 p angles are traditionally found via a hybrid classical/quantum approach: given some initial parame- ters, take enough samples to reliably estimate the expectation value βββ,γγγ H βββ,γγγ , use a classical optimizer to adjust the parameters, and repeat until one has a state with a good (low-energy) expec- tation value. For further description of, and theoretical justifications for this approach, see [ 6, 8, 31]. The role of the phase separating unitaries U (γ ) is to add multiplicative phase factors to the am- plitudes of feasible computational basis states, with phases proportional to respective energies. We −iγ H usually have (up to global phases) U (γ )  e . For many problems, such as MaxCut [8]orthe problems considered in this paper, H is an Ising Hamiltonian with quadratic and sometimes linear ACM Transactions on Quantum Computing, Vol. 3, No. 2, Article 8. Publication date: May 2022. Fair Sampling Error Analysis on NISQ Devices 8:5 Fig. 1. Grover Mixer QAOA: The state preparation unitary U for an equal superposition of all feasible states † 10 1 −β /π |F = √ |x, its conjugate transpose U , and a multi-controlled phase-shift gate Z = x∈F −iβ S 0 e |F| −iβ|FF| are used to implement the mixer U (β) = e . terms; but in other problems, H can also involve higher-order terms, such as for MaxE3Lin2 [9]. The interplay between the state preparation unitary U and the mixing unitary U is particularly S M important, and falls into three categories: Original Approach. The original approach by Farhi et al. [8], termed Quantum Approxi- mate Optimization Algorithm, was inspired by quantum annealing for unconstrained op- ⊗n timization problems, i.e., U = H (preparing an equal superposition of all states) and −iβ X U (β) = e (a transverse field of Pauli-X operators acting on individual qubits). General Ansatz. The general framework introduced in Equation (1) by Hadfield et al. [ 13]is a generalization also suitable for a wide variety of constrained optimization problems. They focus on designing involved mixing unitariesU (β) and usually splitU in a layer of Identity M S and Pauli-X gates preparing some feasible computational basis state, followed by an initial mixing unitary U (β ) with new angle β . M 0 0 Grover Mixer. If one can design efficient state preparation unitaries U preparing an equal superposition of all feasible basis states |F = 1/ |F| |x, then this gives rise to a x∈F mixing unitary resembling Grover’s selective phase-shift operator [12, 22, 35], U (β) = −iβ|FF| −iβ −iβ e = Id− (1− e ) |FF| = U (Id− (1− e ) |↑↑|)U ,see Figure 1.Insomesense, this is the reverse of the method in the general ansatz, and it was shown to be applicable to many relevant optimization problems [4], but there also exists a range of constrained problems where it is ruled out under complexity-theoretic assumptions [5]. Although the Quantum Alternating Operator Ansatz is the most general framework of the three, both the Quantum Approximate Optimization Algorithm as well as Grover Mixer QAOA have their advantages. For example, the original approach by definition cannot do worse than a Trotterization of the adiabatic algorithm [10], thus converging to a ground state in the p →∞ limit [8]. Further- more, its phase separating and mixing operators are periodic, restricting the angles to real values in [−π, π ). Grover Mixer QAOA retrieves these properties for constrained optimization problems with efficient state preparation unitaries U [5], but for unconstrained optimization problems the −iβ|FF| −iβ X Grover Mixer e has a larger circuit depth than the Transverse Field Mixer e .Inthis paper we explore an advantage unique to Grover Mixer QAOA among the three categories: fair sampling. With Grover Mixer QAOA, all feasible basis states begin with amplitude 1/ |F| (after state −iγ H preparation with U ). The phase separating unitary U (γ ) = e then phases the amplitude of S P ACM Transactions on Quantum Computing, Vol. 3, No. 2, Article 8. Publication date: May 2022. 8:6 J. Golden et al. every basis state proportional to its energy and γ , keeping the same phase for basis states of same −iβ energy. The mixing unitary U (β) = Id − (1 − e ) |FF| then deducts from all amplitudes of −iβ their input state ψ the same weighted average of all of its amplitudes, ((1− e )/ |F|)F | ψ. Therefore, basis states with the same energy are sampled with the same amplitude. This property holds for all feasible states (not only ground states) and is independent of the number of QAOA levels or the choice of angles for U ,U . For a complete proof, see [4]. M P Impact of noise: While the original based Quantum Approximate Optimization Algorithm of- fers non-trivial provable performance guarantees already for a single for certain problems such as MaxCut on ( d=3)-regular graphs [8], a higher number of rounds – p ≥ 6[33]oreven p ∈ Ω(log(n)/d) [3] – might be necessary to outperform the best-known classical approximation ratios. However, simulations considering realistic noise models of current hardware show that perfor- mance in terms of approximation ratio may degrade even for p = 2[1]. Noise leading to a decrease in performance has also been observed for growing problem sizes n, with the exception of problem graphs matching the hardware connectivity [15]. For the latter setting, experiments on actual hard- ware have shown an approximation ratio increase up to p = 3(mean)and p = 4 (certain instances), but no further [15]. Furthermore, even with constantly improving hardware, complications may arise in the optimization of the variational angles β,γ due to noise-induced barren plateaus [30]. All of these results refer to transverse field based QAOA, which does not have the property of sampling fairly among ground states. For Grover Mixer QAOA, we have a theoretical guarantee of fair sampling for ideal noise-less devices. At the same time we have a larger circuit depth of the mixer, asymptotically matching the circuit depth of the phase separator [4]. Hence, for current NISQ devices, we restrict ourselves to a single-round Grover Mixer QAOA, analysing the impact of noise on both ground state probability and fair sampling among ground states. 2.3 Description of Test Circuits In addition to the four problems from [18], we additionally studied two simpler Ising models with degenerate ground states but only involving two and three qubits. While these models do not feature ground state suppression on quantum annealing hardware, we include them in this study because they represent the simplest non-trivial models with degenerate ground states and thus serve as useful baselines as compared to the more complicated models of [18]. Allofthe models we studied are depicted in Table 1. Our general procedure for generating test circuits was to begin with a 1-level Grover Mixer QAOA algorithm with q fixed to ↑. We chose a 1-level implementation of Grover-QAOA in order to keep circuit depth low. Following [18], we fix q :=↑ as all of the models in Table 1 are ↑/↓ symmetric. This allows us to reduce the problems from Ising Hamiltonians H with only quadratic terms acting on qubits q ,...,q to new Hamiltonians H with some linear terms acting only on 0 n−1 qubits q ,...,q . This also transforms q into a classical control bit for the consecutive Grover 1 n−1 0 Mixer, which can be removed. Thus, we can embed Ising problems on n qubits onto circuits with only n − 1 qubits. See Figure 9 in Appendix C for a visual representation of a generic 1-level Grover Mixer QAOA for an unconstrained optimization problem, along with a description of the simplifications resulting from fixing q . We then compiled the circuits to match connectivity graphs and gates available on the IBM Q backend. Instead of employing the tools available in IBM’s qiskit software, we compiled the circuits by hand. This had two benefits. First, this reduced circuit depth by roughly a factor of two as compared to qiskit. Second, it allowed us to generate circuits for architectures that can be found as subgraphs of every hardware connectivity graph among the IBM Q backends, see ACM Transactions on Quantum Computing, Vol. 3, No. 2, Article 8. Publication date: May 2022. Fair Sampling Error Analysis on NISQ Devices 8:7 Table 1. Ising Models with Degenerate Ground States to Be Studied on NISQ Hardware Problems (a)–(d) are from [18]. All of the Ising models H = − J Z Z have only quadratic terms and no linear C ij i j terms. The dark red edges indicate a ferromagnetic J =+2 coupling and the light red edges a ferromagnetic J =+1 ij ij coupling; the light blue edges represent an antiferromagnetic J = −1 coupling and the dark blue edges an ij antiferromagnetic J = −2 coupling. Only ground states with q =↑ are listed as all models are symmetric under ij 0 simultaneous ↑ /↓-flips. Note that a symmetry-breaking fixed setting q :=↑ results in a Hamiltonian H on qubits q , ..., q (without q ) with some linear terms. 1 n 0 Figure 2. Thus, the same exact circuit could be evaluated across many backends, and in many instances across many different qubits on the same backend. For Problems (a) and (b), which involve five variables (and with q :=↑ require only 4 qubits) we generated three distinct circuits, each on a different architecture: 4T, 4L, and 5T (see Figure 2 for notation). The 5T circuits for these problems employ an ancilla qubit, which allows us to discard any sample in which the ancilla was not measured in the↑ state. The remaining Problems (c)–(f) each only have a single circuit associated with them. Note that Problem (f), with the Hamiltonian Z Z and ground states |↑↓ and |↓↑, is uniquely simple. In this case we do not fix q :=↑,nor 0 1 0 do we employ Grover-QAOA, as the problem can be “solved” with an almost-trivial circuit of a ACM Transactions on Quantum Computing, Vol. 3, No. 2, Article 8. Publication date: May 2022. 8:8 J. Golden et al. Fig. 2. The hardware architectures we transpile our circuits to: Linear Nearest Neighbor connectivities on 2,3,4 qubits (2L,3L,4L), as well as T-shaped connectivities on 4,5 qubits (4T,5T). These architectures form connectivity subgraphs of all IBM Q devices. By fixing q :=↑,each n-qubit problem in Table 1 can be mapped to a (n − 1)-qubit architecture. Hadamard on qubit q ,a X-gate on qubit q , and a consecutive CNOT with control q and target 0 1 0 q . Due to the small number of qubits and gates, we use this circuit to explore the highest reaches of accuracy and fairness capable with the hardware. See Table 2 for a brief description of all of the circuits, including their architectures, gate count, and βββ, γγγ parameters. We describe our compilation technique in detail in Appendix C,and give QASM representations of our circuits in the ancillary file associated to this note. 3 METHODOLOGY We evaluated the previously described set of circuits across a wide swath of hardware on the IBM Q platform. Furthermore, we collected data in two large sets, one in October 2020 and the other in August 2021, allowing for an analysis of the change in fairness of the IBM Q platform over time. Since our interest was not in identifying the best performing qubits, but rather observing trends in circuit correctness vs. fairness, we evaluated the same circuit many times per chip, each time using a different subset of qubits. For example, testing the circuit for Problem (e), which has only 2 qubits, on a chip with 5 qubits in an LNN architecture would involve four separate evaluations (on the pairs of qubits {q ,q } for i = 1... 4). The machines we tested, along with the number of i i+1 qubit subsets for each architecture, are listed in Table 3. It is important to emphasize that there was no additional transpilation or optimization of these circuits beyond that discussed in the previous section. We collected 40,960 shots for each circuit on each set of applicable qubits, which gave us suffi- cient data to achieve repeatable statistics. None of the hardware consistently reported fair results, i.e., hardware noise introduced significant biases for even the simplest circuits. In order to charac- terize how fair a given set of qubits performed with a given circuit, we adopted the metric: how many shots does one need of a given circuit on a given set of qubits in order to reject the hypothe- sis that these qubits are fair samplers at 95% significance level? This is equivalent to the question of: how many flips (on average) does it take to show that a coin is unfair? A 90/10 coin will take relatively few flips to reveal its bias, whereas a 51/49 coin will take many flips. We call this metric “number of shots to reject fair sampling.” To calculate the number of shots to reject fair sampling, we utilized Pearson’s χ test, which evaluates how likely a set of observations {O } is given some expected null hypothesis values {E }. If a given circuit has d degenerate ground states, and n is the number of ground state i д.s . observations (out of the 40,960 total shots), our null hypothesis is that д.s . E = E = . (2) The χ value is determined via (O − E) χ = . (3) ACM Transactions on Quantum Computing, Vol. 3, No. 2, Article 8. Publication date: May 2022. Fair Sampling Error Analysis on NISQ Devices 8:9 Table 2. Details for 1-round Grover-QAOA Fair Sampling Circuits Studied on NISQ Hardware β γ β γ Prob. Arch. #Rot. #CNOT (β, γ ) ββ,γγ H ββ,γγ GSP (a) 4L 57 35 (−1/2,−11/12) · π –2.682 out of –4 0.498 (a) 4T 51 29 (−1/2,−11/12) · π –2.682 out of –4 0.498 (a) 5T 47 25 (−1/2,−11/12) · π –2.682 out of –4 0.498 (b) 4L 59 39 (−11/15,−17/60) · π –4.228 out of –5 0.846 (b) 4T 53 32 (−11/15,−17/60) · π –4.228 out of –5 0.846 (b) 5T 49 29 (−11/15,−17/60) · π –4.228 out of –5 0.846 (c) 5T 90 62 (−23/60, 1/15) · π –1.563 out of –4 0.215 (d) 3L 26 14 (−5/12, 1/10) · π –1.319 out of –2 0.702 (e) 2L 16 4 (−23/60, 3/5) · π –0.999 out of –1 1.000 (f) 2L 2 1 N/A N/A 1.000 For each circuit, we list the problem from Table 1 being solved, the qubit connectivity from Figure 2 employed, the total number of single-qubit (i.e. rotation) gates, the total number of CNOT gates, the βββ, γγγ parameters used, the expectation value βββ, γγγ H βββ, γγγ , and the probability of selecting a ground state (GSP). The null hypothesis is rejected at a significance level based on the value of χ as compared against the upper-tail critical values of the χ distribution k /2−1 −x /2 x e f (x, k) = , (4) k /2 2 Γ(k/2) where k is the number of degrees of freedom in the observed data. For example, with five degrees of freedom a χ value of 11.070 indicates a rejection of the null hypothesis with 95% significance level. In principle, the value of χ is itself a measure of fairness for a set of observational data, however we found it a poor metric for our application. This is because the actual significance of a given χ value is dependent on the size of the data set as well as the number of degrees of freedom. Since our circuits reported widely varying numbers of ground state observations, and had different numbers of degenerate ground states, comparing χ values across circuits did not compare apples to apples. One could instead report the significance level at which the observational data rejects the fair sam- pling hypothesis, as that takes sample size and degrees of freedom into account. However, current hardware rejects the fair sampling hypothesis at significance levels so close to 100% that numerical differences between rejection levels were difficult to meaningfully interpret. Instead, determining how many samples from a distribution are necessary (on average) to reject fair sampling at 95% significance can be done with observational data of any size and degrees of freedom, and provides an intuitive measure of the fairness of the underlying qubits. A formal description of our algorithm for determining the number of shots to reject fair sam- pling can be found in Appendix B, here we describe the method in general terms. Based on our observational data of a given circuit C on a set of qubits Q, we determined the relative frequency of observing each ground state. We then generated 1,000 synthetic samples, each containing n ground state observations matching the relative frequency in the observed data. For each sample 2 1 we calculated χ , then took the median value for the set of 1,000 samples. We repeated for increas- ing values of n until we arrived at a median value of χ matching the level necessary to reject the We found that taking fewer than 1,000 samples resulted in inconsistent median values, and more then 1,000 samples resulted in slow performance. ACM Transactions on Quantum Computing, Vol. 3, No. 2, Article 8. Publication date: May 2022. 8:10 J. Golden et al. fair sampling hypothesis with 95% statistical significance given the number of degenerate ground states. As an example, with this methodology a coin biased 60/40 requires an average of 74 flips to reject fair sampling. 4 RESULTS As mentioned in the introduction, none of the qubits or circuits consistently reported fair sam- pling. Instead, we sought to understand the correlation between qubit fidelity and fair sampling. Our method for quantifying the degree of fair sampling for a circuit on a piece of hardware was de- scribed in the previous section. We used two complementary methods of quantifying qubit fidelity. The first is simply the frequency with which a given set of qubits accurately identify an optimal solution for the problem at hand. In this context, higher fidelity qubits are those that obtain a ground state more frequently. The benefit of this method is that it relies on direct observation of qubits solving a real problem, while the downside is that it can’t be used to directly compare results from different problems. Our second method for quantifying qubit fidelity is to use the hardware error rates as reported by qiskit. This method, which we will discuss in more detail later in this section, has the benefit of more accurately comparing results from different circuits. However, the practical relevance of this reported error data for our application is unclear. A broad issue for experimental tests of existing NISQ hardware is the highly stochastic nature of the results. We observed significant variance in all of our observed data, and therefore used a very large experiment count (with 7,440 distinct evaluations of the IBM Q hardware) to observe consistent trends. To be precise, each data point in the following plots is generated by specifying: a circuit to be evaluated, an IBM Q backend, and a subset of qubits on the backend which match the connectivity of the given circuit. We then collected 40,960 shots of the circuit on those specific qubits, and the results of those shots correspond to a single data point in the plots below. This large number of data points allowed us to identify repeatable positive or negative correlations, and we include lines of best fit to indicate these trends. 4.1 Ground State Probability vs. Fairness With that proviso out of the way, let us now discuss the results of directly comparing quantity (as measured by the number of ground state observations) vs. quality (as measured by the number of shots to reject fair sampling). We observed a positive correlation between ground state probability and fair sampling for three circuits, those solving the “easy” problems: (d), (e), (f), see Figure 3. Note that these circuits all have ≤3 qubits and circuit depths 40, 20, and 3, respectively. We call this “soft” unfairness, in that improving ground state probability and fairness of sampling are both accomplished by evaluating the circuit on the highest performing qubits possible. With the remaining circuits we observed more complicated behavior. For Problem (a), the 4T cir- cuit exhibits “soft” unfairness, the 4L circuit shows no variation in fairness with respect to ground state probability, and the 5T circuit has a negative correlation between ground state probability and fairness, see Figure 4. In other words, qubits that more frequently obtained ground states did so with increasing bias. All circuits for Problems (b) and (c) share this negative correlation. We call this “hard” unfairness, as improving ground state probability results in less fair results, and vice versa. We believe this “hard” unfairness comes from the fact that long circuits on noisy qubits produce increasingly random results. This means that the likelihood of selecting a ground state decreases, but the distribution across ground states is fairly even. To test this conjecture, we compared the fairness and ground state probabilities of both a “soft” and “hard” circuit as functions of the error rate of the qubits. ACM Transactions on Quantum Computing, Vol. 3, No. 2, Article 8. Publication date: May 2022. Fair Sampling Error Analysis on NISQ Devices 8:11 Fig. 3. Each data point represents 40,960 shots of the specified circuit on a fixed set of qubits on an IBM Q backend. The Quantum Volume (QV) of the backend is indicated by yellow for QV128, blue for QV32, green for QV16, and red for QV8. In these shorter circuits we observe a positive correlation (as indicated by the black lines of best fit) between ground state probability and number of shots to reject fair sampling. 4.2 Aggregate Error Rate vs. Fairness The IBM Q hardware reports the latest calibration data for a given chip at the time of evaluating a circuit. Specifically, qiskit reports the error rates for their native gateset (CX, SX, RX, and RZ) as well as individual qubit measurement error. We used this data to generate what we call an aggregate error of the chip and circuit in combination. For a circuit C acting on qubits q ,...,q , composed of a sequence of gates д ,...,д evaluated on a chip H, this aggregate error E is 1 C,H calculated via k l E = 1− (1− e ) (1− m ) , (5) C,H i i i=1 i=1 where e is the error for gate д ,andm is the measurement error for qubit q , as reported by qiskit i i i i at the time of evaluation. We believe this aggregate error metric evaluates roughly the likelihood of success of a given circuit on a specific piece of hardware. However, the usefulness of the error rates as reported by qiskit has been drawn into question [25, 32]. These experiments therefore serve as a test of the relevance of this reported error value in the context of fair sampling. In Figure 5 we see that both the soft and hard circuits report lower ground state probabilities as aggregate error rates increase, as expected. For Problem (e), we see that increased error rates results in less fair results. In other words, the errors result in systemic bias in favor of some ground states over others. For Problem (b) on the 5T architecture, we see the opposite behavior: increased error rates result in fairer solutions. We can see in Figure 5 that the likelihood of obtaining a ground state is ≈ 0.4 in the high error rate case. There are six degenerate ground states for this problem, out of 16 total states. Therefore a circuit that selected a state at random would select a ground state with 6/16 = 0.375 probability. By combining all of the data in to a single plot we can get a bigger picture view of how struc- tured vs. unstructured errors impact our results. In Figure 6 we see that fairness is highest for low aggregate error, decreases as the error approaches approximately 0.5, and then begins increasing again. This suggests that structured, i.e., biased, errors dominate in the middle of our error spec- trum. As the overall error rate increases, either due to longer circuits or worse performing qubits, the net result is more unstructured error and thus fairer results. In fact, we observed a decrease in ground state probability as aggregrate error increased for all circuits, which serves as a sanity check on our methodology. ACM Transactions on Quantum Computing, Vol. 3, No. 2, Article 8. Publication date: May 2022. 8:12 J. Golden et al. Fig. 4. In these longer circuits, involving four or five qubits and between 72 and 152 gates, we mostly observe a negative correlation between ground state probability and number of shots to reject fair sampling, with Problem (a) on the 4L and 4T architectures as the exceptions. 4.3 Comments on the IBM Q System Performance We believe these results constitute a serious evaluation of the IBM Q system. Our two most promi- nent observations based on this data are: Correlation between Ground State Probability and Fairness shifted. By specifically co- mparing the data generated in October 2020 against the data generated in August 2021, we can see how the overall fairness of the IBM Q system has evolved. Interestingly, if one only looks at the 2020 data, all circuits from Problem (a) exhibit “hard” unfairness. Running the experiments again in 2021, the 4T circuit shifted into the “soft” regime, see Figure 7. Fur- thermore, all circuits saw the slope of the fairness vs. ground state probability trend lines increase. Meanwhile, the average ground state probability across all problems decreased slightly, from 44.7% to 43.4%. The average fairness increased significantly in the new data, ACM Transactions on Quantum Computing, Vol. 3, No. 2, Article 8. Publication date: May 2022. Fair Sampling Error Analysis on NISQ Devices 8:13 Fig. 5. Results for a hard circuit, Problem (b) on the 5T architecture, as compared to results for a soft circuit, Problem (e). All circuits studied featured a negative correlation between ground state probability and aggre- gate error, as pictured here for these two circuits in the leftmost plots. For hard circuits, increase in aggregate error leads to behavior reminiscent of a random number generator, decreasing ground state probability but increasing fairness, as indicated in the top right plot. however this is due solely to a few data points in the two-qubit problems with extremely high (O(10 )) number of shots to reject fair sampling. The median number of shots to re- ject fair sampling actually decreased when looking at the new data, from 190 to 142. From this data we conclude that the newer machines are more dominated by structured error as opposed to random error. Quantum Volume only loosely correlates with fairness. The IBM metric of Quantum Volume [7] did broadly correlate with decreased aggregate error, which led to improved fairness in short circuits (see Figure 5). However, the Quantum Volume metric is only based off of the highly tuned performance of the best performing qubits available on a machine, instead of the overall hardware performance. This study, which ranges over all available qubits on a wide range of available chips, gives a more comprehensive view of performance. For example, we have plotted the results for Problem (d) evaluated on all 190 connected 3-qubit subsets on the manhattan backend (which has QV32 and 65 qubits), see Figure 8.In these plots one can see a wide range of results, and the best performing qubits for a single ACM Transactions on Quantum Computing, Vol. 3, No. 2, Article 8. Publication date: May 2022. 8:14 J. Golden et al. Fig. 6. Fairness as a function of aggregate error rate across all experiments. Fairness is highest for small and large error rates, with a decrease in the middle. This indicates that structured errors dominate in the middle regime, while unstructured errors play an increasingly important role as error increases. The log-quadratic line of best fit (in black) is included to graphically indicate these overall trends. metric (ground state probability, number of samples to reject fair sampling, and aggregate error rate) can perform quite poorly when evaluated by a different metric. Thus, this study emphasizes the need for more comprehensive benchmarking that does not only focus on random circuits evaluated on a small subset of qubits, as also discussed in [25]. 5 MEASUREMENT ERROR MITIGATION In this section we analyze how measurement error affects fairness. The measurement error in- cluded in our aggregate error metric (see Equation (5)) is for each individual qubit. This tensored approach paints an incomplete picture of measurement error, as there are correlations between qubits leading to correlated errors [28]. Therefore, we collected more thorough measurement er- ror data immediately before evaluating our fair sampling circuits. Specifically, for a given n-qubit circuit we explicitly prepared each of the 2 basis states and then measured the likelihood that the all of the other states would be observed. Using this data, we created a measurement error calibra- tion matrix for each set of qubits and used the inverse of that matrix to correct for measurement error [26]. Due to the small number of qubits in our problems under consideration, this simple approach is feasible; we relied on it rather than more advanced measurement error mitigation schemes [11, 23, 28]. Our complete results are visualized in Figure 13, here we describe the results in general. When going from the raw to the mitigated results, we tracked the change in ground state probability, ACM Transactions on Quantum Computing, Vol. 3, No. 2, Article 8. Publication date: May 2022. Fair Sampling Error Analysis on NISQ Devices 8:15 Fig. 7. Results for Problem (a) on the 4T architecture. Data collected across the IBM Q system in October 2020 showed a negative correlation between ground state probability and number of shots to reject fair sampling, whereas data collected in August 2021 showed a positive correlation. This indicates that the newer dates are more dominated by structured errors. number of shots to reject fair sampling, and the slope of the line of best fit correlating the two. We found mixed results in terms of ground state probability and fairness. Averaged over all problems, ground state probability improved by an average of 2.5%. Meanwhile, the mean number of shots to reject fair sampling decreased by an average of 19.1%. Finally, we observed that the slope of the line of best fit increased on 9 out of 10 problems, with an average improvement of 79 .8%. These results show that accounting for measurement error slightly improves ground state probability at the expense of fairness. The increase in slope suggests that the mitigated results are more dominated by structured error than the raw data. This result may appear somewhat surprising, as measurement error on these devices is itself a structured error: qubits prepared in the 0 state are far less likely to be measured as 1 than vice versa. However, in the raw data we observe that ground states that are composed mostly of 0 qubits are disfavored by an average of 2% (i.e., they appear 2% less frequently than expected from a perfectly fair sampler). Meanwhile, states that are composed of a majority of 1 qubits are favored by an average of 3.4%. It is unclear what errors are occurring during the evaluation of these circuits which lead to this bias in favor of 1, however it is likely measurement error is in fact helping results appear more fair by creating a slight counterbalancing bias in favor of 0. Indeed, after imposing our measurement error mitigation scheme we see that ground states that are composed mostly of 0 qubits are disfavored by an average of 5.9%, and those with mostly 1 qubits are favored by an average of 8%. By accounting for the measurement bias in favor of 0, we have highlighted the fact that at least some of the purported fairness of these devices was in fact due to multiple biases partially cancelling each other out. We see from this simple example that fair results can be produced by three different quantum computers: those dominated by random noise, those with biased errors that nearly cancel each other out, and those with low error rates. When determining the quality of these and future quan- tum computers, detailed analysis such as this is helpful in understanding which of the above sce- narios best describes the device at hand. ACM Transactions on Quantum Computing, Vol. 3, No. 2, Article 8. Publication date: May 2022. 8:16 J. Golden et al. Fig. 8. Detailed results for Problem (d) evaluated on all 3-qubit subsets of the manhattan backend. Each plot represents the same underlying data, and the set of qubits with the maximum ground state probability (max GSP), maximum number of shots to reject fair sampling (max NSRFS), and minimum aggregate error (min AR) are highlighted. This plot shows the high variance in results from a single backend. 6 CONCLUSION We have studied the extent to which today’s gate-based quantum computing hardware can fairly sample from the space of optimal solutions for discrete optimization problems. First, we introduced a novel metric for evaluating the fairness of current hardware: the number of shots to reject fair sampling. Using this metric, we found that hardware errors had a significant impact on fairness. For short circuits with ≤3 qubits, higher fidelity qubits resulted in increased fairness. We call these circuits “soft.” However, for longer circuits with ≥4 qubits, higher fidelity qubits actually resulted decreased fairness. We call these circuits “hard.” This is because low fidelity hardware begins behaving like a random number generator for long circuits, which contains no information but is relatively fair. It would be interesting to test multiple quantum computing hardware technologies (e.g., optical, trapped ions, etc.) to see if they have different fairness profiles. Our studies show that individual machines on the IBM Q backend can have dramatically different fairness and ground state proba- bilities when different qubits are used. As quantum computing hardware improves, we hope and expect to see an increase in qubit fairness and consistency. In this study we have also briefly explored the ways in which mitigating errors can affect fair- ness. Specifically, we observed that measurement error was partially correcting other structured er- rors. Measurement error mitigation therefore decreased fairness while modestly increasing ground state probability. We believe that studying fairness in addition to ground state probability can give a more nuanced picture of the errors affecting NISQ hardware. This is important for two reasons. First, random benchmark circuits are less susceptible to structured errors than circuits designed to solve realistic problems. And second, reported calibration and error rates reported are known to present an incomplete picture of how ordered circuits will perform [25]. This can be slightly im- proved upon by employing just-in-time noise data [32]. However, a broader theory of how different types of errors (read-out, crosstalk, etc.) affect not only ground state probability but also fairness is worth future study. This may involve utilizing different circuits or more refined statistical tools to highlight individual sources of error. ACM Transactions on Quantum Computing, Vol. 3, No. 2, Article 8. Publication date: May 2022. Fair Sampling Error Analysis on NISQ Devices 8:17 APPENDICES A IBM Q BACKEND INFORMATION Table 3. Details of IBM Q Backends to Be Studied Backend QV Qubits 2L 3L 4L 4T 5T manhattan 32 65 72 190 232 96 92 montreal 32 27 28 74 80 48 36 toronto 32 27 28 74 80 48 36 rome 32 5 4 6 4 0 0 santiago 32 5 4 6 4 0 0 October 2020 bogota 32 5 4 6 4 0 0 valencia 16 5 4 8 4 6 2 vigo 16 5 4 8 4 6 2 melbourne 8 15 20 72 130 66 120 ourense 8 5 4 8 4 6 2 montreal 128 27 28 74 80 48 36 mumbai 128 27 28 74 80 48 36 manhattan 32 65 72 190 232 96 92 brooklyn 32 65 72 190 232 96 92 toronto 32 27 28 74 80 48 36 August 2021 sydney 32 27 28 74 80 48 36 guadalupe 32 16 16 40 40 24 16 bogota 32 5 4 6 4 0 0 lagos 32 7 6 14 8 12 4 jakarta 16 7 6 14 8 12 4 lima 8 5 4 8 4 6 2 The name of the backend, the Quantum Volume (QV), the number of qubits, and the number of subsets of qubits on each device matching the relevant circuit architectures (2L, 3L, 4L, 4T, and 5T) are listed. B ALGORITHM FOR NUMBER OF SHOTS TO REJECT FAIR SAMPLING For the calculations in this paper we used n = 40960 and n = 1000. As explained in the main s i text, these values were chosen via a ‘goldilocks’ principle: much smaller, and the stochastic nature of the experiments resulted in inconsistent results, much larger and the computational burden was considerable. It is also worth noting that this algorithm becomes less useful as the samples become more evenly distributed, e.g., calculating the number of shots to reject fair sampling on a coin biased 50.001/49.999 takes O(1hr). ACM Transactions on Quantum Computing, Vol. 3, No. 2, Article 8. Publication date: May 2022. 8:18 J. Golden et al. ALGORITHM 1: Number of Shots to Reject Fair Sampling Input: Qubits Q, quantum circuit C for a problem P with k degenerate ground states labeled s ,..., s ; n number of shots, n number of inner loops 1 k s i Output: Number N of shots to reject fair sampling for C on Q. 1: Execute n shots of circuit C on qubits Q 2: o := the number of times ground state s observed i i 3: w := o / o  relative frequency of ground state s i i i i i=1 4: chiSqToReject := the χ value necessary to reject the fair sampling hypothesis with 95% sta- tistical significance for k − 1 degrees of freedom 5: function medianChiSq( N ) 6: chiSqArray := {} 7: while length(chiSqArray) < n do 8: testSamples := N samples drawn from the set {s } with weights {w } i i 9: testChiSq := χ for testSamples 10: append(chiSqArray, testChiSq) 11: end while 12: return median(chiSqArray) 13: end function 14: N := 2  χ ill-defined on sets of length 1 15: while medianChiSq (N ) < chiSqToReject do 16: N := 2N 17: end while 18: upperBound := N 19: lowerBound := N/2 20: while upperBound-lowerBound > 2 do 21: N := (upperBound+ lowerBound)/2 22: if medianChiSq (N ) < chiSqToReject then 23: lowerBound := N 24: else 25: upperBound := N 26: end if 27: end while 28: return N C DETAILS ON CIRCUIT COMPILATION In this appendix we provide supplementary information detailing the specifics of our compilation techniques. Full details of every hand compilation procedure would be too lengthy, instead we illustrate the compilation of Problem (d) as an example. Before we delve into details, we mention some broad ideas which aided in the compilation. • Circuit parameters: The values for βββ andγγγ in Table 2 were found by conducting a grid search over the angles (β,γ ) ∈ [−π, 0) × [−π, π ) with a resolution of π/60 and calculating the expectation value βββ,γγγ H βββ,γγγ classically (where the search space for β was cut in half due to a (β,γ )  (−β,−γ ) symmetry of the expectation value). This was efficient due to the relatively low circuit depths involved; for more complex circuits the hybrid approach detailed in Section 2.2 is more appropriate. ACM Transactions on Quantum Computing, Vol. 3, No. 2, Article 8. Publication date: May 2022. Fair Sampling Error Analysis on NISQ Devices 8:19 n n Fig. 9. (left) A 1-level Grover Mixer QAOA for an unconstrained optimization problem, U |↑  = |→ , n n −iβ|→ → | n n U (β) = e , and the Hamiltonian |→ → |  Id + X + X X + X X X +... M i i i,j i j i j i,j,k k which has also been proposed to achieve fair sampling in quantum annealing [21], going beyond limited higher-order terms. (right) For an unconstrained Ising problem with only quadratic terms in the Hamiltonian H on q ,...,q , C 0 n−1 n ⊗n−1 n n−1 we break symmetries by setting q = |↑ and U |↑  = Id ⊗ H |↑  = ↑→ . The quadratic terms of 0 S H involving q turn into linear terms of a Hamiltonian H on q ,...,q , thereby reducing q to a classical C 0 1 n−1 0 −β /π control bit. The multi-control-Z gate is fully symmetric, thus we may swap controls and target. Fig. 10. Two compilations of a large multi-control phase-shift gate (left) to gates with fewer controls: (middle) Direct decomposition into single-qubit phase-shift gates, large Toffolis, CNOTs, and a smaller multi-control phase-shift gate. (right) Decomposition using a ↑ -initialized ancilla into which we compute an AND (i.e., a Toffoli with mismatched phases), followed by a smaller multi-control phase-shift gate and an uncomputation of the AND. • Multi-controlled phase-shifts: First, we note that since terms in an Ising Hamiltonian pair- wise commute, implementing the phase separating unitary can be done for each term indi- ∓2γ /π vidually, with 2 CNOTs and one phase-shift gate Z . Care has to be taken, however, to address the connectivity constraints in the architectures. More difficult is the compilation of the multi-control phase-shift gate at the heart of the Grover Mixer. Figure 10 gives two recursive compilations into gates with fewer controls, one without ancillas, and one with a |↑-initialized ancilla. We note that careful recursive calls to these compilations and using symmetries can result in some of the smaller multi-control gates to cancel each other. • SWAP operations: We allow SWAP operations for qubits to change location in the circuit without having to go back to the original location. This is possible because the Grover Mixer is fully symmetric, i.e., we can arbitrarily exchange controls with the phase-shift gate, thus the input order to the Grover Mixer does not matter. We track qubit SWAPs with a permuta- tion array from which in the end we read a read-out map for the measurements. This gives us a significant advantage over transpilation tools which are not aware of such symmetries. Introducing SWAPs (which usually cost 3 CNOTs) at the right place in a circuit (next to a CNOT) will cost only 1 CNOT. Similarly, moving SWAPs through to the beginning or end of the circuit results in complete removal of the SWAP cost, as the SWAP gate can be replaced with a change in the permutation array and corresponding readout. Example: Problem (d) Let us now describe in detail the compilation of Problem (d) to the 3L architecture. Starting with the general form of the 1-level Grover Mixer QAOA with fixed qubit q :=↑, we implement each ∓2γ /π quadratic term of the Ising Hamiltonian with 2 CNOTs and a single-qubit phase-shift gate Z = ( ). ∓iγ 0 e ACM Transactions on Quantum Computing, Vol. 3, No. 2, Article 8. Publication date: May 2022. 8:20 J. Golden et al. Fig. 11. Phase Separator Compilation, Problem (d): Implementing the state preparation U and the phase −iγ (−Z Z +Z Z +Z Z +Z Z ) 0 1 1 2 1 3 2 3 separator U (γ ) = e on a Linear Nearest Neighbor architecture. adratic Qu terms commute pairwise, so they can be implemented individually with 2 CNOTs and one phase-shift gate each. Circuit size is reduced in three ways: (i) Setting q :=↑ results in q acting as a classical control bit, which 0 0 can be removed. (ii) Directly combining a single quadratic-term phase separator with a SWAP gate increases the CNOT count by 1 (instead of the usual 3). (iii) We keep track of the resulting permutation of the qubits instead of undoing the SWAPs. Fig. 12. Grover Mixer Compilation, Problem (d): Implementing the grover mixer unitary U (β) according to the high-level approch for multi-control phase-shift gates in Figure 10. Circuit size is reduced in two ways: (i) Moving SWAP gates through the circuit such that they appear at the beginning and the end, and thus can be tracked in the qubit permutation, rather than implemented. (ii) Moving (multi-)control gates through the circuit such that they can be cancelled with adjoint gates (not applicable in this example). The CNOTs with control q can be removed, leaving q in the role of a classical bit, see Fig- 0 0 ure 11. We notice that the other three quadratic terms form a triangle in the problem graph, hence we cannot map them to a 3L Linear Nearest Neighbor architecture, but rather need at least one SWAP. Inserting the SWAP as 3 CNOTs right to an existing CNOT of the phase separating uni- −iγ (Z Z ) 2 3 tary e gives a cancellation of 2 CNOTs, thus the addition of the SWAP results in only 1 additional CNOT. Rather than reversing this SWAP at the end of U (γ ), we keep track of the permutation of the qubits (q and q have swapped places). 2 3 This end configuration (swapped qubits) now builds the start of the Grover Mixer unitary. The X-gates and the control on qubit q can be dropped, as explained in Figure 9 (right). Using the recursive decomposition from Figure 10,weagain seethatweneed to addaSWAP,see Figure 12. The remaining single-controlled phase-shift gate between the middle two circuit wires can be im- plemented with 2 CNOTs and 3 phase-shift gates as shown. Keeping track of the SWAP throughout the end of the circuit, we see that qubits q and q end up back in their original position–in general, 2 3 this need not be the case. Comment on 5-qubit Ising Problems on 5T Using an Ancilla. Since an ancilla offers no benefit for the implementation of the phase separating unitary, and due to limited connectivity in the consid- ered architectures, we found that decomposing multi-controlled phase-shift gates using an ancilla as showninFigure 10 (right) can only be used advantageously for the 5-qubit Ising Problems (a) and (b). ACM Transactions on Quantum Computing, Vol. 3, No. 2, Article 8. Publication date: May 2022. Fair Sampling Error Analysis on NISQ Devices 8:21 Following the approach outlined for Problem (d), both Problems (a) and (b) result in a multi- controlled phase-shift gate acting on 4 qubits. Using a remaining qubit on the 5T architecture as a |↑-initialized ancilla (which has to be swapped around), we can implement the mentioned decomposition, which should return the ancilla to the |↑-state. As mentioned in the primary text, we discarded all samples where the ancilla qubit was not measured in the |↑ state. D MEASUREMENT ERROR MITIGATION RESULTS Fig. 13. Results of measurement error mitigation across all problems. Ground state probability improved by an average of 2.5%, number of shots to reject fair sampling decreased by an average of 19.1%, and the slope of the line correlating the two increased by an average of 79.8%. ACM Transactions on Quantum Computing, Vol. 3, No. 2, Article 8. Publication date: May 2022. 8:22 J. Golden et al. REFERENCES [1] Mahabubul Alam, Abdullah Ash-Saki, and Swaroop Ghosh. 2020. Design-space exploration of quantum approximate optimization algorithm under noise. In 2020 IEEE Custom Integrated Circuits Conference (CICC).1–4. https://doi.org/ 10.1109/CICC48029.2020.9075903 [2] Frank Arute, Kunal Arya, Ryan Babbush, Dave Bacon, Joseph C. Bardin, Rami Barends, Rupak Biswas, Sergio Boixo, Fernando GSL Brandao, David A. Buell, et al. 2019. Quantum supremacy using a programmable superconducting processor. Nature 574, 7779 (2019), 505–510. https://doi.org/10.1038/s41586-019-1666-5 [3] Sergey Bravyi, Alexander Kliesch, Robert Koenig, and Eugene Tang. 2020. Obstacles to variational quantum opti- mization from symmetry protection. Physical Review Letters 125 (2020), 260505. Issue 26. arXiv:1910.08980, https: //doi.org/10.1103/PhysRevLett.125.260505 [4] Andreas Bärtschi and Stephan Eidenbenz. 2020. Grover mixers for QAOA: Shifting complexity from mixer de- sign to state preparation. In IEEE International Conference on Quantum Computing & Engineering (QCE’20). 72–82. arXiv:2006.00354, https://doi.org/10.1109/QCE49297.2020.00020 [5] Andreas Bärtschi and Stephan Eidenbenz. 2021. Grover Mixer QAOA: Feasible State Superpositions, Convergence to Optimum, and Limitations. (2021). in preparation. [6] Jeremy Cook, Stephan Eidenbenz, and Andreas Bärtschi. 2020. The quantum alternating operator ansatz on max- imum k-vertex cover. In IEEE International Conference on Quantum Computing & Engineering (QCE’20). 83–92. arXiv:1910.13483, https://doi.org/10.1109/QCE49297.2020.00021 [7] Andrew W. Cross, Lev S. Bishop, Sarah Sheldon, Paul D. Nation, and Jay M. Gambetta. 2019. Validating quantum computers using randomized model circuits. Physical Review A 100, 3 (2019), 032328. arXiv:1811.12926, https://doi. org/10.1103/PhysRevA.100.032328 [8] Edward Farhi, Jeffrey Goldstone, and Sam Gutmann. 2014. A quantum approximate optimization algorithm. arXiv e-prints (2014). arXiv:1411.4028. [9] Edward Farhi, Jeffrey Goldstone, and Sam Gutmann. 2014. A quantum approximate optimization algorithm applied to a bounded occurrence constraint problem. arXiv e-prints (2014). arXiv:1412.6062. [10] Edward Farhi, Jeffrey Goldstone, Sam Gutmann, and Michael Sipser. 2000. Quantum computation by adiabatic evolu- tion. arXiv e-prints (2000). arXiv:quant-ph/0001106. [11] Lena Funcke, Tobias Hartung, Karl Jansen, Stefan Kühn, Paolo Stornati, and Xiaoyang Wang. 2020. Measurement error mitigation in quantum computers through classical bit-flip correction. (7 2020). arXiv: quant-ph/2007.03663. [12] Lov K. Grover. 2005. Fixed-point quantum search. Physical Review Letters 95, 15 (2005), 150501. https://doi.org/10.1103/ PhysRevLett.95.150501 [13] Stuart Hadfield, Zhihui Wang, Bryan O’Gorman, Eleanor G. Rieffel, Davide Venturelli, and Rupak Biswas. 2019. From the quantum approximate optimization algorithm to a quantum alternating operator ansatz. Algorithms 12, 2 (2019), 34. arXiv:1709.03489, https://doi.org/10.3390/a12020034 [14] Dylan R. Harp, Zhenxue Dai, Andrew V. Wolfsberg, Jasper A. Vrugt, Bruce A. Robinson, and Velimir V. Vesselinov. 2008. Aquifer structure identification using stochastic inversion. Geophysical Research Letters 35, 8 (2008), L08404. https://doi.org/10.1029/2008GL033585 [15] Matthew P. Harrigan et al. 2021. Quantum approximate optimization of non-planar graph problems on a planar su- perconducting processor. Nature Physics 17, 3 (2021), 332–336. arXiv:2004.04197, https://doi.org/10.1038/s41567-020- 01105-y [16] Tadashi Kadowaki and Hidetoshi Nishimori. 1998. Quantum annealing in the transverse Ising model. Physical Review E 58, 5 (1998), 5355. arXiv:cond-mat/9804280, https://doi.org/10.1103/PhysRevE.58.5355 [17] Vaibhaw Kumar, Casey Tomlin, Curt Nehrkorn, Daniel O’Malley, and Joseph Dulny III. 2020. Achieving fair sampling in quantum annealing. arXiv e-prints (2020). arXiv:2007.08487. [18] Mario S. Könz, Guglielmo Mazzola, Andrew J. Ochoa, Helmut G. Katzgraber, and Matthias Troyer. 2019. Uncertain fate of fair sampling in quantum annealing. Physical Review A 100, 3 (2019), 030303. arXiv:1806.06081, https://doi.org/ 10.1103/PhysRevA.100.030303 [19] Salvatore Mandra, Zheng Zhu, and Helmut G. Katzgraber. 2017. Exponentially biased ground-state sampling of quan- tum annealing machines with transverse-field driving Hamiltonians. Physical Review Letters 118, 7 (2017), 070502. arXiv:1606.07146, https://doi.org/10.1103/PhysRevLett.118.070502 [20] Yoshiki Matsuda, Hidetoshi Nishimori, and Helmut G. Katzgraber. 2009. Ground-state statistics from annealing algo- rithms: Quantum versus classical approaches. New Journal of Physics 11, 7 (2009), 073021. https://doi.org/10.1088/1367- 2630/11/7/073021 arXiv:0808.0365 [21] Yoshiki Matsuda, Hidetoshi Nishimori, and Helmut G. Katzgraber. 2009. Quantum annealing for problems with ground-state degeneracy. Journal of Physics: Conference Series 143 (2009), 012003. https://doi.org/10.1088/1742-6596/ 143/1/012003 ACM Transactions on Quantum Computing, Vol. 3, No. 2, Article 8. Publication date: May 2022. Fair Sampling Error Analysis on NISQ Devices 8:23 [22] Mauro E. S. Morales, Timur Tlyachev, and Jacob Biamonte. 2018. Variational learning of Grover’s quantum search algorithm. Physical Review A 98, 6 (2018), 062333. arXiv:1805.09337, https://doi.org/10.1103/PhysRevA.98.062333 [23] Paul D. Nation, Hwajung Kang, Neereja Sundaresan, and Jay M. Gambetta. 2021. Scalable mitigation of measurement errors on quantum computers. arXiv e-prints (2021). arXiv:2108.12518. [24] Daniel O’Malley. 2018. An approach to quantum-computational hydrologic inverse analysis. Scientific Reports 8, 1 (2018), 1–9. https://doi.org/10.1038/s41598-018-25206-0 [25] Timothy Proctor, Kenneth Rudinger, Kevin Young, Erik Nielsen, and Robin Blume-Kohout. 2020. Measuring the capa- bilities of quantum computers. arXiv e-prints (2020). arXiv:2008.11294. [26] Qiskit Development Team. Qiskit Documentation: Measurement Error Mitigation. (n.d.). https://qiskit.org/ documentation/tutorials/noise/3_measurement_error_mitigation.html. [27] Lukas M. Sieberer and Wolfgang Lechner. 2018. Programmable superpositions of Ising configurations. Physical Review A 97, 5 (2018), 052329. arXiv:1708.02533, https://doi.org/10.1103/PhysRevA.97.052329 [28] Swamit S. Tannu and Moinuddin K. Qureshi. 2019. Mitigating measurement errors in quantum computers by exploit- ing state-dependent bias. In Proceedings of the 52nd Annual IEEE/ACM International Symposium on Microarchitecture (MICRO’52). Association for Computing Machinery, New York, NY, USA, 279–290. https://doi.org/10.1145/3352460. [29] Marc Vuffray, Carleton Coffrin, Yaroslav A. Kharkov, and Andrey Y. Lokhov. 2020. Programmable quantum annealers as noisy Gibbs Samplers. (2020). arXiv:quant-ph/2012.08827. [30] Samson Wang, Enrico Fontana, Marco Cerezo, Kunal Sharma, Akira Sone, Lukasz Cincio, and Patrick J. Coles. 2020. Noise-induced barren plateaus in variational quantum algorithms. arXiv e-prints (2020). arXiv:2007.14384. [31] Zhihui Wang, Nicholas C. Rubin, Jason M. Dominy, and Eleanor G. Rieffel. 2020. XY mixers: Analytical and numerical results for the Quantum Alternating Operator Ansatz. Physical Review A 101, 1 (2020), 012320. arXiv:1904.09314, https: //doi.org/10.1103/PhysRevA.101.012320 [32] Ellis Wilson, Sudhakar Singh, and Frank Mueller. 2020. Just-in-time quantum circuit transpilation reduces noise. In IEEE International Conference on Quantum Computing & Engineering (QCE’20). 345–355. arXiv:2005.12820, https://doi. org/10.1109/QCE49297.2020.00050 [33] Jonathan Wurtz and Peter Love. 2021. MaxCut quantum approximate optimization algorithm performance guarantees for p > 1. Physical Review A 103 (2021), 042612. Issue 4. arXiv:2010.11209, https://doi.org/10.1103/PhysRevA.103.042612 [34] Masayuki Yamamoto, Masayuki Ohzeki, and Kazuyuki Tanaka. 2020. Fair sampling by simulated annealing on quan- tum annealer. Journal of the Physical Society of Japan 89, 2 (2020), 025002. arXiv:1912.10701, https://doi.org/10.7566/ JPSJ.89.025002 [35] Theodore J. Yoder, Guang Hao Low, and Isaac L. Chuang. 2014. Fixed-point quantum search with an optimal number of queries. Physical Review Letters 113, 21 (2014), 210501. arXiv:1409.3305, https://doi.org/10.1103/PhysRevLett.113. [36] Brian Hu Zhang, Gene Wagenbreth, Victor Martin-Mayor, and Itay Hen. 2017. Advantages of unfair quantum ground- state sampling. Scientific Reports 7 (2017), 1044. arXiv:1701.01524, https://doi.org/10.1038/s41598-017-01096-6 Received January 2021; revised September 2021; accepted January 2022 ACM Transactions on Quantum Computing, Vol. 3, No. 2, Article 8. Publication date: May 2022. http://www.deepdyve.com/assets/images/DeepDyve-Logo-lg.png ACM Transactions on Quantum Computing Association for Computing Machinery

Fair Sampling Error Analysis on NISQ Devices

Download PDF
23 pages

Loading next page...
 
/lp/association-for-computing-machinery/fair-sampling-error-analysis-on-nisq-devices-uOh5BbDypj

References (46)

Publisher
Association for Computing Machinery
Copyright
Copyright © 2022 Copyright held by the owner/author(s).
ISSN
2643-6809
eISSN
2643-6817
DOI
10.1145/3510857
Publisher site
See Article on Publisher Site

Abstract

JOHN GOLDEN, Information Sciences (CCS-3) and Computational Earth Sciences (EES-16), Los Alamos National Laboratory, USA ANDREAS BÄRTSCHI, Information Sciences (CCS-3), Los Alamos National Laboratory, USA DANIEL O’MALLEY, Computational Earth Sciences (EES-16), Los Alamos National Laboratory, USA STEPHAN EIDENBENZ, Information Sciences (CCS-3), Los Alamos National Laboratory, USA We study the status of fair sampling on Noisy Intermediate Scale Quantum (NISQ) devices, in particular the IBM Q family of backends. Using the recently introduced Grover Mixer-QAOA algorithm for discrete optimization, we generate fair sampling circuits to solve six problems of varying difficulty, each with several optimal solutions, which we then run on twenty backends across the IBM Q system. For a given circuit evaluated on a specific set of qubits, we evaluate: how frequently the qubits return an optimal solution to the problem, the fairness with which the qubits sample from all optimal solutions, and the reported hardware error rate of the qubits. To quantify fairness, we define a novel metric based on Pearson’s χ test. We find that fairness is relatively high for circuits with small and large error rates, but drops for circuits with medium error rates. This indicates that structured errors dominate in this regime, while unstructured errors, which are random and thus inherently fair, dominate in noisier qubits and longer circuits. Our results show that fairness can be a powerful tool for understanding the intricate web of errors affecting current NISQ hardware. CCS Concepts: • Hardware → Quantum error correction and fault tolerance;• Computer systems organization → Quantum computing Additional Key Words and Phrases: Quantum alternating operator ansatz, fair sampling ACM Reference format: John Golden, Andreas Bärtschi, Daniel O’Malley, and Stephan Eidenbenz. 2022. Fair Sampling Error Analysis on NISQ Devices. ACM Trans. Quantum Comput. 3, 2, Article 8 (May 2022), 23 pages. https://doi.org/10.1145/3510857 1 INTRODUCTION Combinatorial optimization problems, such as the Minimum Traveling Salesperson problem, the Maximum Satisfiability problem, or the Spin Glass Ising model, are widely viewed to be one of the most promising application domains for quantum computing. Such problems often have multiple optimal solutions, and depending on the algorithm being used to solve the problem, some optima may be much easier to find than others. This is problematic if one is interested not in finding a single solution, but instead determining the overall landscape of optimal solutions. Algorithms Authors’ addresses: J. Golden, Information Sciences (CCS-3) and Computational Earth Sciences (EES-16), Bldg. 1489, Casa Grande Drive, Los Alamos National Laboratory, Los Alamos, New Mexico, USA, 87545; email: golden@lanl.gov; A. Bärtschi and S. Eidenbenz, Information Sciences (CCS-3), Bldg. 508, Diamond Drive, Los Alamos National Laboratory, Los Alamos, New Mexico, USA, 87545; emails: {baertschi, eidenben}@lanl.gov; D. O’Malley, Computational Earth Sciences (EES-16), Bldg. 1400, Casa Grande Drive, Los Alamos National Laboratory, Los Alamos,New Mexico, USA, 87545; email: omalled@lanl.gov. This work is licensed under a Creative Commons Attribution International 4.0 License. © 2022 Copyright held by the owner/author(s). 2643-6817/2022/05-ART8 https://doi.org/10.1145/3510857 ACM Transactions on Quantum Computing, Vol. 3, No. 2, Article 8. Publication date: May 2022. 8:2 J. Golden et al. which fairly sample from the space of optimal solutions avoid this problem and have practical value in physics and engineering. For instance, it is often difficult in practice to encode all the engineering design goals in the function to be optimized. By allowing an engineer to explore an unbiased assortment of optimal solutions, fair sampling provides an opportunity to apply these other design goals to a more diverse set of options (especially useful when a customer adds one more requirement after the optimization has been performed). It can also be used to produce an ensemble of predictions from calibrated physical models that can be used to make a forecast in- cluding a range of possible outcomes (e.g., in the context of fluid flow in the earth’s subsurface, which can be formulated as a discrete optimization problem [14, 24]). In this paper, we determine the current status of fair sampling in the context of quantum comput- ing. Specifically, we are interested both in algorithms which theoretically guarantee fair sampling (i.e., generate quantum states where all optimal solutions have the same amplitudes squared and thus are measured with equal probability), as well as actual hardware implementations of such algorithms. Previous studies have shown that quantum annealing cannot be used as a fair sampler for all problems [18]. We therefore focus our work on gate-based quantum computing algorithms and hardware. The most widely-used quantum optimization framework is the Quantum Alternating Oper- ator Ansatz (QAOA), which alternates between a mixer operator that mixes the amplitudes of all feasible solutions and a phase separator operator that separates the phases of the feasible so- lutions [13]. Most mixers do not exhibit fair sampling, however, the recently discovered Grover Mixer [4] has the property that all feasible solutions with the same objective value have identical amplitudes. This novel Grover Mixer implementation of QAOA thus solves the theoretical side of fair sampling for gate-based quantum computing. The majority of this work is dedicated to a practical evaluation of how fair sampling circuits are affected by the errors inherent in Noisy Intermediate Scale Quantum (NISQ) devices. In this study, a paradox quickly becomes apparent: while an error-corrected quantum device could obviously be used as a perfectly fair sampler, so could a quantum device completely dominated by random noise. Of course the noisy machine would be no different than a random number generator, and would sample from all feasible solutions rather than just the optimal solutions, but it would do so fairly. Therefore, our study seeks to clarify where current NISQ devices lie on the spectrum between these two extremes. Specifically, are the hardware errors predominantly random and unstructured, thus leading to relatively fair sampling of optimal solutions? Or do hardware biases and other structured errors play an important role, thus leading to unfair behavior? Our methodology for answering these questions consists of the following steps: (1) We define six example problems, which are a slightly expanded set of examples studied in the context of fair sampling in quantum annealing [18]. (2) We design quantum circuits for our problems and hand-tune them to minimize circuit depth and gate counts, taking into account different qubit connectivity topologies as encountered on IBM Q. Our circuits involve two to five qubits and our medium size examples result in circuits with gate counts in the 30–60 range, which is similar to the size of the (random) circuits used to establish the Quantum Volume metric on the IBM Q backends. (3) We execute our circuits in a large number of shots on twenty different IBM Q backends that have qubit counts of up to 65 qubits. In addition, we embed our circuits in all possible topologies on the circuits resulting in a total of 7,440 experiments at 40,960 shots each. (4) We define a novel metric for fairness in quantum computing based on Pearson’s χ test, a common statistical tool to evaluate how likely it is that a set of observational data came from a null hypothesis distribution. Since all NISQ hardware will necessarily involve some biases ACM Transactions on Quantum Computing, Vol. 3, No. 2, Article 8. Publication date: May 2022. Fair Sampling Error Analysis on NISQ Devices 8:3 and imperfections that inhibit fair sampling, we are interested in how much effort must be expended to prove that a piece of hardware is biased, i.e., reject the null hypothesis of fair sampling. We term this metric ‘number of shots to reject fair sampling,’ and it depends on both a specific set of qubits as well as a fair sampling circuit. This fairness metric is a separate contribution to the main results of the paper. (5) We analyze our experiments using our fairness metric and we calculate the aggregate error of each experiment from IBM’s reported gate-specific error rates. We find that the NISQ devices under evaluation provide noticeably unfair sampling. However, fairness increases as aggregate error approaches both 0 and 1, matching with the theoretical ex- tremes of error-corrected and noise-dominated devices discussed previously. In the case of small aggregate error, this is to be expected, as for 0 error we would retrieve the theoretical guarantees of Grover Mixer QAOA. Likewise, for large aggregate error, one should get samples uniformly at random from all computational basis states. Nonetheless, the result is perhaps counter-intuitive, as one might expect a linear interpolation between these two extremes. For example, the hardware- efficient random circuits in Google’s recent quantum supremacy experiment [ 2] give a sample dis- tribution consisting of a convex combination of an ideal (error-free) Porter-Thomas distribution and a Uniform distribution (caused by errors). Our differing results indicate that we should not ex- pect errors themselves to be evenly distributed (i.e., in a fair manner) across qubits for highly struc- tured circuits such as ours. Indeed, a previous study found that structured errors (e.g., crosstalk, coherent noise) play a large role in the performance of structured circuits [25]. Our result holds for our selected set of circuits and across the twenty NISQ backends on IBM Q. We propose that fair sampling be an addition in the emerging set of benchmarks for NISQ de- vices, which so far have largely focused on circuit correctness. As we show in this work, biased errors can artificially inflate circuit accuracy, and fair sampling on its own can be fooled by devices dominated by random noise. However, by studying both accuracy and fairness, we gain a more complete picture of the type and magnitude of errors affecting NISQ hardware. In summary, the main contributions of our work are: (1) We propose a novel methodology for quantifying the fairness of sampling on NISQ hardware. (2) We elucidate two noise regimes (high noise and low noise) that result in fair sampling, and a third regime (intermediate noise) that is dominated by biased errors. (3) In practice, this approach provides a way to determine whether near-term improvements that reduce noise will improve or degrade the performance of fair sampling for a given problem. (4) As an example, we study how a practical error mitigation technique (namely, measurement error mitigation) impacts the quality of sampling on IBM Q hardware. 2REVIEW In this section we describe previous studies of fair sampling in the context of quantum computing. A note on nomenclature: since the optimization problems we study can all be phrased in terms of Ising models, we adopt the more physics-oriented language and refer to ‘optimal solutions’ as ground states. Problems with multiple optimal solutions are thus said to have degenerate ground states. 2.1 Fair Sampling in Quantum Annealing After quantum annealing was proposed as a possibly useful optimization heuristic [16], it was dis- covered that naïve approaches to quantum annealing do not always sample degenerate ground states fairly [20]. In fact, certain Hamiltonians feature ground states that are sampled with ACM Transactions on Quantum Computing, Vol. 3, No. 2, Article 8. Publication date: May 2022. 8:4 J. Golden et al. probability approaching zero as annealing time increases. This effect is inherent to the quantum an- nealing approaches, and is not the consequence of hardware noise or other technical limitations. This behavior was subsequently verified experimentally [ 19]. Since unfair sampling (especially when certain ground states are strongly suppressed) can have negative consequences for many applications, techniques were proposed to improve upon the naïve quantum annealing approach [17, 18, 27, 34]. None of these techniques have been tested on quantum annealing hardware, in- dicating some of the challenges with these approaches. It should also be noted that it has been proposed that there are some advantages of biased samplers, that is, they can be used effectively in an ensemble of samplers each of which has different biases [ 36]. One technique that has been proposed to improve the fairness of quantum annealing is the introduction of more complex Hamiltionians to drive the annealing process [19]. This generally improves fairness, but is hard to implement and does not resolve the underlying problem. Even for small problems, biases sometimes remain despite using fairly high-order Hamiltonians [18]. In studying these higher-order Hamiltonians, Könz et al. [18] introduced four Ising models that are suitable for our QAOA analyses. The problems are described in Table 1 as Problems (a)–(d). These problems all exhibit biased sampling with quantum annealing, so they present a non-trivial challenge for fair sampling algorithms. Two of them involve 4 qubits, one involves 5 qubits, and another involves 6 qubits. This small number of qubits make it plausible that these problems can be solved with reasonably high fidelity on current NISQ hardware. Impact of noise: Recent work [29] achieved high-quality thermal Gibbs Sampling with the D- Wave 2000Q system for a restricted class of Ising Hamiltonians that are native to the hardware connectivity and have coupling strengths J ∈{−1, 0, 1}. This category includes Hamiltonians ij with degenerate ground states, but not the more general Hamiltonians studied in this paper (see Table 1). Fair Sampling of those ground states is achieved by re-scaling the input energy scale to a sweet spot that is large enough to be noise-resilient but small enough to overcome the biases from the transverse field. In the identified sweet spot energy scale, changing the annealing time influences the effective temperature of the Gibbs samples generated by the hardware, and thus the probability to find ground states. 2.2 Fair Sampling in the antum Qu Alternating Operator Ansatz The Quantum Alternating Operator Ansatz [13] (QAOA) is a promising heuristic quantum algorithm for (combinatorial) optimization problems. In its essence, for a problem instance I with feasible states F and cost Hamiltonian H on n qubits, a p-round QAOA prepares a parametrized state from which one would like to sample low-energy states with respect to H : β γ ββ,γγ := U (β )U (γ )···U (β )U (γ )U ↑ . (1) M p P p M 1 P 1 S The circuit consists of an initial state preparation unitary operator U that creates some super- position of all feasible solutions in I, followed by p applications of alternating parametrized phase separating and mixing unitaries U (γ ), U (β ) with real angle parameters γγγ = (γ ,...,γ ) and P k M k 1 p βββ = (β ,..., β ) , and a final measurement in the computational basis, see e.g., Figure 1.The βββ,γγγ 1 p angles are traditionally found via a hybrid classical/quantum approach: given some initial parame- ters, take enough samples to reliably estimate the expectation value βββ,γγγ H βββ,γγγ , use a classical optimizer to adjust the parameters, and repeat until one has a state with a good (low-energy) expec- tation value. For further description of, and theoretical justifications for this approach, see [ 6, 8, 31]. The role of the phase separating unitaries U (γ ) is to add multiplicative phase factors to the am- plitudes of feasible computational basis states, with phases proportional to respective energies. We −iγ H usually have (up to global phases) U (γ )  e . For many problems, such as MaxCut [8]orthe problems considered in this paper, H is an Ising Hamiltonian with quadratic and sometimes linear ACM Transactions on Quantum Computing, Vol. 3, No. 2, Article 8. Publication date: May 2022. Fair Sampling Error Analysis on NISQ Devices 8:5 Fig. 1. Grover Mixer QAOA: The state preparation unitary U for an equal superposition of all feasible states † 10 1 −β /π |F = √ |x, its conjugate transpose U , and a multi-controlled phase-shift gate Z = x∈F −iβ S 0 e |F| −iβ|FF| are used to implement the mixer U (β) = e . terms; but in other problems, H can also involve higher-order terms, such as for MaxE3Lin2 [9]. The interplay between the state preparation unitary U and the mixing unitary U is particularly S M important, and falls into three categories: Original Approach. The original approach by Farhi et al. [8], termed Quantum Approxi- mate Optimization Algorithm, was inspired by quantum annealing for unconstrained op- ⊗n timization problems, i.e., U = H (preparing an equal superposition of all states) and −iβ X U (β) = e (a transverse field of Pauli-X operators acting on individual qubits). General Ansatz. The general framework introduced in Equation (1) by Hadfield et al. [ 13]is a generalization also suitable for a wide variety of constrained optimization problems. They focus on designing involved mixing unitariesU (β) and usually splitU in a layer of Identity M S and Pauli-X gates preparing some feasible computational basis state, followed by an initial mixing unitary U (β ) with new angle β . M 0 0 Grover Mixer. If one can design efficient state preparation unitaries U preparing an equal superposition of all feasible basis states |F = 1/ |F| |x, then this gives rise to a x∈F mixing unitary resembling Grover’s selective phase-shift operator [12, 22, 35], U (β) = −iβ|FF| −iβ −iβ e = Id− (1− e ) |FF| = U (Id− (1− e ) |↑↑|)U ,see Figure 1.Insomesense, this is the reverse of the method in the general ansatz, and it was shown to be applicable to many relevant optimization problems [4], but there also exists a range of constrained problems where it is ruled out under complexity-theoretic assumptions [5]. Although the Quantum Alternating Operator Ansatz is the most general framework of the three, both the Quantum Approximate Optimization Algorithm as well as Grover Mixer QAOA have their advantages. For example, the original approach by definition cannot do worse than a Trotterization of the adiabatic algorithm [10], thus converging to a ground state in the p →∞ limit [8]. Further- more, its phase separating and mixing operators are periodic, restricting the angles to real values in [−π, π ). Grover Mixer QAOA retrieves these properties for constrained optimization problems with efficient state preparation unitaries U [5], but for unconstrained optimization problems the −iβ|FF| −iβ X Grover Mixer e has a larger circuit depth than the Transverse Field Mixer e .Inthis paper we explore an advantage unique to Grover Mixer QAOA among the three categories: fair sampling. With Grover Mixer QAOA, all feasible basis states begin with amplitude 1/ |F| (after state −iγ H preparation with U ). The phase separating unitary U (γ ) = e then phases the amplitude of S P ACM Transactions on Quantum Computing, Vol. 3, No. 2, Article 8. Publication date: May 2022. 8:6 J. Golden et al. every basis state proportional to its energy and γ , keeping the same phase for basis states of same −iβ energy. The mixing unitary U (β) = Id − (1 − e ) |FF| then deducts from all amplitudes of −iβ their input state ψ the same weighted average of all of its amplitudes, ((1− e )/ |F|)F | ψ. Therefore, basis states with the same energy are sampled with the same amplitude. This property holds for all feasible states (not only ground states) and is independent of the number of QAOA levels or the choice of angles for U ,U . For a complete proof, see [4]. M P Impact of noise: While the original based Quantum Approximate Optimization Algorithm of- fers non-trivial provable performance guarantees already for a single for certain problems such as MaxCut on ( d=3)-regular graphs [8], a higher number of rounds – p ≥ 6[33]oreven p ∈ Ω(log(n)/d) [3] – might be necessary to outperform the best-known classical approximation ratios. However, simulations considering realistic noise models of current hardware show that perfor- mance in terms of approximation ratio may degrade even for p = 2[1]. Noise leading to a decrease in performance has also been observed for growing problem sizes n, with the exception of problem graphs matching the hardware connectivity [15]. For the latter setting, experiments on actual hard- ware have shown an approximation ratio increase up to p = 3(mean)and p = 4 (certain instances), but no further [15]. Furthermore, even with constantly improving hardware, complications may arise in the optimization of the variational angles β,γ due to noise-induced barren plateaus [30]. All of these results refer to transverse field based QAOA, which does not have the property of sampling fairly among ground states. For Grover Mixer QAOA, we have a theoretical guarantee of fair sampling for ideal noise-less devices. At the same time we have a larger circuit depth of the mixer, asymptotically matching the circuit depth of the phase separator [4]. Hence, for current NISQ devices, we restrict ourselves to a single-round Grover Mixer QAOA, analysing the impact of noise on both ground state probability and fair sampling among ground states. 2.3 Description of Test Circuits In addition to the four problems from [18], we additionally studied two simpler Ising models with degenerate ground states but only involving two and three qubits. While these models do not feature ground state suppression on quantum annealing hardware, we include them in this study because they represent the simplest non-trivial models with degenerate ground states and thus serve as useful baselines as compared to the more complicated models of [18]. Allofthe models we studied are depicted in Table 1. Our general procedure for generating test circuits was to begin with a 1-level Grover Mixer QAOA algorithm with q fixed to ↑. We chose a 1-level implementation of Grover-QAOA in order to keep circuit depth low. Following [18], we fix q :=↑ as all of the models in Table 1 are ↑/↓ symmetric. This allows us to reduce the problems from Ising Hamiltonians H with only quadratic terms acting on qubits q ,...,q to new Hamiltonians H with some linear terms acting only on 0 n−1 qubits q ,...,q . This also transforms q into a classical control bit for the consecutive Grover 1 n−1 0 Mixer, which can be removed. Thus, we can embed Ising problems on n qubits onto circuits with only n − 1 qubits. See Figure 9 in Appendix C for a visual representation of a generic 1-level Grover Mixer QAOA for an unconstrained optimization problem, along with a description of the simplifications resulting from fixing q . We then compiled the circuits to match connectivity graphs and gates available on the IBM Q backend. Instead of employing the tools available in IBM’s qiskit software, we compiled the circuits by hand. This had two benefits. First, this reduced circuit depth by roughly a factor of two as compared to qiskit. Second, it allowed us to generate circuits for architectures that can be found as subgraphs of every hardware connectivity graph among the IBM Q backends, see ACM Transactions on Quantum Computing, Vol. 3, No. 2, Article 8. Publication date: May 2022. Fair Sampling Error Analysis on NISQ Devices 8:7 Table 1. Ising Models with Degenerate Ground States to Be Studied on NISQ Hardware Problems (a)–(d) are from [18]. All of the Ising models H = − J Z Z have only quadratic terms and no linear C ij i j terms. The dark red edges indicate a ferromagnetic J =+2 coupling and the light red edges a ferromagnetic J =+1 ij ij coupling; the light blue edges represent an antiferromagnetic J = −1 coupling and the dark blue edges an ij antiferromagnetic J = −2 coupling. Only ground states with q =↑ are listed as all models are symmetric under ij 0 simultaneous ↑ /↓-flips. Note that a symmetry-breaking fixed setting q :=↑ results in a Hamiltonian H on qubits q , ..., q (without q ) with some linear terms. 1 n 0 Figure 2. Thus, the same exact circuit could be evaluated across many backends, and in many instances across many different qubits on the same backend. For Problems (a) and (b), which involve five variables (and with q :=↑ require only 4 qubits) we generated three distinct circuits, each on a different architecture: 4T, 4L, and 5T (see Figure 2 for notation). The 5T circuits for these problems employ an ancilla qubit, which allows us to discard any sample in which the ancilla was not measured in the↑ state. The remaining Problems (c)–(f) each only have a single circuit associated with them. Note that Problem (f), with the Hamiltonian Z Z and ground states |↑↓ and |↓↑, is uniquely simple. In this case we do not fix q :=↑,nor 0 1 0 do we employ Grover-QAOA, as the problem can be “solved” with an almost-trivial circuit of a ACM Transactions on Quantum Computing, Vol. 3, No. 2, Article 8. Publication date: May 2022. 8:8 J. Golden et al. Fig. 2. The hardware architectures we transpile our circuits to: Linear Nearest Neighbor connectivities on 2,3,4 qubits (2L,3L,4L), as well as T-shaped connectivities on 4,5 qubits (4T,5T). These architectures form connectivity subgraphs of all IBM Q devices. By fixing q :=↑,each n-qubit problem in Table 1 can be mapped to a (n − 1)-qubit architecture. Hadamard on qubit q ,a X-gate on qubit q , and a consecutive CNOT with control q and target 0 1 0 q . Due to the small number of qubits and gates, we use this circuit to explore the highest reaches of accuracy and fairness capable with the hardware. See Table 2 for a brief description of all of the circuits, including their architectures, gate count, and βββ, γγγ parameters. We describe our compilation technique in detail in Appendix C,and give QASM representations of our circuits in the ancillary file associated to this note. 3 METHODOLOGY We evaluated the previously described set of circuits across a wide swath of hardware on the IBM Q platform. Furthermore, we collected data in two large sets, one in October 2020 and the other in August 2021, allowing for an analysis of the change in fairness of the IBM Q platform over time. Since our interest was not in identifying the best performing qubits, but rather observing trends in circuit correctness vs. fairness, we evaluated the same circuit many times per chip, each time using a different subset of qubits. For example, testing the circuit for Problem (e), which has only 2 qubits, on a chip with 5 qubits in an LNN architecture would involve four separate evaluations (on the pairs of qubits {q ,q } for i = 1... 4). The machines we tested, along with the number of i i+1 qubit subsets for each architecture, are listed in Table 3. It is important to emphasize that there was no additional transpilation or optimization of these circuits beyond that discussed in the previous section. We collected 40,960 shots for each circuit on each set of applicable qubits, which gave us suffi- cient data to achieve repeatable statistics. None of the hardware consistently reported fair results, i.e., hardware noise introduced significant biases for even the simplest circuits. In order to charac- terize how fair a given set of qubits performed with a given circuit, we adopted the metric: how many shots does one need of a given circuit on a given set of qubits in order to reject the hypothe- sis that these qubits are fair samplers at 95% significance level? This is equivalent to the question of: how many flips (on average) does it take to show that a coin is unfair? A 90/10 coin will take relatively few flips to reveal its bias, whereas a 51/49 coin will take many flips. We call this metric “number of shots to reject fair sampling.” To calculate the number of shots to reject fair sampling, we utilized Pearson’s χ test, which evaluates how likely a set of observations {O } is given some expected null hypothesis values {E }. If a given circuit has d degenerate ground states, and n is the number of ground state i д.s . observations (out of the 40,960 total shots), our null hypothesis is that д.s . E = E = . (2) The χ value is determined via (O − E) χ = . (3) ACM Transactions on Quantum Computing, Vol. 3, No. 2, Article 8. Publication date: May 2022. Fair Sampling Error Analysis on NISQ Devices 8:9 Table 2. Details for 1-round Grover-QAOA Fair Sampling Circuits Studied on NISQ Hardware β γ β γ Prob. Arch. #Rot. #CNOT (β, γ ) ββ,γγ H ββ,γγ GSP (a) 4L 57 35 (−1/2,−11/12) · π –2.682 out of –4 0.498 (a) 4T 51 29 (−1/2,−11/12) · π –2.682 out of –4 0.498 (a) 5T 47 25 (−1/2,−11/12) · π –2.682 out of –4 0.498 (b) 4L 59 39 (−11/15,−17/60) · π –4.228 out of –5 0.846 (b) 4T 53 32 (−11/15,−17/60) · π –4.228 out of –5 0.846 (b) 5T 49 29 (−11/15,−17/60) · π –4.228 out of –5 0.846 (c) 5T 90 62 (−23/60, 1/15) · π –1.563 out of –4 0.215 (d) 3L 26 14 (−5/12, 1/10) · π –1.319 out of –2 0.702 (e) 2L 16 4 (−23/60, 3/5) · π –0.999 out of –1 1.000 (f) 2L 2 1 N/A N/A 1.000 For each circuit, we list the problem from Table 1 being solved, the qubit connectivity from Figure 2 employed, the total number of single-qubit (i.e. rotation) gates, the total number of CNOT gates, the βββ, γγγ parameters used, the expectation value βββ, γγγ H βββ, γγγ , and the probability of selecting a ground state (GSP). The null hypothesis is rejected at a significance level based on the value of χ as compared against the upper-tail critical values of the χ distribution k /2−1 −x /2 x e f (x, k) = , (4) k /2 2 Γ(k/2) where k is the number of degrees of freedom in the observed data. For example, with five degrees of freedom a χ value of 11.070 indicates a rejection of the null hypothesis with 95% significance level. In principle, the value of χ is itself a measure of fairness for a set of observational data, however we found it a poor metric for our application. This is because the actual significance of a given χ value is dependent on the size of the data set as well as the number of degrees of freedom. Since our circuits reported widely varying numbers of ground state observations, and had different numbers of degenerate ground states, comparing χ values across circuits did not compare apples to apples. One could instead report the significance level at which the observational data rejects the fair sam- pling hypothesis, as that takes sample size and degrees of freedom into account. However, current hardware rejects the fair sampling hypothesis at significance levels so close to 100% that numerical differences between rejection levels were difficult to meaningfully interpret. Instead, determining how many samples from a distribution are necessary (on average) to reject fair sampling at 95% significance can be done with observational data of any size and degrees of freedom, and provides an intuitive measure of the fairness of the underlying qubits. A formal description of our algorithm for determining the number of shots to reject fair sam- pling can be found in Appendix B, here we describe the method in general terms. Based on our observational data of a given circuit C on a set of qubits Q, we determined the relative frequency of observing each ground state. We then generated 1,000 synthetic samples, each containing n ground state observations matching the relative frequency in the observed data. For each sample 2 1 we calculated χ , then took the median value for the set of 1,000 samples. We repeated for increas- ing values of n until we arrived at a median value of χ matching the level necessary to reject the We found that taking fewer than 1,000 samples resulted in inconsistent median values, and more then 1,000 samples resulted in slow performance. ACM Transactions on Quantum Computing, Vol. 3, No. 2, Article 8. Publication date: May 2022. 8:10 J. Golden et al. fair sampling hypothesis with 95% statistical significance given the number of degenerate ground states. As an example, with this methodology a coin biased 60/40 requires an average of 74 flips to reject fair sampling. 4 RESULTS As mentioned in the introduction, none of the qubits or circuits consistently reported fair sam- pling. Instead, we sought to understand the correlation between qubit fidelity and fair sampling. Our method for quantifying the degree of fair sampling for a circuit on a piece of hardware was de- scribed in the previous section. We used two complementary methods of quantifying qubit fidelity. The first is simply the frequency with which a given set of qubits accurately identify an optimal solution for the problem at hand. In this context, higher fidelity qubits are those that obtain a ground state more frequently. The benefit of this method is that it relies on direct observation of qubits solving a real problem, while the downside is that it can’t be used to directly compare results from different problems. Our second method for quantifying qubit fidelity is to use the hardware error rates as reported by qiskit. This method, which we will discuss in more detail later in this section, has the benefit of more accurately comparing results from different circuits. However, the practical relevance of this reported error data for our application is unclear. A broad issue for experimental tests of existing NISQ hardware is the highly stochastic nature of the results. We observed significant variance in all of our observed data, and therefore used a very large experiment count (with 7,440 distinct evaluations of the IBM Q hardware) to observe consistent trends. To be precise, each data point in the following plots is generated by specifying: a circuit to be evaluated, an IBM Q backend, and a subset of qubits on the backend which match the connectivity of the given circuit. We then collected 40,960 shots of the circuit on those specific qubits, and the results of those shots correspond to a single data point in the plots below. This large number of data points allowed us to identify repeatable positive or negative correlations, and we include lines of best fit to indicate these trends. 4.1 Ground State Probability vs. Fairness With that proviso out of the way, let us now discuss the results of directly comparing quantity (as measured by the number of ground state observations) vs. quality (as measured by the number of shots to reject fair sampling). We observed a positive correlation between ground state probability and fair sampling for three circuits, those solving the “easy” problems: (d), (e), (f), see Figure 3. Note that these circuits all have ≤3 qubits and circuit depths 40, 20, and 3, respectively. We call this “soft” unfairness, in that improving ground state probability and fairness of sampling are both accomplished by evaluating the circuit on the highest performing qubits possible. With the remaining circuits we observed more complicated behavior. For Problem (a), the 4T cir- cuit exhibits “soft” unfairness, the 4L circuit shows no variation in fairness with respect to ground state probability, and the 5T circuit has a negative correlation between ground state probability and fairness, see Figure 4. In other words, qubits that more frequently obtained ground states did so with increasing bias. All circuits for Problems (b) and (c) share this negative correlation. We call this “hard” unfairness, as improving ground state probability results in less fair results, and vice versa. We believe this “hard” unfairness comes from the fact that long circuits on noisy qubits produce increasingly random results. This means that the likelihood of selecting a ground state decreases, but the distribution across ground states is fairly even. To test this conjecture, we compared the fairness and ground state probabilities of both a “soft” and “hard” circuit as functions of the error rate of the qubits. ACM Transactions on Quantum Computing, Vol. 3, No. 2, Article 8. Publication date: May 2022. Fair Sampling Error Analysis on NISQ Devices 8:11 Fig. 3. Each data point represents 40,960 shots of the specified circuit on a fixed set of qubits on an IBM Q backend. The Quantum Volume (QV) of the backend is indicated by yellow for QV128, blue for QV32, green for QV16, and red for QV8. In these shorter circuits we observe a positive correlation (as indicated by the black lines of best fit) between ground state probability and number of shots to reject fair sampling. 4.2 Aggregate Error Rate vs. Fairness The IBM Q hardware reports the latest calibration data for a given chip at the time of evaluating a circuit. Specifically, qiskit reports the error rates for their native gateset (CX, SX, RX, and RZ) as well as individual qubit measurement error. We used this data to generate what we call an aggregate error of the chip and circuit in combination. For a circuit C acting on qubits q ,...,q , composed of a sequence of gates д ,...,д evaluated on a chip H, this aggregate error E is 1 C,H calculated via k l E = 1− (1− e ) (1− m ) , (5) C,H i i i=1 i=1 where e is the error for gate д ,andm is the measurement error for qubit q , as reported by qiskit i i i i at the time of evaluation. We believe this aggregate error metric evaluates roughly the likelihood of success of a given circuit on a specific piece of hardware. However, the usefulness of the error rates as reported by qiskit has been drawn into question [25, 32]. These experiments therefore serve as a test of the relevance of this reported error value in the context of fair sampling. In Figure 5 we see that both the soft and hard circuits report lower ground state probabilities as aggregate error rates increase, as expected. For Problem (e), we see that increased error rates results in less fair results. In other words, the errors result in systemic bias in favor of some ground states over others. For Problem (b) on the 5T architecture, we see the opposite behavior: increased error rates result in fairer solutions. We can see in Figure 5 that the likelihood of obtaining a ground state is ≈ 0.4 in the high error rate case. There are six degenerate ground states for this problem, out of 16 total states. Therefore a circuit that selected a state at random would select a ground state with 6/16 = 0.375 probability. By combining all of the data in to a single plot we can get a bigger picture view of how struc- tured vs. unstructured errors impact our results. In Figure 6 we see that fairness is highest for low aggregate error, decreases as the error approaches approximately 0.5, and then begins increasing again. This suggests that structured, i.e., biased, errors dominate in the middle of our error spec- trum. As the overall error rate increases, either due to longer circuits or worse performing qubits, the net result is more unstructured error and thus fairer results. In fact, we observed a decrease in ground state probability as aggregrate error increased for all circuits, which serves as a sanity check on our methodology. ACM Transactions on Quantum Computing, Vol. 3, No. 2, Article 8. Publication date: May 2022. 8:12 J. Golden et al. Fig. 4. In these longer circuits, involving four or five qubits and between 72 and 152 gates, we mostly observe a negative correlation between ground state probability and number of shots to reject fair sampling, with Problem (a) on the 4L and 4T architectures as the exceptions. 4.3 Comments on the IBM Q System Performance We believe these results constitute a serious evaluation of the IBM Q system. Our two most promi- nent observations based on this data are: Correlation between Ground State Probability and Fairness shifted. By specifically co- mparing the data generated in October 2020 against the data generated in August 2021, we can see how the overall fairness of the IBM Q system has evolved. Interestingly, if one only looks at the 2020 data, all circuits from Problem (a) exhibit “hard” unfairness. Running the experiments again in 2021, the 4T circuit shifted into the “soft” regime, see Figure 7. Fur- thermore, all circuits saw the slope of the fairness vs. ground state probability trend lines increase. Meanwhile, the average ground state probability across all problems decreased slightly, from 44.7% to 43.4%. The average fairness increased significantly in the new data, ACM Transactions on Quantum Computing, Vol. 3, No. 2, Article 8. Publication date: May 2022. Fair Sampling Error Analysis on NISQ Devices 8:13 Fig. 5. Results for a hard circuit, Problem (b) on the 5T architecture, as compared to results for a soft circuit, Problem (e). All circuits studied featured a negative correlation between ground state probability and aggre- gate error, as pictured here for these two circuits in the leftmost plots. For hard circuits, increase in aggregate error leads to behavior reminiscent of a random number generator, decreasing ground state probability but increasing fairness, as indicated in the top right plot. however this is due solely to a few data points in the two-qubit problems with extremely high (O(10 )) number of shots to reject fair sampling. The median number of shots to re- ject fair sampling actually decreased when looking at the new data, from 190 to 142. From this data we conclude that the newer machines are more dominated by structured error as opposed to random error. Quantum Volume only loosely correlates with fairness. The IBM metric of Quantum Volume [7] did broadly correlate with decreased aggregate error, which led to improved fairness in short circuits (see Figure 5). However, the Quantum Volume metric is only based off of the highly tuned performance of the best performing qubits available on a machine, instead of the overall hardware performance. This study, which ranges over all available qubits on a wide range of available chips, gives a more comprehensive view of performance. For example, we have plotted the results for Problem (d) evaluated on all 190 connected 3-qubit subsets on the manhattan backend (which has QV32 and 65 qubits), see Figure 8.In these plots one can see a wide range of results, and the best performing qubits for a single ACM Transactions on Quantum Computing, Vol. 3, No. 2, Article 8. Publication date: May 2022. 8:14 J. Golden et al. Fig. 6. Fairness as a function of aggregate error rate across all experiments. Fairness is highest for small and large error rates, with a decrease in the middle. This indicates that structured errors dominate in the middle regime, while unstructured errors play an increasingly important role as error increases. The log-quadratic line of best fit (in black) is included to graphically indicate these overall trends. metric (ground state probability, number of samples to reject fair sampling, and aggregate error rate) can perform quite poorly when evaluated by a different metric. Thus, this study emphasizes the need for more comprehensive benchmarking that does not only focus on random circuits evaluated on a small subset of qubits, as also discussed in [25]. 5 MEASUREMENT ERROR MITIGATION In this section we analyze how measurement error affects fairness. The measurement error in- cluded in our aggregate error metric (see Equation (5)) is for each individual qubit. This tensored approach paints an incomplete picture of measurement error, as there are correlations between qubits leading to correlated errors [28]. Therefore, we collected more thorough measurement er- ror data immediately before evaluating our fair sampling circuits. Specifically, for a given n-qubit circuit we explicitly prepared each of the 2 basis states and then measured the likelihood that the all of the other states would be observed. Using this data, we created a measurement error calibra- tion matrix for each set of qubits and used the inverse of that matrix to correct for measurement error [26]. Due to the small number of qubits in our problems under consideration, this simple approach is feasible; we relied on it rather than more advanced measurement error mitigation schemes [11, 23, 28]. Our complete results are visualized in Figure 13, here we describe the results in general. When going from the raw to the mitigated results, we tracked the change in ground state probability, ACM Transactions on Quantum Computing, Vol. 3, No. 2, Article 8. Publication date: May 2022. Fair Sampling Error Analysis on NISQ Devices 8:15 Fig. 7. Results for Problem (a) on the 4T architecture. Data collected across the IBM Q system in October 2020 showed a negative correlation between ground state probability and number of shots to reject fair sampling, whereas data collected in August 2021 showed a positive correlation. This indicates that the newer dates are more dominated by structured errors. number of shots to reject fair sampling, and the slope of the line of best fit correlating the two. We found mixed results in terms of ground state probability and fairness. Averaged over all problems, ground state probability improved by an average of 2.5%. Meanwhile, the mean number of shots to reject fair sampling decreased by an average of 19.1%. Finally, we observed that the slope of the line of best fit increased on 9 out of 10 problems, with an average improvement of 79 .8%. These results show that accounting for measurement error slightly improves ground state probability at the expense of fairness. The increase in slope suggests that the mitigated results are more dominated by structured error than the raw data. This result may appear somewhat surprising, as measurement error on these devices is itself a structured error: qubits prepared in the 0 state are far less likely to be measured as 1 than vice versa. However, in the raw data we observe that ground states that are composed mostly of 0 qubits are disfavored by an average of 2% (i.e., they appear 2% less frequently than expected from a perfectly fair sampler). Meanwhile, states that are composed of a majority of 1 qubits are favored by an average of 3.4%. It is unclear what errors are occurring during the evaluation of these circuits which lead to this bias in favor of 1, however it is likely measurement error is in fact helping results appear more fair by creating a slight counterbalancing bias in favor of 0. Indeed, after imposing our measurement error mitigation scheme we see that ground states that are composed mostly of 0 qubits are disfavored by an average of 5.9%, and those with mostly 1 qubits are favored by an average of 8%. By accounting for the measurement bias in favor of 0, we have highlighted the fact that at least some of the purported fairness of these devices was in fact due to multiple biases partially cancelling each other out. We see from this simple example that fair results can be produced by three different quantum computers: those dominated by random noise, those with biased errors that nearly cancel each other out, and those with low error rates. When determining the quality of these and future quan- tum computers, detailed analysis such as this is helpful in understanding which of the above sce- narios best describes the device at hand. ACM Transactions on Quantum Computing, Vol. 3, No. 2, Article 8. Publication date: May 2022. 8:16 J. Golden et al. Fig. 8. Detailed results for Problem (d) evaluated on all 3-qubit subsets of the manhattan backend. Each plot represents the same underlying data, and the set of qubits with the maximum ground state probability (max GSP), maximum number of shots to reject fair sampling (max NSRFS), and minimum aggregate error (min AR) are highlighted. This plot shows the high variance in results from a single backend. 6 CONCLUSION We have studied the extent to which today’s gate-based quantum computing hardware can fairly sample from the space of optimal solutions for discrete optimization problems. First, we introduced a novel metric for evaluating the fairness of current hardware: the number of shots to reject fair sampling. Using this metric, we found that hardware errors had a significant impact on fairness. For short circuits with ≤3 qubits, higher fidelity qubits resulted in increased fairness. We call these circuits “soft.” However, for longer circuits with ≥4 qubits, higher fidelity qubits actually resulted decreased fairness. We call these circuits “hard.” This is because low fidelity hardware begins behaving like a random number generator for long circuits, which contains no information but is relatively fair. It would be interesting to test multiple quantum computing hardware technologies (e.g., optical, trapped ions, etc.) to see if they have different fairness profiles. Our studies show that individual machines on the IBM Q backend can have dramatically different fairness and ground state proba- bilities when different qubits are used. As quantum computing hardware improves, we hope and expect to see an increase in qubit fairness and consistency. In this study we have also briefly explored the ways in which mitigating errors can affect fair- ness. Specifically, we observed that measurement error was partially correcting other structured er- rors. Measurement error mitigation therefore decreased fairness while modestly increasing ground state probability. We believe that studying fairness in addition to ground state probability can give a more nuanced picture of the errors affecting NISQ hardware. This is important for two reasons. First, random benchmark circuits are less susceptible to structured errors than circuits designed to solve realistic problems. And second, reported calibration and error rates reported are known to present an incomplete picture of how ordered circuits will perform [25]. This can be slightly im- proved upon by employing just-in-time noise data [32]. However, a broader theory of how different types of errors (read-out, crosstalk, etc.) affect not only ground state probability but also fairness is worth future study. This may involve utilizing different circuits or more refined statistical tools to highlight individual sources of error. ACM Transactions on Quantum Computing, Vol. 3, No. 2, Article 8. Publication date: May 2022. Fair Sampling Error Analysis on NISQ Devices 8:17 APPENDICES A IBM Q BACKEND INFORMATION Table 3. Details of IBM Q Backends to Be Studied Backend QV Qubits 2L 3L 4L 4T 5T manhattan 32 65 72 190 232 96 92 montreal 32 27 28 74 80 48 36 toronto 32 27 28 74 80 48 36 rome 32 5 4 6 4 0 0 santiago 32 5 4 6 4 0 0 October 2020 bogota 32 5 4 6 4 0 0 valencia 16 5 4 8 4 6 2 vigo 16 5 4 8 4 6 2 melbourne 8 15 20 72 130 66 120 ourense 8 5 4 8 4 6 2 montreal 128 27 28 74 80 48 36 mumbai 128 27 28 74 80 48 36 manhattan 32 65 72 190 232 96 92 brooklyn 32 65 72 190 232 96 92 toronto 32 27 28 74 80 48 36 August 2021 sydney 32 27 28 74 80 48 36 guadalupe 32 16 16 40 40 24 16 bogota 32 5 4 6 4 0 0 lagos 32 7 6 14 8 12 4 jakarta 16 7 6 14 8 12 4 lima 8 5 4 8 4 6 2 The name of the backend, the Quantum Volume (QV), the number of qubits, and the number of subsets of qubits on each device matching the relevant circuit architectures (2L, 3L, 4L, 4T, and 5T) are listed. B ALGORITHM FOR NUMBER OF SHOTS TO REJECT FAIR SAMPLING For the calculations in this paper we used n = 40960 and n = 1000. As explained in the main s i text, these values were chosen via a ‘goldilocks’ principle: much smaller, and the stochastic nature of the experiments resulted in inconsistent results, much larger and the computational burden was considerable. It is also worth noting that this algorithm becomes less useful as the samples become more evenly distributed, e.g., calculating the number of shots to reject fair sampling on a coin biased 50.001/49.999 takes O(1hr). ACM Transactions on Quantum Computing, Vol. 3, No. 2, Article 8. Publication date: May 2022. 8:18 J. Golden et al. ALGORITHM 1: Number of Shots to Reject Fair Sampling Input: Qubits Q, quantum circuit C for a problem P with k degenerate ground states labeled s ,..., s ; n number of shots, n number of inner loops 1 k s i Output: Number N of shots to reject fair sampling for C on Q. 1: Execute n shots of circuit C on qubits Q 2: o := the number of times ground state s observed i i 3: w := o / o  relative frequency of ground state s i i i i i=1 4: chiSqToReject := the χ value necessary to reject the fair sampling hypothesis with 95% sta- tistical significance for k − 1 degrees of freedom 5: function medianChiSq( N ) 6: chiSqArray := {} 7: while length(chiSqArray) < n do 8: testSamples := N samples drawn from the set {s } with weights {w } i i 9: testChiSq := χ for testSamples 10: append(chiSqArray, testChiSq) 11: end while 12: return median(chiSqArray) 13: end function 14: N := 2  χ ill-defined on sets of length 1 15: while medianChiSq (N ) < chiSqToReject do 16: N := 2N 17: end while 18: upperBound := N 19: lowerBound := N/2 20: while upperBound-lowerBound > 2 do 21: N := (upperBound+ lowerBound)/2 22: if medianChiSq (N ) < chiSqToReject then 23: lowerBound := N 24: else 25: upperBound := N 26: end if 27: end while 28: return N C DETAILS ON CIRCUIT COMPILATION In this appendix we provide supplementary information detailing the specifics of our compilation techniques. Full details of every hand compilation procedure would be too lengthy, instead we illustrate the compilation of Problem (d) as an example. Before we delve into details, we mention some broad ideas which aided in the compilation. • Circuit parameters: The values for βββ andγγγ in Table 2 were found by conducting a grid search over the angles (β,γ ) ∈ [−π, 0) × [−π, π ) with a resolution of π/60 and calculating the expectation value βββ,γγγ H βββ,γγγ classically (where the search space for β was cut in half due to a (β,γ )  (−β,−γ ) symmetry of the expectation value). This was efficient due to the relatively low circuit depths involved; for more complex circuits the hybrid approach detailed in Section 2.2 is more appropriate. ACM Transactions on Quantum Computing, Vol. 3, No. 2, Article 8. Publication date: May 2022. Fair Sampling Error Analysis on NISQ Devices 8:19 n n Fig. 9. (left) A 1-level Grover Mixer QAOA for an unconstrained optimization problem, U |↑  = |→ , n n −iβ|→ → | n n U (β) = e , and the Hamiltonian |→ → |  Id + X + X X + X X X +... M i i i,j i j i j i,j,k k which has also been proposed to achieve fair sampling in quantum annealing [21], going beyond limited higher-order terms. (right) For an unconstrained Ising problem with only quadratic terms in the Hamiltonian H on q ,...,q , C 0 n−1 n ⊗n−1 n n−1 we break symmetries by setting q = |↑ and U |↑  = Id ⊗ H |↑  = ↑→ . The quadratic terms of 0 S H involving q turn into linear terms of a Hamiltonian H on q ,...,q , thereby reducing q to a classical C 0 1 n−1 0 −β /π control bit. The multi-control-Z gate is fully symmetric, thus we may swap controls and target. Fig. 10. Two compilations of a large multi-control phase-shift gate (left) to gates with fewer controls: (middle) Direct decomposition into single-qubit phase-shift gates, large Toffolis, CNOTs, and a smaller multi-control phase-shift gate. (right) Decomposition using a ↑ -initialized ancilla into which we compute an AND (i.e., a Toffoli with mismatched phases), followed by a smaller multi-control phase-shift gate and an uncomputation of the AND. • Multi-controlled phase-shifts: First, we note that since terms in an Ising Hamiltonian pair- wise commute, implementing the phase separating unitary can be done for each term indi- ∓2γ /π vidually, with 2 CNOTs and one phase-shift gate Z . Care has to be taken, however, to address the connectivity constraints in the architectures. More difficult is the compilation of the multi-control phase-shift gate at the heart of the Grover Mixer. Figure 10 gives two recursive compilations into gates with fewer controls, one without ancillas, and one with a |↑-initialized ancilla. We note that careful recursive calls to these compilations and using symmetries can result in some of the smaller multi-control gates to cancel each other. • SWAP operations: We allow SWAP operations for qubits to change location in the circuit without having to go back to the original location. This is possible because the Grover Mixer is fully symmetric, i.e., we can arbitrarily exchange controls with the phase-shift gate, thus the input order to the Grover Mixer does not matter. We track qubit SWAPs with a permuta- tion array from which in the end we read a read-out map for the measurements. This gives us a significant advantage over transpilation tools which are not aware of such symmetries. Introducing SWAPs (which usually cost 3 CNOTs) at the right place in a circuit (next to a CNOT) will cost only 1 CNOT. Similarly, moving SWAPs through to the beginning or end of the circuit results in complete removal of the SWAP cost, as the SWAP gate can be replaced with a change in the permutation array and corresponding readout. Example: Problem (d) Let us now describe in detail the compilation of Problem (d) to the 3L architecture. Starting with the general form of the 1-level Grover Mixer QAOA with fixed qubit q :=↑, we implement each ∓2γ /π quadratic term of the Ising Hamiltonian with 2 CNOTs and a single-qubit phase-shift gate Z = ( ). ∓iγ 0 e ACM Transactions on Quantum Computing, Vol. 3, No. 2, Article 8. Publication date: May 2022. 8:20 J. Golden et al. Fig. 11. Phase Separator Compilation, Problem (d): Implementing the state preparation U and the phase −iγ (−Z Z +Z Z +Z Z +Z Z ) 0 1 1 2 1 3 2 3 separator U (γ ) = e on a Linear Nearest Neighbor architecture. adratic Qu terms commute pairwise, so they can be implemented individually with 2 CNOTs and one phase-shift gate each. Circuit size is reduced in three ways: (i) Setting q :=↑ results in q acting as a classical control bit, which 0 0 can be removed. (ii) Directly combining a single quadratic-term phase separator with a SWAP gate increases the CNOT count by 1 (instead of the usual 3). (iii) We keep track of the resulting permutation of the qubits instead of undoing the SWAPs. Fig. 12. Grover Mixer Compilation, Problem (d): Implementing the grover mixer unitary U (β) according to the high-level approch for multi-control phase-shift gates in Figure 10. Circuit size is reduced in two ways: (i) Moving SWAP gates through the circuit such that they appear at the beginning and the end, and thus can be tracked in the qubit permutation, rather than implemented. (ii) Moving (multi-)control gates through the circuit such that they can be cancelled with adjoint gates (not applicable in this example). The CNOTs with control q can be removed, leaving q in the role of a classical bit, see Fig- 0 0 ure 11. We notice that the other three quadratic terms form a triangle in the problem graph, hence we cannot map them to a 3L Linear Nearest Neighbor architecture, but rather need at least one SWAP. Inserting the SWAP as 3 CNOTs right to an existing CNOT of the phase separating uni- −iγ (Z Z ) 2 3 tary e gives a cancellation of 2 CNOTs, thus the addition of the SWAP results in only 1 additional CNOT. Rather than reversing this SWAP at the end of U (γ ), we keep track of the permutation of the qubits (q and q have swapped places). 2 3 This end configuration (swapped qubits) now builds the start of the Grover Mixer unitary. The X-gates and the control on qubit q can be dropped, as explained in Figure 9 (right). Using the recursive decomposition from Figure 10,weagain seethatweneed to addaSWAP,see Figure 12. The remaining single-controlled phase-shift gate between the middle two circuit wires can be im- plemented with 2 CNOTs and 3 phase-shift gates as shown. Keeping track of the SWAP throughout the end of the circuit, we see that qubits q and q end up back in their original position–in general, 2 3 this need not be the case. Comment on 5-qubit Ising Problems on 5T Using an Ancilla. Since an ancilla offers no benefit for the implementation of the phase separating unitary, and due to limited connectivity in the consid- ered architectures, we found that decomposing multi-controlled phase-shift gates using an ancilla as showninFigure 10 (right) can only be used advantageously for the 5-qubit Ising Problems (a) and (b). ACM Transactions on Quantum Computing, Vol. 3, No. 2, Article 8. Publication date: May 2022. Fair Sampling Error Analysis on NISQ Devices 8:21 Following the approach outlined for Problem (d), both Problems (a) and (b) result in a multi- controlled phase-shift gate acting on 4 qubits. Using a remaining qubit on the 5T architecture as a |↑-initialized ancilla (which has to be swapped around), we can implement the mentioned decomposition, which should return the ancilla to the |↑-state. As mentioned in the primary text, we discarded all samples where the ancilla qubit was not measured in the |↑ state. D MEASUREMENT ERROR MITIGATION RESULTS Fig. 13. Results of measurement error mitigation across all problems. Ground state probability improved by an average of 2.5%, number of shots to reject fair sampling decreased by an average of 19.1%, and the slope of the line correlating the two increased by an average of 79.8%. ACM Transactions on Quantum Computing, Vol. 3, No. 2, Article 8. Publication date: May 2022. 8:22 J. Golden et al. REFERENCES [1] Mahabubul Alam, Abdullah Ash-Saki, and Swaroop Ghosh. 2020. Design-space exploration of quantum approximate optimization algorithm under noise. In 2020 IEEE Custom Integrated Circuits Conference (CICC).1–4. https://doi.org/ 10.1109/CICC48029.2020.9075903 [2] Frank Arute, Kunal Arya, Ryan Babbush, Dave Bacon, Joseph C. Bardin, Rami Barends, Rupak Biswas, Sergio Boixo, Fernando GSL Brandao, David A. Buell, et al. 2019. Quantum supremacy using a programmable superconducting processor. Nature 574, 7779 (2019), 505–510. https://doi.org/10.1038/s41586-019-1666-5 [3] Sergey Bravyi, Alexander Kliesch, Robert Koenig, and Eugene Tang. 2020. Obstacles to variational quantum opti- mization from symmetry protection. Physical Review Letters 125 (2020), 260505. Issue 26. arXiv:1910.08980, https: //doi.org/10.1103/PhysRevLett.125.260505 [4] Andreas Bärtschi and Stephan Eidenbenz. 2020. Grover mixers for QAOA: Shifting complexity from mixer de- sign to state preparation. In IEEE International Conference on Quantum Computing & Engineering (QCE’20). 72–82. arXiv:2006.00354, https://doi.org/10.1109/QCE49297.2020.00020 [5] Andreas Bärtschi and Stephan Eidenbenz. 2021. Grover Mixer QAOA: Feasible State Superpositions, Convergence to Optimum, and Limitations. (2021). in preparation. [6] Jeremy Cook, Stephan Eidenbenz, and Andreas Bärtschi. 2020. The quantum alternating operator ansatz on max- imum k-vertex cover. In IEEE International Conference on Quantum Computing & Engineering (QCE’20). 83–92. arXiv:1910.13483, https://doi.org/10.1109/QCE49297.2020.00021 [7] Andrew W. Cross, Lev S. Bishop, Sarah Sheldon, Paul D. Nation, and Jay M. Gambetta. 2019. Validating quantum computers using randomized model circuits. Physical Review A 100, 3 (2019), 032328. arXiv:1811.12926, https://doi. org/10.1103/PhysRevA.100.032328 [8] Edward Farhi, Jeffrey Goldstone, and Sam Gutmann. 2014. A quantum approximate optimization algorithm. arXiv e-prints (2014). arXiv:1411.4028. [9] Edward Farhi, Jeffrey Goldstone, and Sam Gutmann. 2014. A quantum approximate optimization algorithm applied to a bounded occurrence constraint problem. arXiv e-prints (2014). arXiv:1412.6062. [10] Edward Farhi, Jeffrey Goldstone, Sam Gutmann, and Michael Sipser. 2000. Quantum computation by adiabatic evolu- tion. arXiv e-prints (2000). arXiv:quant-ph/0001106. [11] Lena Funcke, Tobias Hartung, Karl Jansen, Stefan Kühn, Paolo Stornati, and Xiaoyang Wang. 2020. Measurement error mitigation in quantum computers through classical bit-flip correction. (7 2020). arXiv: quant-ph/2007.03663. [12] Lov K. Grover. 2005. Fixed-point quantum search. Physical Review Letters 95, 15 (2005), 150501. https://doi.org/10.1103/ PhysRevLett.95.150501 [13] Stuart Hadfield, Zhihui Wang, Bryan O’Gorman, Eleanor G. Rieffel, Davide Venturelli, and Rupak Biswas. 2019. From the quantum approximate optimization algorithm to a quantum alternating operator ansatz. Algorithms 12, 2 (2019), 34. arXiv:1709.03489, https://doi.org/10.3390/a12020034 [14] Dylan R. Harp, Zhenxue Dai, Andrew V. Wolfsberg, Jasper A. Vrugt, Bruce A. Robinson, and Velimir V. Vesselinov. 2008. Aquifer structure identification using stochastic inversion. Geophysical Research Letters 35, 8 (2008), L08404. https://doi.org/10.1029/2008GL033585 [15] Matthew P. Harrigan et al. 2021. Quantum approximate optimization of non-planar graph problems on a planar su- perconducting processor. Nature Physics 17, 3 (2021), 332–336. arXiv:2004.04197, https://doi.org/10.1038/s41567-020- 01105-y [16] Tadashi Kadowaki and Hidetoshi Nishimori. 1998. Quantum annealing in the transverse Ising model. Physical Review E 58, 5 (1998), 5355. arXiv:cond-mat/9804280, https://doi.org/10.1103/PhysRevE.58.5355 [17] Vaibhaw Kumar, Casey Tomlin, Curt Nehrkorn, Daniel O’Malley, and Joseph Dulny III. 2020. Achieving fair sampling in quantum annealing. arXiv e-prints (2020). arXiv:2007.08487. [18] Mario S. Könz, Guglielmo Mazzola, Andrew J. Ochoa, Helmut G. Katzgraber, and Matthias Troyer. 2019. Uncertain fate of fair sampling in quantum annealing. Physical Review A 100, 3 (2019), 030303. arXiv:1806.06081, https://doi.org/ 10.1103/PhysRevA.100.030303 [19] Salvatore Mandra, Zheng Zhu, and Helmut G. Katzgraber. 2017. Exponentially biased ground-state sampling of quan- tum annealing machines with transverse-field driving Hamiltonians. Physical Review Letters 118, 7 (2017), 070502. arXiv:1606.07146, https://doi.org/10.1103/PhysRevLett.118.070502 [20] Yoshiki Matsuda, Hidetoshi Nishimori, and Helmut G. Katzgraber. 2009. Ground-state statistics from annealing algo- rithms: Quantum versus classical approaches. New Journal of Physics 11, 7 (2009), 073021. https://doi.org/10.1088/1367- 2630/11/7/073021 arXiv:0808.0365 [21] Yoshiki Matsuda, Hidetoshi Nishimori, and Helmut G. Katzgraber. 2009. Quantum annealing for problems with ground-state degeneracy. Journal of Physics: Conference Series 143 (2009), 012003. https://doi.org/10.1088/1742-6596/ 143/1/012003 ACM Transactions on Quantum Computing, Vol. 3, No. 2, Article 8. Publication date: May 2022. Fair Sampling Error Analysis on NISQ Devices 8:23 [22] Mauro E. S. Morales, Timur Tlyachev, and Jacob Biamonte. 2018. Variational learning of Grover’s quantum search algorithm. Physical Review A 98, 6 (2018), 062333. arXiv:1805.09337, https://doi.org/10.1103/PhysRevA.98.062333 [23] Paul D. Nation, Hwajung Kang, Neereja Sundaresan, and Jay M. Gambetta. 2021. Scalable mitigation of measurement errors on quantum computers. arXiv e-prints (2021). arXiv:2108.12518. [24] Daniel O’Malley. 2018. An approach to quantum-computational hydrologic inverse analysis. Scientific Reports 8, 1 (2018), 1–9. https://doi.org/10.1038/s41598-018-25206-0 [25] Timothy Proctor, Kenneth Rudinger, Kevin Young, Erik Nielsen, and Robin Blume-Kohout. 2020. Measuring the capa- bilities of quantum computers. arXiv e-prints (2020). arXiv:2008.11294. [26] Qiskit Development Team. Qiskit Documentation: Measurement Error Mitigation. (n.d.). https://qiskit.org/ documentation/tutorials/noise/3_measurement_error_mitigation.html. [27] Lukas M. Sieberer and Wolfgang Lechner. 2018. Programmable superpositions of Ising configurations. Physical Review A 97, 5 (2018), 052329. arXiv:1708.02533, https://doi.org/10.1103/PhysRevA.97.052329 [28] Swamit S. Tannu and Moinuddin K. Qureshi. 2019. Mitigating measurement errors in quantum computers by exploit- ing state-dependent bias. In Proceedings of the 52nd Annual IEEE/ACM International Symposium on Microarchitecture (MICRO’52). Association for Computing Machinery, New York, NY, USA, 279–290. https://doi.org/10.1145/3352460. [29] Marc Vuffray, Carleton Coffrin, Yaroslav A. Kharkov, and Andrey Y. Lokhov. 2020. Programmable quantum annealers as noisy Gibbs Samplers. (2020). arXiv:quant-ph/2012.08827. [30] Samson Wang, Enrico Fontana, Marco Cerezo, Kunal Sharma, Akira Sone, Lukasz Cincio, and Patrick J. Coles. 2020. Noise-induced barren plateaus in variational quantum algorithms. arXiv e-prints (2020). arXiv:2007.14384. [31] Zhihui Wang, Nicholas C. Rubin, Jason M. Dominy, and Eleanor G. Rieffel. 2020. XY mixers: Analytical and numerical results for the Quantum Alternating Operator Ansatz. Physical Review A 101, 1 (2020), 012320. arXiv:1904.09314, https: //doi.org/10.1103/PhysRevA.101.012320 [32] Ellis Wilson, Sudhakar Singh, and Frank Mueller. 2020. Just-in-time quantum circuit transpilation reduces noise. In IEEE International Conference on Quantum Computing & Engineering (QCE’20). 345–355. arXiv:2005.12820, https://doi. org/10.1109/QCE49297.2020.00050 [33] Jonathan Wurtz and Peter Love. 2021. MaxCut quantum approximate optimization algorithm performance guarantees for p > 1. Physical Review A 103 (2021), 042612. Issue 4. arXiv:2010.11209, https://doi.org/10.1103/PhysRevA.103.042612 [34] Masayuki Yamamoto, Masayuki Ohzeki, and Kazuyuki Tanaka. 2020. Fair sampling by simulated annealing on quan- tum annealer. Journal of the Physical Society of Japan 89, 2 (2020), 025002. arXiv:1912.10701, https://doi.org/10.7566/ JPSJ.89.025002 [35] Theodore J. Yoder, Guang Hao Low, and Isaac L. Chuang. 2014. Fixed-point quantum search with an optimal number of queries. Physical Review Letters 113, 21 (2014), 210501. arXiv:1409.3305, https://doi.org/10.1103/PhysRevLett.113. [36] Brian Hu Zhang, Gene Wagenbreth, Victor Martin-Mayor, and Itay Hen. 2017. Advantages of unfair quantum ground- state sampling. Scientific Reports 7 (2017), 1044. arXiv:1701.01524, https://doi.org/10.1038/s41598-017-01096-6 Received January 2021; revised September 2021; accepted January 2022 ACM Transactions on Quantum Computing, Vol. 3, No. 2, Article 8. Publication date: May 2022.

Journal

ACM Transactions on Quantum Computing – Association for Computing Machinery

Published: May 24, 2022

Keywords: Quantum alternating operator ansatz

There are no references for this article.