Publications

Quantum Architecture Search with Unsupervised Representation Learning

Yize Sun, Zixin Wu, Yunpu Ma, Volker Tresp

Unsupervised representation learning presents new opportunities for advancing Quantum Architecture Search (QAS) on Noisy Intermediate-Scale Quantum (NISQ) devices. QAS is designed to optimize quantum circuits for Variational Quantum Algorithms (VQAs). Most QAS algorithms tightly couple the search space and search algorithm, typically requiring the evaluation of numerous quantum circuits, resulting in high computational costs and limiting scalability to larger quantum circuits. Predictor-based QAS algorithms mitigate this issue by estimating circuit performance based on structure or embedding. However, these methods often demand time-intensive labeling to optimize gate parameters across many circuits, which is crucial for training accurate predictors. Inspired by the classical neural architecture search algorithm Arch2vec, we investigate the potential of unsupervised representation learning for QAS without relying on predictors. Our framework decouples unsupervised architecture representation learning from the search process, enabling the learned representations to be applied across various downstream tasks. Additionally, it integrates an improved quantum circuit graph encoding scheme, addressing the limitations of existing representations and enhancing search efficiency. This predictor-free approach removes the need for large labeled datasets. During the search, we employ REINFORCE and Bayesian Optimization to explore the latent representation space and compare their performance against baseline methods. Our results demonstrate that the framework efficiently identifies high-performing quantum circuits with fewer search iterations.

Warm-Start Variational Quantum Policy Iteration

Nico Meyer, Jakob Murauer, Alexander Popov, Christian Ufrecht, Axel Plinge, Christopher Mutschler, Daniel D. Scherer

Reinforcement learning is a powerful framework aiming to determine optimal behavior in highly complex decision-making scenarios. This objective can be achieved using policy iteration, which requires to solve a typically large linear system of equations. We propose the variational quantum policy iteration (VarQPI) algorithm, realizing this step with a NISQ-compatible quantum-enhanced subroutine. Its scalability is supported by an analysis of the structure of generic reinforcement learning environments, laying the foundation for potential quantum advantage with utility-scale quantum computers. Furthermore, we introduce the warm-start initialization variant (WS-VarQPI) that significantly reduces resource overhead. The algorithm solves a large FrozenLake environment with an underlying 256x256-dimensional linear system, indicating its practical robustness.

Qiskit-Torch-Module: Fast Prototyping of Quantum Neural Networks

Nico Meyer, Christian Ufrecht, Maniraman Periyasamy, Axel Plinge, Christopher Mutschler, Daniel D. Scherer, Andreas Maier

Quantum computer simulation software is an integral tool for the research efforts in the quantum computing community. An important aspect is the efficiency of respective frameworks, especially for training variational quantum algorithms. Focusing on the widely used Qiskit software environment, we develop the qiskit-torch-module. It improves runtime performance by two orders of magnitude over comparable libraries, while facilitating low-overhead integration with existing codebases. Moreover, the framework provides advanced tools for integrating quantum neural networks with PyTorch. The pipeline is tailored for single-machine compute systems, which constitute a widely employed setup in day-to-day research efforts.

Comprehensive Library of Variational LSE Solvers

Nico Meyer, Martin Röhn, Jakob Murauer, Axel Plinge, Christopher Mutschler, Daniel D. Scherer

Linear systems of equations can be found in various mathematical domains, as well as in the field of machine learning. By employing noisy intermediate-scale quantum devices, variational solvers promise to accelerate finding solutions for large systems. Although there is a wealth of theoretical research on these algorithms, only fragmentary implementations exist. To fill this gap, we have developed the variational-lse-solver framework, which realizes existing approaches in literature, and introduces several enhancements. The user-friendly interface is designed for researchers that work at the abstraction level of identifying and developing end-to-end applications.

Differentiable Quantum Architecture Search For Job Shop Scheduling Problem

Yize Sun, Jiarui Liu, Yunpu Ma, Volker Tresp

The Job shop scheduling problem (JSSP) plays a pivotal role in industrial applications, such as signal processing (SP) and steel manufacturing, involving sequencing machines and jobs to maximize scheduling efficiency. Before, JSSP was solved using manually defined circuits by variational quantum algorithm (VQA). Finding a good circuit architecture is task-specific and time-consuming. Differentiable quantum architecture search (DQAS) is a gradient-based framework that can automatically design circuits. However, DQAS is only tested on quantum approximate optimization algorithm (QAOA) and error mitigation tasks. Whether DQAS applies to JSSP based on a more flexible algorithm, such as variational quantum eigensolver (VQE), is still open for optimization problems. In this work, we redefine the operation pool and extend DQAS to a framework JSSP-DQAS by evaluating circuits to generate circuits for JSSP automatically. The experiments conclude that JSSP-DQAS can automatically find noise-resilient circuit architectures that perform much better than manually designed circuits. It helps to improve the efficiency of solving JSSP.

SA-DQAS: Self-attention Enhanced Differentiable Quantum Architecture Search

Yize Sun, Jiarui Liu, Zixin Wu, Zifeng Ding, Yunpu Ma, Thomas Seidl, Volker Tresp

We introduce SA-DQAS in this paper, a novel framework that enhances the gradient-based Differentiable Quantum Architecture Search (DQAS) with a self-attention mechanism, aimed at optimizing circuit design for Quantum Machine Learning (QML) challenges. Analogous to a sequence of words in a sentence, a quantum circuit can be viewed as a sequence of placeholders containing quantum gates. Unlike DQAS, each placeholder is independent, while the self-attention mechanism in SA-DQAS helps to capture relation and dependency information among each operation candidate placed on placeholders in a circuit. To evaluate and verify, we conduct experiments on job-shop scheduling problems (JSSP), Max-cut problems, and quantum fidelity. Incorporating self-attention improves the stability and performance of the resulting quantum circuits and refines their structural design with higher noise resilience and fidelity. Our research demonstrates the first successful integration of self-attention with DQAS.

Model-based Offline Quantum Reinforcement Learning

Simon Eisenmann, Daniel Hein, Steffen Udluft, Thomas A. Runkler

This paper presents the first algorithm for model-based offline quantum reinforcement learning and demonstrates its functionality on the cart-pole benchmark. The model and the policy to be optimized are each implemented as variational quantum circuits. The model is trained by gradient descent to fit a pre-recorded data set. The policy is optimized with a gradient-free optimization scheme using the return estimate given by the model as the fitness function. This model-based approach allows, in principle, full realization on a quantum computer during the optimization phase and gives hope that a quantum advantage can be achieved as soon as sufficiently powerful quantum computers are available.

Guided-SPSA: Simultaneous Perturbation Stochastic Approximation assisted by the Parameter Shift Rule

Maniraman Periyasamy, Axel Plinge, Christopher Mutschler, Daniel D. Scherer, Wolfgang Mauerer

The study of variational quantum algorithms (VQCs) has received significant attention from the quantum computing community in recent years. These hybrid algorithms, utilizing both classical and quantum components, are well-suited for noisy intermediate-scale quantum devices. Though estimating exact gradients using the parameter-shift rule to optimize the VQCs is realizable in NISQ devices, they do not scale well for larger problem sizes. The computational complexity, in terms of the number of circuit evaluations required for gradient estimation by the parameter-shift rule, scales linearly with the number of parameters in VQCs. On the other hand, techniques that approximate the gradients of the VQCs, such as the simultaneous perturbation stochastic approximation (SPSA), do not scale with the number of parameters but struggle with instability and often attain suboptimal solutions. In this work, we introduce a novel gradient estimation approach called Guided-SPSA, which meaningfully combines the parameter-shift rule and SPSA-based gradient approximation. The Guided-SPSA results in a 15% to 25% reduction in the number of circuit evaluations required during training for a similar or better optimality of the solution found compared to the parameter-shift rule. The Guided-SPSA outperforms standard SPSA in all scenarios and outperforms the parameter-shift rule in scenarios such as suboptimal initialization of the parameters. We demonstrate numerically the performance of Guided-SPSA on different paradigms of quantum machine learning, such as regression, classification, and reinforcement learning.

BCQQ: Batch-Constraint Quantum Q-Learning with Cyclic Data Re-uploading

Maniraman Periyasamy, Marc Hölle, Marco Wiedmann, Daniel D. Scherer, Axel Plinge, Christopher Mutschler

Deep reinforcement learning (DRL) often requires a large number of data and environment interactions, making the training process time-consuming. This challenge is further exacerbated in the case of batch RL, where the agent is trained solely on a pre-collected dataset without environment interactions. Recent advancements in quantum computing suggest that quantum models might require less data for training compared to classical methods. In this paper, we investigate this potential advantage by proposing a batch RL algorithm that utilizes VQC as function approximators within the discrete batch-constraint deep Q-learning (BCQ) algorithm. Additionally, we introduce a novel data re-uploading scheme by cyclically shifting the order of input variables in the data encoding layers. We evaluate the efficiency of our algorithm on the OpenAI CartPole environment and compare its performance to the classical neural network-based discrete BCQ.

Batch Quantum Reinforcement Learning

Maniraman Periyasamy, Marc Hölle, Marco Wiedmann, Daniel D. Scherer, Axel Plinge, Christopher Mutschler

Training DRL agents is often a time-consuming process as a large number of samples and environment interactions is required. This effect is even amplified in the case of Batch RL, where the agent is trained without environment interactions solely based on a set of previously collected data. Novel approaches based on quantum computing suggest an advantage compared to classical approaches in terms of sample efficiency. To investigate this advantage, we propose a batch RL algorithm leveraging VQC as function approximators in the discrete BCQ algorithm. Additionally, we present a novel data re-uploading scheme based on cyclically shifting the input variables' order in the data encoding layers. We show the efficiency of our algorithm on the OpenAI CartPole environment and compare its performance to classical neural network-based discrete BCQ.

A Survey on Quantum Reinforcement Learning

Nico Meyer, Christian Ufrecht, Maniraman Periyasamy, Daniel D. Scherer, Axel Plinge, Christopher Mutschler

Quantum reinforcement learning is an emerging field at the intersection of quantum computing and machine learning. While we intend to provide a broad overview of the literature on quantum reinforcement learning - our interpretation of this term will be clarified below - we put particular emphasis on recent developments. With a focus on already available noisy intermediate-scale quantum devices, these include variational quantum circuits acting as function approximators in an otherwise classical reinforcement learning setting. In addition, we survey quantum reinforcement learning algorithms based on future fault-tolerant hardware, some of which come with a provable quantum advantage. We provide both a birds-eye-view of the field, as well as summaries and reviews for selected parts of the literature.

A Comparison of Variational Quantum Circuits with Classical Neural Networks

Simon Eisenmann

This thesis explores the combination of machine learning and quantum computing using variational quantum circuits. It compares the performance of these circuits with classical neural networks on various levels. The data used are classical, but the processing is done on a quantum computer or in a simulation of a quantum computer. The training is hybrid, which means that it involves both a classical computer and a quantum computer. All experiments were conducted in simulation, as no suitable quantum computer was available at the time of writing. The results show that variational quantum circuits are effective in both supervised learning and model-based reinforcement learning but cannot match the performance and speed of classical neural networks. Data reuploading is identified as a way to improve performance, and classical neural networks seem to be more data efficient. It is suggested that variational quantum circuits may become more attractive in the future if a fast and powerful quantum computer is available.

Differentiable Quantum Architecture Search with Self-Attention Mechanism

Jiarui Liu

In recent years, more and more studies of quantum machine learning (QML) have been done. QML always solve problems with manually designed circuits that are not flexible. And finding a good circuit architecture, especially for a large circuit is pretty time-consuming. Self-attention is a mechanism in the language models which provides the semantic relations between each word in a sentence. DQAS is a gradient-based framework that can automatically design circuits. DQAS aims to compose a circuit with a sequence of operations. Analogous to a sequence of words in a sentence, self-attention can provide the relation information between each operation in a circuit. Our goal is to combine DQAS with self-attention to build relationship between each operation in the search process. We propose new DQAS which contains self-attention mechanism and use it to automatically design circuits for solving QML problems and study the fidelity of generated circuits in noisy environments. We first generate circuits using DQAS with self-attention for JSSP. We find some circuits that outperform the pre-defined circuit and circuits designed by DQAS without self-attention, and have stable performances in noisy environments. Then we compare the performances for maxcut problems of circuits generated by DQAS with random information, DQAS incorporating self-attention in different ways, and DQAS without self-attention. By using self-attention, the searched circuits have more stable and better performances, and better quantum circuit structures. We also generate circuits in ideal and noisy environments and calculate the fidelity of these generated circuits in different environments. DQAS with self-attention can generate noise-resilient circuits and circuits searched in noisy environment have higher fidelity. When choosing controlled gates to build circuits, the generated circuits suffer more from noise errors and idling errors, leading to lower fidelity. Our work is the first to show the combination of DQAS and self-attention.

QNEAT: Natural Evolution of Variational Quantum Circuit Architecture

Alessandro Giovagnoli, Yunpu Ma, Volker Tresp

Quantum Machine Learning (QML) is a recent and rapidly evolving field where the theoretical framework and logic of quantum mechanics are employed to solve machine learning tasks. Various techniques with different levels of quantum-classical hybridization have been proposed. Here we focus on variational quantum circuits (VQC), which emerged as the most promising candidates for the quantum counterpart of neural networks in the noisy intermediate-scale quantum (NISQ) era. Although showing promising results, VQCs can be hard to train because of different issues, e.g., barren plateau, periodicity of the weights, or choice of architecture. This paper focuses on this last problem for finding optimal architectures of variational quantum circuits for various tasks. To address it, we propose a gradient-free algorithm inspired by natural evolution to optimize both the weights and the architecture of the VQC. In particular, we present a version of the well-known neuroevolution of augmenting topologies (NEAT) algorithm and adapt it to the case of variational quantum circuits. We refer to the proposed architecture search algorithm for VQC as QNEAT. We test the algorithm with different benchmark problems of classical fields of machine learning i.e. reinforcement learning and combinatorial optimization.

Differentiable Quantum Architecture Search for Quantum Reinforcement Learning

Yize Sun, Yunpu Ma, Volker Tresp

Differentiable quantum architecture search (DQAS) is a gradient-based framework to design quantum circuits automatically in the NISQ era. It was motivated by such as low fidelity of quantum hardware, low flexibility of circuit architecture, high circuit design cost, barren plateau (BP) problem, and periodicity of weights. People used it to address error mitigation, unitary decomposition, and quantum approximation optimization problems based on fixed datasets. Quantum reinforcement learning (QRL) is a part of quantum machine learning and often has various data. QRL usually uses a manually designed circuit. However, the pre-defined circuit needs more flexibility for different tasks, and the circuit design based on various datasets could become intractable in the case of a large circuit. The problem of whether DQAS can be applied to quantum deep Q-learning with various datasets is still open. The main target of this work is to discover the capability of DQAS to solve quantum deep Q-learning problems. We apply a gradient-based framework DQAS on reinforcement learning tasks and evaluate it in two different environments - cart pole and frozen lake. It contains input- and output weights, progressive search, and other new features. The experiments conclude that DQAS can design quantum circuits automatically and efficiently. The evaluation results show significant outperformance compared to the manually designed circuit. Furthermore, the performance of the automatically created circuit depends on whether the super-circuit learned well during the training process. This work is the first to show that gradient-based quantum architecture search is applicable to QRL tasks.

Workshop Summary: Quantum Machine Learning

Volker Tresp, Steffen Udluft, Daniel Hein, Werner Hauptmann, Martin Leib, Christopher Mutschler, Daniel D. Scherer, Wolfgang Mauerer

Quantum computing (QC) has made significant progress in recent years, and scientists are exploring its applications across various fields, including quantum machine learning (QML). The workshop ‘Quantum Machine Learning: From Foundations to Applications’, part of the IEEE International Conference on Quantum Computing and Engineering 2023, aims to bring together researchers and industry practitioners from different disciplines to discuss the challenges and applications of QML. Quantum computing (QC) has seen remarkable progress after first commercial hardware prototypes have become available in recent years, following decades of foundational research. Scientists across all fields are seeking concrete applications for the technology. Fault-tolerant and large-scale quantum computers are still unavailable, yet many opportunities beckon for machine learning: Given reasonable complexity-theoretic assumptions, quantum algorithms are expected to be capable of outperforming known approaches to problems even on noisy, intermediate-scale quantum (NISQ) computers, and exhibit advantages over classical systems.

Quantum Policy Gradient Algorithm with Optimized Action Decoding

Nico Meyer, Daniel D. Scherer, Axel Plinge, Christopher Mutschler, Michael J. Hartmann

Quantum machine learning implemented by variational quantum circuits (VQCs) is considered a promising concept for the noisy intermediate-scale quantum computing era. Focusing on applications in quantum reinforcement learning, we propose a specific action decoding procedure for a quantum policy gradient approach. We introduce a novel quality measure that enables us to optimize the classical post-processing required for action selection, inspired by local and global quantum measurements. The resulting algorithm demonstrates a significant performance improvement in several benchmark environments. With this technique, we successfully execute a full training routine on a 5-qubit hardware device. Our method introduces only negligible classical overhead and has the potential to improve VQC-based algorithms beyond the field of quantum reinforcement learning.

Incremental Data-Uploading for Full-Quantum Classification

Maniraman Periyasamy, Nico Meyer, Christian Ufrecht, Daniel D. Scherer, Axel Plinge, Christopher Mutschler

The data representation in a machine-learning model strongly influences its performance. This becomes even more important for quantum machine learning models implemented on noisy intermediate scale quantum (NISQ) devices. Encoding high dimensional data into a quantum circuit for a NISQ device without any loss of information is not trivial and brings a lot of challenges. While simple encoding schemes (like single qubit rotational gates to encode high dimensional data) often lead to information loss within the circuit, complex encoding schemes with entanglement and data re-uploading lead to an increase in the encoding gate count. This is not well-suited for NISQ devices. This work proposes 'incremental data-uploading', a novel encoding pattern for high dimensional data that tackles these challenges. We spread the encoding gates for the feature vector of a given data point throughout the quantum circuit with parameterized gates in between them. This encoding pattern results in a better representation of data in the quantum circuit with a minimal pre-processing requirement. We show the efficiency of our encoding pattern on a classification task using the MNIST and Fashion-MNIST datasets, and compare different encoding methods via classification accuracy and the effective dimension of the model.

Quantum Natural Policy Gradients: Towards Sample-Efficient Reinforcement Learning

Nico Meyer, Daniel D. Scherer, Axel Plinge, Christopher Mutschler, Michael J. Hartmann

Reinforcement learning is a growing field in AI with a lot of potential. Intelligent behavior is learned automatically through trial and error in interaction with the environment. However, this learning process is often costly. Using variational quantum circuits as function approximators potentially can reduce this cost. In order to implement this, we propose the quantum natural policy gradient (QNPG) algorithm – a second-order gradientbased routine that takes advantage of an efficient approximation of the quantum Fisher information matrix. We experimentally demonstrate that QNPG outperforms first-order based training on different Contextual Bandits environments regarding convergence speed and stability and moreover reduces the sample complexity. Furthermore, we provide evidence for the practical feasibility of our approach by training on a 12-qubit hardware device.

An Empirical Comparison of Optimizers for Quantum Machine Learning with SPSA-based Gradients

Marco Wiedmann, Marc Hölle, Maniraman Periyasamy, Nico Meyer, Christian Ufrecht, Daniel D. Scherer, Axel Plinge, Christopher Mutschler

VQA have attracted a lot of attention from the quantum computing community for the last few years. Their hybrid quantum-classical nature with relatively shallow quantum circuits makes them a promising platform for demonstrating the capabilities of NISQ devices. Although the classical machine learning community focuses on gradient-based parameter optimization, finding near-exact gradients for VQC with the parameter-shift rule introduces a large sampling overhead. Therefore, gradient-free optimizers have gained popularity in quantum machine learning circles. Among the most promising candidates is the SPSA algorithm, due to its low computational cost and inherent noise resilience. We introduce a novel approach that uses the approximated gradient from SPSA in combination with state-of-the-art gradient-based classical optimizers. We demonstrate numerically that this outperforms both standard SPSA and the parameter-shift rule in terms of convergence rate and absolute error in simple regression tasks. The improvement of our novel approach over SPSA with stochastic gradient decent is even amplified when shot- and hardware-noise are taken into account. We also demonstrate that error mitigation does not significantly affect our results.

Quantum-Inspired Digital Annealing for Join Ordering

Manuel Schönberger, Immanuel Trummer, Wolfgang Mauerer

Finding the optimal join order (JO) is one of the most important problems in query optimisation, and has been extensively considered in research and practise. As it involves huge search spaces, approximation approaches and heuristics are commonly used, which explore a reduced solution space at the cost of solution quality. To explore even large JO search spaces, we may consider special-purpose software, such as mixed-integer linear programming (MILP) solvers, which have successfully solved JO problems. However, even mature solvers cannot overcome the limitations of conventional hardware prompted by the end of Moore's law.

We consider quantum-inspired digital annealing hardware, which takes inspiration from quantum processing units (QPUs). Unlike QPUs, which likely remain limited in size and reliability in the near and mid-term future, the digital annealer (DA) can solve large instances of mathematically encoded optimisation problems today. We derive a novel, native encoding for the JO problem tailored to this class of machines that substantially improves over known MILP and quantum-based encodings, and reduces encoding size over the state-of-the-art. By augmenting the computation with a novel readout method, we derive valid join orders for each solution obtained by the (probabilistically operating) DA. Most importantly and despite an extremely large solution space, our approach scales to practically relevant dimensions of around 50 relations and improves result quality over conventionally employed approaches, adding a novel alternative to solving the long-standing JO problem.

Quantum Optimisation of General Join Trees

Manuel Schönberger, Immanuel Trummer, Wolfgang Mauerer

Recent advances in the manufacture of quantum computers attract much attention over a wide range of fields, as early-stage quantum processing units (QPU) have become accessible. While contemporary quantum machines are very limited in size and capabilities, mature QPUs are speculated to eventually excel at optimisation problems. This makes them an attractive technology for database problems, many of which are based on complex optimisation problems with large solution spaces. Yet, the use of quantum approaches on database problems remains largely unexplored. In this paper, we address the long-standing join ordering problem, one of the most extensively researched database problems. Rather than running arbitrary code, QPUs require specific mathematical problem encodings. An encoding for the join ordering problem was recently proposed, allowing first small-scale queries to be optimised on quantum hardware. However, it is based on a faithful transformation of a mixed integer linear programming (MILP) formulation for JO, and inherits all limitations of the MILP method. Most strikingly, the existing encoding only considers a solution space with left-deep join trees, which tend to yield larger costs than general, bushy join trees. We propose a novel QUBO encoding for the join ordering problem. Rather than transforming existing formulations, we construct a native encoding tailored to quantum systems, which allows us to process general bushy join trees. This makes the full potential of QPUs available for solving join order optimisation problems.

Challenges for Quantum Software Engineering: An Industrial Application Scenario Perspective

Cecilia Carbonelli, Michael Felderer, Matthias Jung, Elisabeth Lobe, Malte Lochau, Sebastian Luber, Wolfgang Mauerer, Rudolf Ramler, Ina Schaefer, Christoph Schroth

Quantum software is becoming a key enabler for applying quantum computing to industrial use cases. This poses challenges to quantum software engineering in providing efficient and effective means to develop such software. Eventually, this must be reliably achieved in time, on budget, and in quality, using sound and wellprincipled engineering approaches. Given that quantum computers are based on fundamentally different principles than classical machines, this raises the question if, how, and to what extent established techniques for systematically engineering software need to be adapted. In this paper, we analyze three paradigmatic application scenarios for quantum software engineering from an industrial perspective. The respective use cases center around (1) optimization & quantum cloud services, (2) quantum simulation, and (3) embedded quantum computing. Our aim is to provide a concise overview of the current and future applications of quantum computing in diverse industrial settings. We derive presumed challenges for quantum software engineering and thus provide research directions for this emerging field. 

Quantum Machine Learning: Foundation, New Techniques, and Opportunities for Database Research

Tobias Winker, Sven Groppe, Valter Uotila, Zhengtong Yan, Jiaheng Lu, Maja Franz, Wolfgang Mauerer

In the last few years, the field of quantum computing has experienced remarkable progress. The prototypes of quantum computers already exist and have been made available to users through cloud services (e.g., IBM Q experience, Google quantum AI, or Xanadu quantum cloud). While fault-tolerant and large-scale quantum computers are not available yet (and may not be for a long time, if ever), the potential of this new technology is undeniable. Quantum algorithms have the proven ability to either outperform classical approaches for several tasks, or are impossible to be efficiently simulated by classical means under reasonable complexity-theoretic assumptions. Even imperfect current-day technology is speculated to exhibit computational advantages over classical systems. Recent research is using quantum computers to solve machine learning tasks. Meanwhile, the database community has already successfully applied various machine learning algorithms for data management tasks, so combining the fields seems to be a promising endeavour. However, quantum machine learning is a new research field for most database researchers. In this tutorial, we provide a fundamental introduction to quantum computing and quantum machine learning and show the potential benefits and applications for database research. In addition, we demonstrate how to apply quantum machine learning to the join order optimization problem in databases.

Co-Design of Quantum Hardware and Algorithms in Nuclear and High Energy Physics

Maja Franz, Pía Zurita, Markus Diefenthaler, Wolfgang Mauerer

Quantum computing (QC) has emerged as a promising technology, and is believed to have the potential to advance nuclear and high energy physics (NHEP) by harnessing quantum mechanical phenomena to accelerate computations. In this paper, we give a brief overview of the current state of quantum computing by highlighting challenges it poses and opportunities it offers to the NHEP community. Noisy intermediate-scale quantum (NISQ) computers, while limited by imperfections and small scale, may hold promise for near-term quantum advantages when coupled with co-designed quantum algorithms and special-purpose quantum processing units (QPUs). We explore various applications in NHEP, including quantum simulation, event classification, and realtime experiment control, emphasising the potential of variational quantum circuits and related techniques. To identify current interests of the community, we perform an analysis of recent literature in NHEP related to QC.

Challenges and Opportunities in Quantum Software Architecture

Tao Yue, Wolfgang Mauerer, Shaukat Ali, Davide Taibi

Quantum computing is a relatively new paradigm that has raised considerable interest in physics and computer science in general but has so far received little attention in software engineering and architecture. Hybrid applications that consist of both quantum and classical components require the development of appropriate quantum software architectures. However, given that quantum software engineering (QSE) in general is a new research area, quantum software architecture–a subresearch area in QSE is also understudied. The goal of this chapter is to provide a list of research challenges and opportunities for such architectures. In addition, to make the content understandable to a broader computer science audience, we provide a brief overview of quantum computing and explain the essential technical foundations. 

Ready to Leap (by Co-Design)? Join Order Optimisation on Quantum Hardware

Manuel Schönberger, Stefanie Scherzinger, Wolfgang Mauerer

The prospect of achieving computational speedups by exploiting quantum phenomena makes the use of quantum processing units (QPUs) attractive for many algorithmic database problems. Query optimisation, which concerns problems that typically need to explore large search spaces, seems like an ideal match for quantum algorithms. We present the first quantum implementation of join ordering, one of the most investigated and fundamental query optimisation problems, based on a reformulation to quadratic binary unconstrained optimisation problems. We empirically characterise our method on two state-of-the-art approaches (gate-based quantum computing and quantum annealing), and identify speed-ups compared to the best know classical join ordering approaches for input sizes conforming to current quantum annealers. Yet, we also confirm that limits of early-stage technology are quickly reached.

State Space Discretization for Reinforcement Learning

H. Yousruf

Return-optimized Vector Quantization of Reinforcement Learning 

D. Grenz

This work deals with the generation of an optimal policy in the discrete domain, which should also allow an application in the continuous domain. For this purpose, a method called Return-optimized Vector Quantization for Reinforcement Learning is generated. The individual components consist of a neural network, dynamic programming and an evolutionary algorithm. The two well-known examples mountain car and cartpole were successfully solved in an offline reinforcement learning environment. Furthermore, it was investigated whether the policy can be improved by further online reinforcement learning.

Quantum Policy Iteration via Amplitude Estimation and Grover Search -- Towards Quantum Advantage for Reinforcement Learning

Simon Wiedemann, Daniel Hein, Steffen Udluft, Christian Mendl

We present a full implementation and simulation of a novel quantum reinforcement learning (RL) method and mathematically prove a quantum advantage. Our approach shows in detail how to combine amplitude estimation and Grover search into a policy evaluation and improvement scheme. We first develop quantum policy evaluation (QPE) which is quadratically more efficient compared to an analogous classical Monte Carlo estimation and is based on a quantum mechanical realization of a finite Markov decision process (MDP). Building on QPE, we derive a quantum policy iteration that repeatedly improves an initial policy using Grover search until the optimum is reached. Finally, we present an implementation of our algorithm for a two-armed bandit MDP which we then simulate. The results confirm that QPE provides a quantum advantage in RL problems.

Variational Quantum Circuits for Policy Approximation

Nico Meyer

Driven by the availability of the first actual quantum devices, the interest in developing quantum algorithms for a wide variety of tasks has grown in the research community. Yet, those noisy intermediate scale quantum (NISQ) computing devices are still quite small and error-prone. To overcome this, there is the idea of using variational quantum circuits as a platform for quantum machine learning.

In the context of reinforcement learning, the learning of distributions from data plays a crucial role, both when generating surrogate models for the system dynamics from observational data, and when training the reinforcement-learning policy to approximately solve the underlying stochastic optimization problem. The latter case is closely tied to the trade-off between exploration and exploitation during the training phase of the reinforcement-learning procedure. To properly balance exploration and exploitation, it is particularly relevant to employ methods capable of learning conditional, multimodal distributions.

Applicability of Quantum Computing on Database Query Optimization

Manuel Schönberger

We evaluate the applicability of quantum computing on two fundamental query optimization problems, join order optimization and multi query optimization (MQO). We analyze the problem dimensions that can be solved on current gate-based quantum systems and quantum annealers, the two currently commercially available architectures. First, we evaluate the use of gate-based systems on MQO, previously solved with quantum annealing. We show that, contrary to classical computing, a different architecture requires involved adaptations. We moreover propose a multi-step reformulation for join ordering problems to make them solvable on current quantum systems. Finally, we systematically evaluate our contributions for gate-based quantum systems and quantum annealers. Doing so, we identify the scope of current limitations, as well as the future potential of quantum computing technologies for database systems.

1-2-3 Reproducibility for Quantum Software Experiments

Wolfgang Mauerer, Stefanie Scherzinger

Various fields of science face a reproducibility crisis. For quantum software engineering as an emerging field, it is therefore imminent to focus on proper reproducibility engineering from the start. Yet the provision of reproduction packages is almost universally lacking. Actionable advice on how to build such packages is rare, particularly unfortunate in a field with many contributions from researchers with backgrounds outside computer science. In this article, we argue how to rectify this deficiency by proposing a 1-2-3~approach to reproducibility engineering for quantum software experiments: Using a meta-generation mechanism, we generate DOI-safe, long-term functioning and dependency-free reproduction packages. They are designed to satisfy the requirements of professional and learned societies solely on the basis of project-specific research artefacts (source code, measurement and configuration data), and require little temporal investment by researchers. Our scheme ascertains long-term traceability even when the quantum processor itself is no longer accessible. By drastically lowering the technical bar, we foster the proliferation of reproduction packages in quantum software experiments and ease the inclusion of non-CS researchers entering the field.

Peel | Pile? Cross-Framework Portability of Quantum Software

Manuel Schönberger, Maja Franz, Stefanie. Scherzinger, Wolfgang Mauerer

In recent years, various vendors have made quantum software frameworks available. Yet with vendor-specific frameworks, code portability seems at risk, especially in a field where hardware and software libraries have not yet reached a consolidated state, and even foundational aspects of the technologies are still in flux. Accordingly, the development of vendor-independent quantum programming languages and frameworks is often suggested. This follows the established architectural pattern of introducing additional levels of abstraction into software stacks, thereby piling on layers of abstraction. Yet software architecture also provides seemingly less abstract alternatives, namely to focus on hardware-specific formulations of problems that peel off unnecessary layers. In this article, we quantitatively and experimentally explore these strategic alternatives, and compare popular quantum frameworks from the software implementation perspective. We find that for several specific, yet generalisable problems, the mathematical formulation of the problem to be solved is not just sufficiently abstract and serves as precise description, but is likewise concrete enough to allow for deriving framework-specific implementations with little effort. Additionally, we argue, based on analysing dozens of existing quantum codes, that porting between frameworks is actually low-effort, since the quantum- and framework-specific portions are very manageable in terms of size, commonly in the order of mere hundreds of lines of code. Given the current state-of-the-art in quantum programming practice, this leads us to argue in favour of peeling off unnecessary abstraction levels.

Uncovering Instabilities in Variational-Quantum Deep Q-Networks

Maja Franz, Lucas Wolf, Maniraman Periyasamy, Christian Ufrecht, Daniel D. Scherer, Axel Plinge, Christopher Mutschler, Wolfgang Mauerer

Deep Reinforcement Learning (RL) has considerably advanced over the past decade. At the same time, state-of-the-art RL algorithms require a large computational budget in terms of training time to converge. Recent work has started to approach this problem through the lens of quantum computing, which promises theoretical speed-ups for several traditionally hard tasks. In this work, we examine a class of hybrid quantum-classical RL algorithms that we collectively refer to as variational quantum deep Q-networks (VQ-DQN). We show that VQ-DQN approaches are subject to instabilities that cause the learned policy to diverge, study the extent to which this afflicts reproducibility of established results based on classical simulation, and perform systematic experiments to identify potential explanations for the observed instabilities. Additionally, and in contrast to most existing work on quantum reinforcement learning, we execute RL algorithms on an actual quantum processing unit (an IBM Quantum Device) and investigate differences in behavior between simulated and physical quantum systems that suffer from implementation deficiencies. Our experiments show that, contrary to opposite claims in the literature, it cannot be conclusively decided if known quantum approaches, even if simulated without physical imperfections, can provide an advantage as compared to classical approaches. Finally, we provide a robust, universal and well-tested implementation of VQ-DQN as a reproducible testbed for future experiments.

Modelling and Solving Reinforcement Learning Problems on Quantum Computers

Simon Wiedemann

Master Thesis at the Technische Universität München 15.11.2021

In the thesis, a new approach to use Quantum Computing for Reinforcement Learning is developed. The procedure is analogous to policy iteration that is using policy evaluation and policy improvement. These are realized by the quantum subroutines of quantum phase estimation and Grover search.

To this end, we first describe a quantum MDP model and based on it develop a quantum policy iteration algorithm. Allowing quantum agent-environment interaction via the exchange of states, actions and rewards in superposition to reduce the (q)sample complexity of learning appears to be a promising way to obtain quantum advantage in RL.

An implementation for two rounds of two-armed bandit with discount.

Reinforcement Learning with Parameterised Quantum Circuits

Maja Franz

Bachelor Thesis at the OTH Regensburg 31.08.2021

In the last decade, great progress has been made in the research fields of reinforcement learning (RLs) and quantum computing. Parameterised quantum circuits (PQCs) have established themselves as a model for quantum-based machine learning. These have the potential to demonstrate the superiority of quantum computers over classical computers in the near future.

In this thesis, methods and concepts of quantum-based RL are presented and technologies developed from them are experimentally investigated. The results show that quantum-based approaches, similar to classical approaches, can optimally solve simple RL applications. It is assumed that these procedures can also be carried out on real quantum systems with low error influx if the quantum circuits used have a comparably small size.