Documents

Quantum Neural Network

- Optical Neural Networks operating at the Quantum Limit -

- Optical Neural Networks operating at the Quantum Limit -

- Preface
- I. Introduction to Quantum Neural Networks
- II. Physics of Optical Parametric Oscillator Network
- III. Theory of Optical Delay Line Coupling Quantum Neural Network
- IV. Theory of Quantum Measurements
- V. Theory of Measurement Feedback Based Quantum Neural Networks
- VI. Coherent Ising Machines

Technical Papers

Quantum Neural Network

- Optical Neural Networks operating at the Quantum Limit -

- Optical Neural Networks operating at the Quantum Limit -

We describe the basic concepts, operational principles and expected performance of a novel computing machine in this white paper. The first version of the machine was originally proposed in 2011 and named a coherent Ising machine. The network of injection-locked lasers represents Ising spin network, in which a coherent mean-field produced at above the oscillation threshold searches the ground state of an Ising Hamiltonian as a single lasing mode with a minimum overall loss. Later in 2013, the concept was extended to the network of degenerate optical parametric oscillators (DOPO), in which the intrinsic quantum uncertainty of the squeezed vacuum state, formed in each DOPO at below the oscillation threshold, searches the ground state of NP-hard Ising problems. The DOPO network based computing machine is called quantum neural network (QNN), which is distinct from classical neural network due to the following three properties:

- 1. Each DOPO is in a superposition state of different excitation amplitudes so that a quantum parallel search can be implemented.
- 2. A network of DOPOs makes a decision to reach a final computational result by correlated and collective symmetry breaking at a critical point of DOPO phase transition, i.e. oscillation threshold.
- 3.A network of DOPOs amplifies the above quantum solution obtained at a critical point to a classical signal via bosonic final state stimulation.

If a network of DOPOs is arranged into a recurrent neural network, such a device can find efficiently the satisfying solution of NP-complete k-SAT problems. Quantum noise is utilized as a useful computational resource and various quantum effects are employed to escape from wrong solutions (local minima) and chaotic traps in this case.

Various alternative approaches to modern digital computers have been intensively explored in recent years. There are four streams in the current efforts, which are

- 1. Return to analog computers

This approach promises a faster computer but suffers from intrinsic lack of precision due to noise injection and gate error. - 2 Learn from nature

For example, a phase transition phenomenon at a critical temperature is considered as a super-efficient computational process, because such a system spontaneously realizes a single lowest-energy state out of huge number of possible states. - 3 Mimic human brains

This approach covers a broad spectrum from commercial technology of deep machine learning chips to fundamental study on emergent mechanism for cognition and consciousness. - 4 Utilize quantum effects

Quantum parallel search and quantum suppression of chaos are identified as two promising principles in quantum computers to exceed the limit of classical information processing, but they also suffer from intrinsic lack of precision just like analog computers.

QNN has all of the above four aspects so that it is hard to assign it to a specific category. As for the relation to category 1, the DOPOs operate as ideal analog memories which are robust against photon loss and noise injection. This fact allows QNN to solve not only a combinatorial optimization problem but also a continuous-variable optimization problem efficiently. As for the relation to category 2, the DOPOs make a decision to oscillate as either $0$-phase coherent state or $\pi$-phase coherent state at a DOPO threshold (phase transition point). After this collective symmetry breaking (or supercritical pitchfork bifurcation) happens, bosonic final state stimulation is kicked in. The exponentially increasing success rate to find a ground state in QNN stems from the onset of this stimulated emission of photons at above threshold. As for the relation to category 3, the quantum dynamics in QNN resemble to the classical dynamics governed by the majority vote among many copies of classical neural networks (CNN) in human brain. A partial wave of the single quantum wavefunction in QNN represents simultaneously many trajectories in CNN and therefore the final decision made by the single quantum wavefunction in QNN can be also based on the majority vote of many partial waves. As for the relation to category 4, quantum parallel search realized by squeezed vacuum states near the threshold provides an important step to solve an Ising problem and quantum suppression of classical chaos is a key to solve efficiently a k-SAT problem.

There are two types of quantum neural networks (QNN), i.e. optical delay line coupling based machine (DL-QNN) and measurement feedback coupling based machine (MF-QNN). These two machines use distinct quantum processes during the crucial preparation before the final decision making: quantum noise correlation (or entanglement) for DL-QNN and quantum wavepacket reduction for MF-QNN. The MF-QNN can implement not only symmetric neural network but also asymmetric recurrent neural network, which possesses a unique function of error detection and correction.

This white paper is organized as follows. Chapter I introduces the basic concepts, operational principles and expected performance of the DL-QNN and MF-QNN. The physics and nonlinear dynamics of degenerate optical parametric oscillators (DOPO) are presented in Chapter II, in which such topics as DOPO phase transition, quantum tunneling and effective temperatures are introduced. The quantum theory for the DL-QNN is presented in Chapter III, where quantum noise correlation and entanglement are identified as the important computational resource of the machine. Chapter IV briefly reviews the theory of quantum measurements, in particular the approximate measurements and continuous nonlinear measurements are formulated. The quantum theory of the MF-QNN is presented in Chapter V, where wavepacket reduction and contextuality are identified as the important computational mechanisms of this machine. Chapter VI describes the principles of the coherent Ising machines (CIM) based on numerical simulation results. The performance of CIM for NP-hard Ising problems is compared to the four types of classical neural networks: Hopfield network (discrete variables, deterministic evolution), simulated annealing (discrete variables, stochastic evolution), Hopfield-Tank neural network (continuous variables, deterministic evolution) and Langevin dynamics (continuous variables, stochastic evolution). Chapter VII describes the coherent SAT machines (CSM). The performance of CSM for NP-complete k-SAT problems is compared with the classical approach.

The readers interested in obtaining the minimum knowledge about the basic concepts and principles of the QNN can start by reading Chapter I. If he/she is interested in the cloud service starting in November, 2017, Chapter VI provides a good summary for this novel computing machine. Finally, those who wish to understand the basic physics and quantum theory of the two types of QNN at a deeper level may read Chapter II-V as well as the above two chapters.

We will release several additional chapters for presenting coherent SAT machines and actual algorithms for real world problems: drug discovery, wireless communications, compressed sensing, deep machine learning and fintech in November, 2018.

In this Chapter, we will introduce the basic concepts and operational principles of a novel computing machine, optical neural networks at quantum limit, and describe their unique characteristics. We start with the discussion how to construct such physical devices as the quantum version of classical neurons and synapses.

- 1.1 Quantum neurons
- 1.1.1 Degenerate optical parametric amplifiers/oscillators
- 1.1.2 Linear superposition states in DOPA/DOPO
- 1.1.3 Amplitude and phase error correction by phase sensitive amplification
- 1.2 Quantum synapses
- 1.2.1 Optical delay line coupling scheme
- 1.2.2 Measurement feedback coupling scheme
- 1.3 Mapping of an Ising model to DOPO network: coherent Ising machines
- 1.3.1 Pitchfork bifurcation
- 1.3.2 Conditional mappling of the Ising Hamiltonian
- 1.3.3 Effect of the amplitude heterogeneity
- 1.4 Optical neural network at quantum limit and classical limit
- 1.5 Gottesman-Knill theorem
- 1.6 Summary

This chapter describes the physics of the optical parametric oscillator (OPO) and OPO network, a key component in the quantum neural network (QNN). The OPO is based on a second-order ($\chi^{(2)}$) optical nonlinearity in non-centrosymmetric crystals, which allows photons of a pump frequency to be down-converted into pairs of photons at the half-harmonic. Such a phenomenon allows for phase-sensitive amplification and oscillation: below a threshold pump power, the system lives in a squeezed vacuum state, while above threshold, it can oscillate in one of two (coherent) phase states. This bifurcation provides the computational mechanism for the Ising machine: a network of OPOs can be driven from below to above threshold, while the optical coupling encodes the Ising problem and allows the system to search for the Ising ground state.

- 2.1 Parametric amplification
- 2.2 General quantum limit of linear amplifiers
- 2.3 Supercritical pitchfork bifurcation
- 2.4 OPO network
- 2.5 Example: Ising spins on cubic graph
- 2.6 Example: 1D Ising spin chains
- 2.6.1 Growth stage
- 2.6.2 Saturation stage
- 2.7 Correlation length and defect density
- 2.8 Multimode tunneling
- 2.8.1 Equations for signal fields and Hermite function expansion
- 2.8.2 Dynamics of the multimode tunneling
- 2.8.3 Simulated performance
- 2.9 XY machine based on nondegenerate OPO
- 2.9.1 Mechanism
- 2.9.2 1D chain
- 2.9.3 2D lattice

In this Chapter, we present the quantum theory of degenerate optical parametric oscillator (DOPO) networks based on optical delay line coupling, which have been recently demonstrated at Stanford, NII and NTT. Two types of c-number stochastic differential equations (CSDE) are derived using the (exact) positive $P(\alpha,\beta)$ representation and the (approximate) truncated Wigner representation of the density operator. We introduce the EPR-like operator and the quantum discord to evaluate the quantum noise correlation and entanglement formed in the DOPOs during the computation process.

- 3.1 Standard theoretical approach and computational difficulty
- 3.2 Positive $P(\alpha,\beta)$ representation
- 3.3 Truncated Wigner representation $W(\alpha)$
- 3.4 Quantum entanglement and inseparability
- 3.5 Quantum discord
- 3.6 Summary

The standard quantum theory governed by Schrodinger (or Heisenberg) equation describes the evolution of a system with two unique features. The evolution described by the standard quantum theory is a deterministic and reversible process. If the initial state of a system is known and a Hamiltonian is given, the final state is uniquely determined by the unitary operator $\hat{U}$. We can always undo this time evolution by imposing the inverse unitary operator $\hat{U}^{-1}$ and recover the initial state. On the other hand, a process of quantum measurement is non-deterministic and irreversible. Even though we have a perfect information about a system, a measurement result is generally random and unpredictable. This is not due to the detector noise but rather due to the intrinsic uncertainty of the system. Once a measurement is completed and a result is read out, we cannot go back to the initial state. We even do not know what the initial state was. In this Chapter, we present the theory of a quantum measurement process. We start with the von Neumann’s projection postulate and extend it to an approximate measurement with a finite error. We then explain the most relevant concept for our purpose: the difference between linear and nonlinear continuous measurements. A quantum Zeno effect in continuous measurements and contextuality in a measurement-feedback system are mentioned as representative examples of nonlinear continuous measurements.

- 4.1 Exact measurements
- 4.2 Approximate measurements
- 4.2.1 Measurement error and back action noise
- 4.2.2 Measurement probability and post-measurement state
- 4.2.3. Optical homodyne detection
- 4.3 Continuous measurements
- 4.4 Non-referred measurements
- 4.5 Linear and nonlinear continuous measurements
- 4.5.1 Quantum Zeno effect
- 4.5.2 Measurement-feedback QNN
- 4.6 Contextuality in quantum measurements
- 4.7 Summary

In this Chapter, we present the theory of quantum neural networks connected by a measurement feedback circuit, which have been recently demonstrated independently at NTT and Stanford University. As an earlier study, the non-degenerate optical parametric oscillators with idler-measurement and signal-feedback control was theoretically studied in the context of generating various quantum states of light, such as coherent states, squeezed states and photon number states. Those three specific states are generated by optical heterodyne detection, homodyne detection and photon number detection, respectively. The semiconductor lasers with junction voltage-measurement and injection current-feedback control was experimentally studied in the context of generating number-phase squeezed states of light. An indirect quantum measurement using a probe, treated in the previous Chapter IV, plays a central role in such measurement-feedback oscillator systems. The concept can be extended here to implement the NP-hard Ising problems and NP-complete k-SAT problems in quantum neural networks. Two complementary theories will be presented: one is directly based on the density operator master equation in in-phase amplitude eigenstate $|x\rangle$ representation and the homodyne measurement projectors, while the other utilizes the c-number stochastic differential equations (CSDE) derived by the positive $P(\alpha,\beta)$ representation of the density operator and the replicator dynamics.

- 5.1 A Quantum model based on density operator master equations and homodyne measurement projectors
- 5.1.1 Theoretical formulation
- 5.1.2 Numerical simulation results
- 5.2 A Quantum model based on c-number stochastic differential equations and replicator dynamics
- 5.2.1 Master equations
- 5.2.2 Stochastic differential equations
- 5.2.3 Numerical simulation results
- 5.3 Summary

In this Chapter, we will discuss the operational principles of two types of optical neural networks operating at quantum limit ($\hbar\omega\gg k_BT$), which are the optical delay line coupling QNN and measurement feedback QNN, and compare their performances with those of classical neural networks (CNN) operating at the opposite thermal limit ($k_BT\gg\hbar\omega$). In the case of optical delay line coupling scheme, the quantum noise correlation formed among DOPOs at pump rates below the threshold is an important preparation step for the decision making process at the DOPO phase transition point. On the other hand, in the case of measurement feedback scheme based on local operation and classical communication (LOCC), such quantum correlation does not exist in the system. Instead, another important preparation step can be identified for the decision making process at the DOPO threshold, which is the contextuality in quantum measurements. Finally, the correlated symmetry breaking and the bosonic final state stimulation provide a crucial mechanism to amplify a weak quantum solution to a strong classical solution.

Quantum neural networks can solve various hard problems by mapping them to either the NP-hard Ising problems or the NP-complete satisfiability (SAT) problems. According to the computational complexity theorem, the three-dimensional Ising model and the two dimensional Ising model with a dc field belong to the NP-hard class, while the k-SAT problems ($k > 3$) belong to the NP-complete class. We will evaluate the performance of the particular QNN, coherent Ising machines, for solving the Ising problems in this Chapter. The performance of the other type of QNN for solving the k-SAT problems will be described in the next Chapter. Various combinatorial optimization problems, such as maximum clique problems, graph coloring problems and many others, can be directly mapped to the Ising model and thus they are universally solved by the coherent Ising machine.

- 6.1 Correlated spontaneous symmetry breaking in optical direct coupling QNN
- 6.2 Contextuality in measurement feedback QNN
- 6.3 Quantum parallel search, correlated symmetry breaking and bosonic final state stimulation
- 6.4 MAX-CUT problems
- 6.5 Coherent Ising machines vs. classical neural networks
- 6.5.1 Coherent Ising machines (CIM)
- 6.5.2 Classical neural networks
- 6.5.3 Implementation of classical neural network algorithms
- 6.6 Simulation results
- 6.7 Discussion
- 6.7.1 Validity for the hardware selection
- 6.7.2 Optimization for PEZY-SC implementation for HTNN and CLD
- 6.8 Conclusion

T. Aonishi, K. Mimura, S. Utsunomiya, M. Okada and Y. Yamamoto, “Statistical mechanics of coherent Ising machine - The case of ferromagnetic and finite-Loading Hopfield Models,” J. Phys. Soc. Jpn. 86, 104002 (2017.09).

DOI:10.7566/JPSJ.86.104002 http://journals.jps.jp/doi/10.7566/JPSJ.86.104002

T. Leleu, Y. Yamamoto, S. Utsunomiya, and K. Aihara, “Combinatorial optimization using dynamical phase transitions in driven-dissipative systems,” Phys. Rev. E 95 022118 (2017.02).

DOI: https://doi.org/10.1103/PhysRevE.95.022118 https://journals.aps.org/pre/abstract/10.1103/PhysRevE.95.022118

R. Hamerly, A. Marandi, M. Jankowski, M. Fejer, Y. Yamamoto, and H. Mabuchi, “Reduced models and design principles for half-harmonic generation in synchronously pumped optical parametric oscillators,” Phys. Rev. A 94, 063809 (2016.12).

DOI: 10.1103/PhysRevA.94.063809 https://journals.aps.org/pra/abstract/10.1103/PhysRevA.94.063809

T. Inagaki, Y. Haribara, K. Igarashi, T. Sonobe, S. Tamate, T. Honjo, A. Marandi, P. L. McMahon, T. Umeki, K. Enbutsu, O. Tadanaga, H. Takenouchi, K. Aihara, K. Kawarabayashi, K. Inoue, S. Utsunomiya, and H. Takesue, “A coherent Ising machine for 2000-node optimization problems,” Science 354, 603-606 (2016.10).

DOI: 10.1126/science.aah4243 http://science.sciencemag.org/content/early/2016/10/19/science.aah4243

P. L. McMahon, A. Marandi, Y. Haribara, R. Hamerly, C. Langrock, S. Tamate, T. Inagaki, H. Takesue, S. Utsunomiya, K. Aihara, R. L. Byer, M. M. Fejer, H. Mabuchi, and Y. Yamamoto, “A fully-programmable 100-spin coherent Ising machine with all-to-all connections,” Science 354, 614-417 (2016.10).

DOI: 10.1126/science.aah5178 http://science.sciencemag.org/content/354/6312/614

R. Hamerly, K. Inaba, T. Inagaki, H. Takesue, Y. Yamamoto, and H. Mabuchi, “Topological defect formation in 1D and 2D spin chains realized by network of optical parametric oscillators,” Int. J. Mod. Phys. B 30, 1630014 (2016.10).

DOI: http://dx.doi.org/10.1142/S0217979216300140 http://www.worldscientific.com/doi/abs/10.1142/S0217979216300140

H. Sakaguchi, K. Ogata, T. Isomura, S. Utsunomiya, Y. Yamamoto, and K. Aihara, “Boltzmann sampling by degenerate optical parametric oscillator network for structure-based virtual screening,” Entropy 18, 365 (2016.10).

DOI: 10.3390/e18100365 http://www.mdpi.com/1099-4300/18/10/365

K. Takata, A. Marandi, R. Hamerly, Y. Haribara, D. Maruo, S. Tamate, H. Sakaguchi, S. Utsunomiya, and Y. Yamamoto, “A 16-bit coherent Ising machine for one-dimensional ring and cubic graph problems,” Scientific Reports 6, 34089 (2016.09).

DOI: 10.3390/e18040151 https://www.nature.com/articles/srep34089

H. Takesue and T. Inagaki, "10 GHz clock time-multiplexed degenerate optical parametric oscillators for a photonic Ising spin network," Opt. Lett. 41, 4273-4276 (2016.09).

DOI: https://doi.org/10.1364/OL.41.004273 https://www.osapublishing.org/ol/abstract.cfm?uri=ol-41-18-4273

D. Maruo, S. Utsunomiya, and Y. Yamamoto, “Truncated Wigner theory of coherent Ising machines based on degenerate optical parametric oscillator network,” Phys. Scr. 91, 083010 (2016.07).

DOI: https://doi.org/10.1088/0031-8949/91/8/083010 http://iopscience.iop.org/article/10.1088/0031-8949/91/8/083010

T. Inagaki, K. Inaba, R. Hamerly, K. Inoue, Y. Yamamoto, and H. Takesue, “Large-scale Ising spin network based on degenerate optical parametric oscillators,” Nature Photonics 10, 415-419 (2016.06).

DOI: 10.1038/NPHOTON.2016.68 http://www.nature.com/nphoton/journal/v10/n6/abs/nphoton.2016.68.html

Y. Haribara, S. Utsunomiya, and Y. Yamamoto, “Computational principle and performance evaluation of coherent Ising machine based on degenerate optical parametric oscillator network,” Entropy 18, 151 (2016.04).

DOI: 10.3390/e18040151 http://www.mdpi.com/1099-4300/18/4/151

K. Takata, A. Marandi, and Y. Yamamoto, “Quantum correlation in degenerate optical parametric oscillators with mutual injections,” Phys. Rev. A 92, 043821 (2015.10).

DOI: 10.1103/PhysRevA.92.043821 https://journals.aps.org/pra/abstract/10.1103/PhysRevA.92.043821

Copyright QNNcloud All Rights Reserved.