Infinite order automatic differentiation for Monte Carlo with unnormalized probability distribution
Introduction
Due to the nature of Metropolis-Hasting algorithm, we can simulate the distribution by Monte Carlo as long as we have the knowledge of the ratio between probabilities (densities) for two different configurations. Namely, we can sample data from unnormalized probability distributions with unknown normalized factors of the distribution (which usually denoted as partition function in statistics physics). There are various scenarios for such MC with unormalized probability, including MCMC to estimate posteriors in Bayesian inference context and classical Monte Carlo as well as Quantum Monte Carlo methods to evaluate observable quantities from Hamiltonian models in statistical physics context.
But the method to compute the derivatives of such MC expectation from unnormalized probability is lack in the literature, leaving a huge gap in more flexible combination between differentiable programming and probabilistic programming.
Solution
Check our work here.
The associate codebase implemented with tensorflow can be found here.
Conclusion
We presented the general theory and framework of ADMC. We also showed how Monte Carlo expectations, KL divergence, and objectives from various settings can be expressed in an infinitely AD aware fashion. We further applied the ADMC approach on various Monte Carlo applications including classical Monte Carlo and end-to-end VMC. Especially, the ADVMC enables us to efficiently study interacting quantum models in higher dimensions.
We believe that the ADMC approach can inspire more accurate and efficient Monte Carlo designs with machine learning toolbox in the future. At the intersection of differentiable programming and probabilistic programming, ADMC framework provides a promising route to advance Monte Carlo applications in the fields of statistics, machine learning, and physics.