As I mentioned in my previous post I was doing some review of Gaussian Processes. As I was doing so, I came across a recent series of videos [1] by Prof. Sergei Kalinin. I guess they are putting on a free summer school for applications of ML in microscopy, however, its more general towards experimentation. These videos (see below) are really good in my view if you want to get a leading experts perspective on the challenges of applying AI/ML to a domain science area. More over, you get to see how ML is used outside well crafted datasets that are typically used in the data science and ML community.
The part I really liked was the emphasis on knowing how to formulate the domain problem within ML frameworks, and not the other way around. The videos on bayesian inference are also great (haven't finished them though) and I become aware of a new python package called GPax[2] which allows you to encode prior knowledge about the physics of your system. The API looks nice two! For example standard Bayesian inference and GP with optimized parameters is done with 4 lines:
import gpax
#... some dataset X,y
randkeys = gpax.utils.get_keys()
gp = gpax.ExactGP(1.0,kernel="RBF")
gp.fit(randkeys[0],X,y)
#... some points Xsample
mean,samples = gp.predict(randkeys[1],Xsample)
Which fits the parameters for the kernel and model noise and then we can do inference. Here is the notebook for the simple GP. But again, this isn't actually where this package shines, its on structured GP[2], where you can construct the GP prior such that it accounts for known functional behavior that has a deterministic form. Then its possible using probabilistic programming interface (NumPyro in GPax case) to place priors over the parameters for the functional form. Graphical what you get is something like:
 |
| Example of structured GP from [2]. Right figure shows that the samples capture the discontinuous transition that has been encoded into the GP prior |
If I get the time I'd love to explore doing this using Turing.jl in combination with one of the Julia GP packages. Glad I came across this research and activity along with the videos.[1-2]
References
[1] Machine Learning for Automated Experiment - Summer 2023. M*N: Microscopy, Machine Learning, Materials. YouTube, 2023. https://www.youtube.com/playlist?list=PLS6ZvEWHZ3OMNp6heWgnitn7jwT7p7b8H.
[2] Ziatdinov, Maxim, Ayana Ghosh, and Sergei V Kalinin. “Physics Makes the Difference: Bayesian Optimization and Active Learning via Augmented Gaussian Process.” arXiv Preprint arXiv:2108.10280, 2021.
