Forget Me Nots

Notes of various kinds, for various things. Some of these might be helpful for some students in some classes. Some might help non-students. Most help me in some way.

• Forget Me Nots

• The paper The Markov Chain Monte Carlo Revolution by Persi Diaconis has a fun application of MCMC. I should probably try to implement such an idea as an example of the coolness/power of MCMC.

• The free and online book Block Relaxation Methods in Statistics by Jan de Leeuw looks a reasonable generalization of some popular maximization methods.

• Statistics and Data Science is a Jupyter book that '... is the start of a book for a graduate-level course at NYU Physics titled Statistics and Data Science.'

• The paper Estimation of Non-Normalized Statistical Models by Score Matching contains a '... basic trick of partial integration needed the proof is simple'

• The book Entropy and Information Theory by Robert M. Gray needs to be higher on my reading list.

• The book Probability, Random Processes, and Ergodic Properties by Robert M. Gray needs to be higher on my reading list.

• Read: Approximating data

• Properties of the Affine Invariant Ensemble Sampler’s ‘stretch move’ in high dimensions details some unusual and undesirable properties of the Affince Invariant Ensemble Sampler in high dimensions

• A fun class project for MATH 450 Computational Statistics: port Jian Chen and Jeffrey S. Rosenthal's 2010 paper Decrypting Classical Cipher Text Using Markov Chain Monte Carlo and accompanying C code to Python.

• While at Flatiron Institute in Summer 2023, I attended a hands on tutorial based around Andrej Karpathy's YouTube video Let's build GPT: from scratch, in code, spelled out., which itself is part of Andrej's Neural Networks: Zero to Hero course. Here's a collection of resources that accompanied this tutorial: colab notebook, The Annotated Transformer: Attention is All You Need, FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awareness, Deep Residual Learning for Image Recognition, Layer Normalization, Layer norm in PyTorch.

• Abramowitz and Stegun's Handbook of Mathematical Functions, freely available from Colin B. Macdonald

• John D. Cook's blog post Don't invert that matrix

• Another blog post about not inverting matrices: Why Shouldn't I Invert That Matrix?

• C style guide from Malcolm Inglis.

• Wolfram's Functions Site is a convenient way to evaluate special mathematical functions to arbitrary precision. Could be useful for testing some code.

• John Burkardt provides C code for a bunch of random number generators of common statistical distributions.

• John Burkardt reimplemented TOMS 708 in Fortran77 and Fortran 90 to calculate the incomplete beta function. One day, I'd like to port this to C Javascript. Though, I think I should go to the source, because I'm pretty sure the C translation is flawed.

• C. Sherlock, S. Urbas and M. Ludkin's Apogee to Apogee Path Sampler implementation on GitHub.

• Somewhere in the world of audio, DACs come into play. Somewhere in the world of DACs, near the quantization of noise, the uniform distribution can elucidate the rough guideline of SNR around 6N dB: ASR has a post about it, which includes this PDF, and this PDF, which seems clearer albeit with less context.

• 21st Century C by Ben Klemens

• Probability: Theory and Examples by Rick Durrett

• Austin Rochford provides a good explanation of uniform resevoir sampling on his blog post Reservoir Sampling for Streaming Data. I tend to think of this as online uniform sampling, instead of the phrase reservoir sampling.

• David Nochols hosts a nice webpage of various color palettes designed to minimize the reduced color space of colorblindness: Coloring for Colorblindness.

• Probably some useful tidbits burried in the Netlib/random source code.

• Quansight Labs has a nice blog with topics that I'm often interested in: Quansight Labs Blog.