Some personal notes on probability theory
A self-contained summary of some key results in probability theory. I’ll mostly summarize the results from Durrett’s Probability: Theory and Examples, Bandit...
About
I am a Master’s student in Computer Science at ETH Zürich, enrolled in Machine Intelligence major. Before that, I obtained a Bachelor’s degree in Computer Science at Vrije Universiteit Amsterdam. During my undergraduate, I was fortunate to work on two machine learning research under the supervision of Prof. Dr. Piek Vossen and Prof. Vincent François-Lavet. My current mentor at ETH Zürich is Prof. Thomas Hofmann.
I find research papers like this, this, and this one are elegant in many ways. The impact of these works is needless to say, yet their deliveries are very succinct and easy to read. Academic papers often go in the opposite direction where complexity implies novelty. Therefore, I always aim for the simplest presentation of ideas. This is, by no means, an easy endeavor.
“When I started graduate school, my adviser told me that the best work would prune the tree of knowledge, rather than grow it.” - from How Claude Shannon Invented the Future
Recent posts
What is Fisher information?
It bugs me that I often have hard time to recall what Fisher information is. So let’s write it down here.
Confidence sets for Gaussian linear models
In statistical inference, we often want to construct confidence sets for the parameter of interest; for example, you may have heard of the 95% confidence int...
The upper confidence bound algorithm
Bandits are a class of reinforcement learning (RL) problems where a learner has to choose between different actions, each of which has an unknown reward. Th...
Why I think mathematics is beautiful
A friend of mine asked me why I think mathematics is beautiful. I believe the exact answer does not matter, but it is important to have an answer.
Modern hashing made simple
This is a short personal note on “Modern Hashing Made Simple” by Bender et al. (2024). I’m going to present this paper for the Advanced Graph Algorithms and...
Jane Street Puzzle - August 2024
I decided to slack off my exam studying with Jane Street’s puzzle. I first read the problem in early August, forgot about it, and then submitted my answer o...
Counting with expectation
Problem 10 (Individual test, ARML 2013). For a positive integer \(n\), let \(C(n)\) equal the number of pairs of consecutive \(1\)’s in the binary representa...
An upper bound for the Kullback-Leibler divergence
Zürich is so hot now in the night that I couldn’t fall asleep. While lying on the bed, I arrived at an interesting upper bound for the KL divergence (relativ...
Mathematical optimization cheatsheet
In this cheatsheet, I will discuss many concepts that are essential in the analysis of optimization algorithms.
Rate-distortion theory
In this post, I will summarize Shannon’s rate-distortion theory, including some self-contained definitions and proofs to the converse and direction part of t...
Noisy-channel coding theorem
Here is my second attempt to understand the proof for the channel coding theorem. My initial exposure to this concept was through Professor Amos Lapidoth’s l...