Aug 1
From RNNs to Transformers
As previously mentioned, input data for a neural network must first be transformed into word embeddings. To overcome the limitations of sequential co...
4 min read
Writing
Notes on systems, LLMs, and the industry building them.
I'm Wenhao, a software engineer. I write up system design problems I've worked through, what I'm learning about LLMs, notes from shipping side projects like ResumeMatch, and the occasional take on layoffs and where the industry is heading.
22 postsSince 2025
Jul 31
From Word Embeddings to RNNs
Given a couple of words, can we generate the next word? Or, given a sentence, can we determine whether each word expresses a positive or negative sen...
4 min read
Jul 30
From Matrices to CNNs
In the previous sections, we saw how matrix operations can be used to calculate neural outputs using linear transformations followed by nonlinear act...
3 min read
Jul 27
Tuning Neural Networks
The essence of neural networks lies in progressively applying activation functions to linear transformations. Input → Hidden → Output (x) → (a...
3 min read
Jul 26
Calculating Neural Network Parameters
Let's start with a simple linear function: y = wx + b Q: What's a good w and b? A: A good set of parameters produces predictions that closely match...
3 min read
Jul 25
From Functions to Neural Networks
What if I told you that everything we know about the world - every rule, every pattern, every piece of logic - can be described by functions? This ch...
4 min read