Welcome to the WWW site for the Spring Semester 2025 CS 696 Applied Large Language Models course at San Diego State University.

• Lecture notes
• Assignments
• Reading Assignments
• Syllabus

• Speech and Language Processing
• Building a Large Language Model (from Scratch)

• Hugging Face - LLM Model Repository
• Things we learned about LLMs in 2024
• OReilly Technology Trends for 2025
• Beyond Prompt-and-Pray
• The impact of competition and DeepSeek on Nvidia
• Building LLMs is probably not going be a brilliant business
• How has DeepSeek improved the Transformer architecture?
• SMOL-GPT Train small LLM from Scratch
• The Illustrated Transformer

• Qwen2_5_1M Technical Report
• DeepSeek-R1 Technical Report
• DeepSeek-V3 Technical Report
• Illustrated DeepSeek-R1
• Limitations of Math Reasoning in LLMs
• Think Dot by Dot
• DeepSeek's Multi-Head Latent Attention and Other KV Cache Tricks
• Are Sixteen Heads Really Better than One

• Foundations of Large Language Models (PDF Text) by Tong Xiao and Jingbo Zhu, 1/17/2025

• DeepSeek - DeepSeek LLM
• Gemini - Google LLM
• Claude - Anthropic LLM
• Perplexity - Perplexity AI LLM
• Llama - Meta LLM
• Mistral - Mistral LLM
• Tiktokenizer

• minGPT PyTorch re-implementation of GPT
• nanoGPT rewrite of minGPT to be faster
• PicoGPT minimal implementation of GPT-2 in plain NumPy
• SMOL-GPT PyTorch implementation for training your own small LLM from scratch
• smolGPT Source and runtime