made with excalidraw

如果你觉得这个路线图有帮助，请在Twitter和LinkedIn上关注我！也订阅这个时事通讯，我每周都会和朋友分享有趣的链接。

那么，你想学习人工智能吗？但你不知道如何或从哪里开始？

早在2020年，我就在互联网上写了20大免费数据科学、ML和AI MOOC。但我已经意识到，修很多课程不是办法。

要想逃离教程地狱并真正学习，你必须亲自动手，从头开始写算法，实现论文，并使用人工智能解决问题，做有趣的附带项目。

这篇文章试图制定一个遵循这一哲学的免费课程。我正在学习其中的一些课程，所以如果你想一起学习，请访问Twitter或Linkedin！

此外，如果你认为它遗漏了什么，请留言评论！

但首先，请注意课程设置和一些学习建议。

自上而下的方法

这门课程遵循自上而下的方法——先代码后理论。

我喜欢在必要的时候学习。所以，如果我必须弄清楚一些事情，一个要解决的问题，或者一个要制作的原型，我会广泛地获取我需要的信息，研究，理解它，然后采取行动。

例如，我的目标是成为一名从根本上理解LLM的人工智能工程师，这涉及到从头开始对转换器进行编码和在GPU上微调LLM等技能。我现在无法做到这一点，因为我的知识存在空白，我的目标是填补这些空白。

它也以NLP为重点；如果你正在寻找其他人工智能专业，如计算机视觉或强化学习，请在下面评论或在Twitter或Linkedin上DM我。我会给你一些建议。

在我向你抛出一堆链接之前，我希望有人在我开始学习之前告诉我两件重要的事情。

在公共场合学习

有很多东西需要学习，你永远不会完成学习，尤其是在人工智能方面，每周都会发布新的革命性论文和想法。

你可能犯的最大错误就是私下学习。如果你这样做，你不会为自己创造任何机会。除了能够说你完成了一些事情之外，你没有什么可以展示的。更重要的是你如何利用这些信息，如何将其转化为与公众共享的知识，以及这些信息带来了哪些新颖的想法和解决方案。

所以，你应该在公共场合学习。

这意味着要有创造的习惯。

这可能意味着：

• 撰写博客和教程
• 加入黑客马拉松并与他人合作
• 在Discord社区中问答问题
• 从事你热衷的副业
• 在推特上谈论你发现的有趣的新事物
• 说到推特，

使用Twitter

如果你关注正确的人并正确使用它，推特是当今任何人都可以使用的价值最高的社交平台。

跟随谁？请参阅 AI list by Suhail 。

如何使用Twitter？阅读Near的《如何成功推特》。

推特上的DM人。要真诚，保持简短，并有一个具体的要求。这份由Sriram Krishnan撰写的关于如何撰写冷电子邮件的指南也适用于DM。

如何发推特？阅读讲师创始人Jason的推文解剖，他从0开始成长→ 几个月内有14000名追随者。

如果你正在读这篇文章，请在推特上关注我！

DM我你在干什么！我总是愿意在很酷的项目上合作。

现在让我们开始讨论。

目录

• Mathematics
• Tools
  ∘ Python
  ∘ PyTorch
• Machine Learning
  ∘ Write from Scratch
  ∘ Compete
  ∘ Do side projects
  ∘ Deploy them
  ∘ Supplementary
• Deep Learning
  ∘ Fast.ai
  ∘ Do more competitions
  ∘ Implement papers
  ∘ Computer Vision
  ∘ NLP
• Large Language Models
  ∘ Watch Neural Networks: Zero to Hero
  ∘ Free LLM boot camp
  ∘ Build with LLMs
  ∘ Participate in hackathons
  ∘ Read papers
  ∘ Write Transformers from scratch.
  ∘ Some good blogs
  ∘ Watch Umar Jamil
  ∘ Learn how to run open-source models.
  ∘ Prompt Engineering
  ∘ Fine-tuning LLMs
  ∘ RAG
• How to stay updated
• Other curriculums/listicles you may find useful

数学

DALL·E

机器学习在很大程度上依赖于数学的三大支柱：线性代数、微积分、概率和统计学。每种算法在使算法有效运行方面都发挥着独特的作用。

• 线性代数：用于数据表示和操作的数学工具包，其中矩阵和向量构成算法解释和处理信息的语言
• 微积分：机器学习中的优化引擎，使算法能够通过理解梯度和变化率来学习和改进。
• 概率和统计学：不确定性下决策的基础，允许算法通过随机性和可变性模型预测结果并从数据中学习。

从程序员的角度来看，这是一个关于ML数学的伟大系列：基于权重和偏差的机器学习数学（代码）

如果你想要一种代码优先的线性代数方法，请由fast.ai的创建者进行计算线性代数（视频，代码）。

在课程的同时阅读Python应用机器学习的线性代数导论。

如果你想要更传统的东西，可以看看伦敦帝国理工学院的讲座——线性代数和多元微积分。

观看3Blue1Brown的《线性代数的本质》和《微积分的本质》。

观看StatQuest的统计学基础

补充的

• Book: Mathematics for Machine Learning
• Paper: The Matrix Calculus You Need For Deep Learning

工具

Python

初学者从这里开始：实用Python编程。

如果您已经熟悉Python，请进行以下高级Python掌握

它们都是Python烹饪书作者David Beazley的精彩课程。

之后，观看詹姆斯·鲍威尔的一些演讲

阅读Python设计模式。

补充的

• Book: Fluent Python, 2nd Edition (code)
• Podcasts: Real Python & Talk Python

PyTorch

观看Aladdin Persson的PyTorch教程

PyTorch网站是个不错的地方。

• PyTorch Examples
• Official PyTorch Tutorials
• FAQ page

用一些谜题测试你的知识

srush/Tensor谜题：解决谜题。改进PyTorch

补充的

• Book: Programming PyTorch for Deep Learning

机器学习

Read the 100-page ML book.

Write from Scratch

While you’re reading, write the algorithms from scratch.

Look at the repositories below

• eriklindernoren/ML-From-Scratch
• JeremyNixon/oracle
• trekhleb/homemade-machine-learning

If you want a challenge, write PyTorch from scratch by following this course.

• MiniTorch: A DIY Course on Machine Learning Engineering (videos, code)

Compete

Apply what you learn in competitions.

• Join ML competitions on platforms like bitgrit and Kaggle; find more in this article.
• Look at past winning solutions and study them

Do side projects

Read Getting machine learning to production by Vicki Boykis

She also wrote about what she learned building Viberary, a semantic search for books.

Get a dataset and build a model (i.e., use earthaccess to get NASA Earth data).

Create a UI with streamlit and share it on Twitter.

Deploy them

Get the models in production. Track your experiments. Learn how to monitor models. Experience data and model drift firsthand.

Here are some excellent resources

• Made With ML
• DataTalksClub/mlops-zoomcamp: Free MLOps course
• chiphuyen/machine-learning-systems-design
• Evidently AI — ML system design: 300 case studies
• stas00/ml-engineering: Machine Learning Engineering Online Book

Supplementary

• Machine Learning with PyTorch and Scikit-Learn (code)
• [1811.12808] Model Evaluation, Model Selection, and Algorithm Selection in Machine Learning
• Introduction to Machine Learning Interviews Book · MLIB

Deep Learning

If you want top-down, start with fast.ai.

Fast.ai

• fast.ai (part1, part2) + W&B Study Group

Liked fast.ai? Check out Full Stack Deep Learning.

If you want a more comprehensive, traditional course, check out UNIGE 14x050 — Deep Learning by François Fleuret.

If you need to reach for theory at some point, these are great books.

• Dive into Deep Learning (has code examples in PyTorch, NumPy/MXNet, JAX, and TensorFlow)
• Deep Learning by Ian Goodfellow, Yoshua Bengio, and Aaron Courville
• Neural networks and deep learning
• Understanding Deep Learning (with hands-on notebooks)

Read The Little Book of Deep Learning on your phone instead of scrolling Twitter.

Read these while your neural networks are converging.

• A Recipe for Training Neural Networks
• Deep Neural Nets: 33 years ago and 33 years from now

Do more competitions

• PlantTraits2024 — FGVC11 | Kaggle (computer vision)

Implement papers

Check out labml.ai Annotated PyTorch Paper Implementations

Papers with Code is a great resource; here’s BERT explained on their website.

Below are some resources for the specializations within Deep Learning

Computer Vision

A lot of people recommend CS231n: Deep Learning for Computer Vision. It’s challenging but worth it if you get through it.

Reinforcement Learning

For RL, these two are great:

• Spinning Up in Deep RL by OpenAI
• 🤗 Deep Reinforcement Learning Course — Hugging Face

NLP

Another great Stanford course, CS 224N | Natural Language Processing with Deep Learning

Learn Hugging Face: Hugging Face NLP Course

Check out this Super Duper NLP Repo

Good articles and breakdowns

• BERT Research — Ep. 1 — Key Concepts & Sources · Chris McCormick
• The Illustrated Word2vec — Jay Alammar
• The Illustrated BERT, ELMo, and co. (How NLP Cracked Transfer Learning
• Understanding LSTM Networks — colah’s blog
• PyTorch RNN from Scratch — Jake Tae

Supplementary

• Natural Language Processing with Transformers Book

Large Language Models

First, watch [1hr Talk] Intro to Large Language Models by Andrej.

Then Large Language Models in Five Formulas, by Alexander Rush — Cornell Tech

Watch Neural Networks: Zero to Hero

It starts with explaining and coding backpropagation from scratch and ends with writing GPT from scratch.

Neural Networks: Zero To Hero by Andrej Karpathy

He just released a new video → Let’s build the GPT Tokenizer

You can also look at GPT in 60 Lines of NumPy | Jay Mody while you’re at it.

Free LLM boot camp

A paid LLM Bootcamp released for free by Full Stack Deep Learning.

It teaches prompt engineering, LLMOps, UX for LLMs, and how to launch an LLM app in an hour.

Now that you’re itching to build after this boot camp,

Build with LLMs

Want to build apps with LLMs?

Watch Application Development using Large Language Models
by Andrew Ng

Read Building LLM applications for production by Huyen Chip

As well as Patterns for Building LLM-based Systems & Products by Eugene Yan

Refer to the OpenAI Cookbook for recipes.

Use Vercel AI templates to get started.

Participate in hackathons

lablab.ai has new AI hackathons every week. Let me know if you want to team up!

If you want to go deeper into the theory and understand how everything works:

Read papers

A great article by Sebastian Raschka on Understanding Large Language Models, where he lists some papers you should read.

He also recently published another article with papers you should read in January 2024, covering mistral models.

Follow his substack Ahead of AI.

Write Transformers from scratch.

Read The Transformer Family Version 2.0 | Lil’Log for an overview.

Choose whichever format suits you best and implement it from scratch.

Paper

• Attention Is All You Need
• The Illustrated Transformer
• The Annotated Transformer by Harvard
• Thinking like Transformer

Blogs

• Creating a Transformer From Scratch — Part One: The Attention Mechanism (part 2) (code)
• Understanding and Coding the Self-Attention Mechanism of Large Language Models From Scratch by Sebastian Raschka, PhD
• Transformers from scratch

Videos

• Coding a Transformer from scratch on PyTorch, with full explanation, training and inference
• NLP: Implementing BERT and Transformers from Scratch

You can code transformers from scratch now. But there’s still more.

Watch these Stanford CS25 — Transformers United videos.

Some good blogs

• Gradient Descent into Madness — Building an LLM from scratch
• The Illustrated Transformer — Jay Alammar
• Some Intuition on Attention and the Transformer by Eugene Yan
• Speeding up the GPT — KV cache | Becoming The Unbeatable
• Beyond Self-Attention: How a Small Language Model Predicts the Next Token
• Llama from scratch (or how to implement a paper without crying) | Brian Kitano
• Improving LoRA: Implementing Weight-Decomposed Low-Rank Adaptation (DoRA) from Scratch

Watch Umar Jamil

He has fantastic in-depth videos explaining papers. He also shows you the code.

• LoRA: Low-Rank Adaptation of Large Language Models — Explained visually + PyTorch code from scratch
• Mistral / Mixtral Explained: Sliding Window Attention, Sparse Mixture of Experts, Rolling Buffer
• Attention is all you need (Transformer) — Model explanation (including math), Inference and Training
• LLaMA explained: KV-Cache, Rotary Positional Embedding, RMS Norm, Grouped Query Attention, SwiGLU
• Retrieval Augmented Generation (RAG) Explained: Embedding, Sentence BERT, Vector Database (HNSW)

Some more links related to LLMs that are not exhaustive. Look at LLM Syllabus for a more comprehensive syllabus for LLMs.

Learn how to run open-source models.

Use ollama: Get up and running with Llama 2, Mistral, and other large language models locally

They recently released Python & JavaScript Libraries

Prompt Engineering

Read Prompt Engineering | Lil’Log

ChatGPT Prompt Engineering for Developers by Ise Fulford (OpenAI) and Andrew Ng

DeepLearning.ai also has other short courses you can enroll in for free.

Fine-tuning LLMs

Read the Hugging Face fine-tuning guide.

A good guidebook: Fine-Tuning — The GenAI Guidebook

Check out axolotl.

This is a good article: Fine-tune a Mistral-7b model with Direct Preference Optimization | by Maxime Labonne

RAG

A great article by Anyscale: Building RAG-based LLM Applications for Production

A comprehensive overview of Retrieval Augmented Generation by Aman Chadha

How to stay updated

Combination of newsletters + podcasts + Twitter

For papers, you can follow AK (@_akhaliq)

For podcasts, the best I’ve found is Latent Space by Swyx & Alessio

Join their Discord.

They also have this newsletter, Smol Talk, which summarizes all big AI discords.

Some other newsletters I like are:

• The Batch | DeepLearning.AI | AI News & Insights
• Deep Learning Weekly
• Interconnects | Nathan Lambert
• AI Tidbits | Sahar Mor

More in this article.

Other curriculums/listicles you may find useful.

My list was not meant to be exhaustive, but if you still want to find more, here are some.

• openai/syllabus.md
• AI Canon | Andreessen Horowitz
• AI Learning Curation — LLM Utils
• Threshold to the AI Multiverse | Open DeepLearning
• louisfb01/start-llms: A complete guide to start and improve your LLM skills in 2023

I’ve spent enough time writing and organizing this that it’s diminishing returns. It’s time to learn and build.

I hope this will help you in your AI journey!

If you’ve read this far, don’t forget to reach out or leave a comment :