Vizuara Kernel Engineering Cohort begins Oct 12, 2026
Registration is open. The launch price of $3,000 is valid July 22 – August 15, 2026, after which it rises by 10%.
Vizuara
Vizuara AI Labs

Vizuara's Kernel Engineering
Workshop is here

In partnership withCrusoe
Vizuara is an Anthropic partner

The most modern GPU kernel-engineering program anywhere, built over two years, taught live, and explained from the ground up, from the silicon up to Flash Attention 4, Blackwell, and kernels written by AI. If you want to actually be ready for GPU performance-engineering roles at the frontier, this is it.

The third and final chapter of Vizuara's GPU trilogy, after 5D Parallelism and the Inference Engineering Workshop.

25 sessions 6 parts lectures paired with live coding Mon · Wed · Fri, 7:00–9:00 AM begins October 12, 2026
Enroll → See the curriculum
Built on the modern inference stack
NVIDIA Hopper · H100 Blackwell · GB200 CUDA Triton CUTLASS PyTorch vLLM DeepSeek Crusoe
The exact tools and hardware you'll write kernels for: Hopper & Blackwell tensor cores, CUDA, Triton, CUTLASS/CuTe, and the production inference stack.
Why this is different

The most modern kernel curriculum anywhere

Most GPU courses stop at 2022, and the few modern ones are written for people who already know everything. This one lives on the 2025-2026 frontier but explains each idea from first principles, with, for every topic, why it matters, where it's used, and which companies use it.

Explained simply

Every session opens with the idea in simple words before the jargon, you'll leave knowing what the terms actually mean.

The real frontier

Flash Attention 4, Blackwell, NVFP4, DeepSeek, AI-written kernels, the newest material, not a 2022 rerun.

Built by hand, live

Every lecture is paired with a live-coding session. You write the kernels yourself, step by step.

Job-ready output

Maps 1:1 to what frontier companies like Wafer, NVIDIA and the top labs actually hire for.

The curriculum

Six parts, from the CPU to AI-written kernels

A single, deliberate arc. Each concept lecture (Dr. Raj Dandekar) is paired with a live-coding session (Shubham Panchal) so theory and practice go hand in hand. Click any session, each one explains the idea simply, plus why it matters, where it's used, and who uses it.

Part IParallelism, from the CPU up1 session + 3 pairs

Get comfortable with "doing many things at once" on a CPU first, then meet the GPU and how you program it, so every kernel that follows sits on solid foundations.

CPUParallelism on the CPUThe three ways a computer does many things at once, before you watch the GPU do it at massive scale.Live session · Shubham›
Inside
  • Do the same math on many numbers at once (SIMD): one instruction, eight results, using AVX, Arm Neon, and Google Highway, and watching what the compiler speeds up for you automatically.
  • Run many tasks at the same time: lightweight "threads" done the modern way (Go goroutines, Java virtual threads, Rust's tokio) versus the classic way (pthreads, OpenMP).
  • How one CPU core secretly overlaps work (pipelining, out-of-order execution), the same trick the GPU takes to the extreme.
Why it matters It's the mental model behind all fast code, do more work at the same time.
Where it's used Every high-performance library, long before you touch a GPU.
Who uses it Game engines, high-frequency trading, NumPy & PyTorch's CPU paths.
Lecture 1 · Raj
How fast can this go?
↔
Coding session 1 · Shubham
Roofline lab: predict then measure
›
The lecture
What's the speed limit, and why?
  • Every kernel is held back by one of two things: how fast the chip can do math, or how fast it can move data.
  • The "roofline": a simple chart that tells you the fastest a task could run before you write a line of code.
  • One number ("math done per byte loaded") predicts almost everything.
The live build
Predict, then check on a real GPU.
  • Time real PyTorch operations and see which limit each one hits.
  • Guess "math-bound or memory-bound?", then confirm it on hardware.
Why it matters You learn the speed limit before optimizing, so you never waste effort chasing the wrong thing.
Where it's used The first thing any performance engineer does with a new kernel.
Who uses it NVIDIA, PyTorch, and every ML-infra team.
Lecture 2 · Raj
The CUDA programming model
↔
Coding session 2 · Shubham
Your first CUDA kernels + GPU Puzzles
›
The lecture
How a GPU actually runs your code.
  • Thousands of tiny threads, grouped into "warps" of 32 that move in lockstep.
  • What it costs when threads in a group disagree and have to wait ("warp divergence").
  • How your code turns into the real machine instructions, and why it pays to read them.
The live build
Write your first kernels.
  • Launch your very first GPU programs from scratch.
  • Work through GPU Puzzles, bite-sized kernel challenges.
  • Look at the actual assembly your code compiled to.
Why it matters The foundation, you can't speed up what you don't understand.
Where it's used Every CUDA program ever written.
Who uses it Anyone doing GPU work, NVIDIA and every AI lab.
Lecture 3 · Raj
The memory hierarchy in anger
↔
Coding session 3 · Shubham
The matrix-transpose ladder
›
The lecture
Why moving data is the real problem.
  • GPUs are fast at math but slow to fetch data, so how you arrange memory access is everything.
  • When neighbouring threads read neighbouring memory, it's ~10× faster ("coalescing"), the single biggest lever.
  • Two quiet traps that halve your speed: "bank conflicts" and low "occupancy".
The live build
Make a simple kernel fast, step by step.
  • Take a matrix "transpose" and speed it up one fix at a time.
  • Use NVIDIA's profiler (Nsight Compute) to see exactly where the time goes.
Why it matters Memory, not math, is usually the bottleneck, fixing it gives the biggest wins.
Where it's used Every GPU kernel that touches a lot of data.
Who uses it NVIDIA, DeepSeek, vLLM, and every inference team.
Part IIThe GEMM worklog: naive → cuBLAS3 pairs

Take the single most important operation in AI, matrix multiply, from a version that uses ~1% of the GPU to one that beats NVIDIA's own library, one improvement at a time.

Lecture 4 · Raj
GEMM worklog I
↔
Coding session 4 · Shubham
GEMM by hand I
›
The lecture
The first four versions of matrix multiply.
  • Matrix multiply ("GEMM") is ~90% of what a neural network does.
  • A naive version uses only 1.3% of the GPU, we'll see exactly why.
  • Two classic fixes (coalescing + "tiling") take it to 36.5%, measuring each gain.
The live build
Write them yourself.
  • Code the first four matrix-multiply kernels by hand.
  • Profile each and watch the number climb from 1.3% → 36.5%.
Why it matters Matrix multiply is the heart of every model, make it fast and you make everything fast.
Where it's used Training and running every AI model on Earth.
Who uses it NVIDIA (cuBLAS) and every AI lab.
Lecture 5 · Raj
GEMM worklog II
↔
Coding session 5 · Shubham
GEMM by hand II
›
The lecture
The advanced tricks that reach library speed.
  • Reuse data in the fastest memory (registers), load four numbers at once ("float4"), and let the code auto-pick its best settings.
  • "Warptiling", the technique that gets you to 93.7% of NVIDIA's hand-tuned library.
The live build
Reach near-library speed by hand.
  • Push your own matrix multiply toward 93.7% of cuBLAS.
  • Watch the compiled code shrink from 8 memory loads to 2, the "aha" moment.
Why it matters This is the gap between a hobby kernel and a production one, real, measurable speed.
Where it's used The core of every fast ML library.
Who uses it NVIDIA, OpenAI, Meta, anyone training big models.
Lecture 6 · Raj
Tensor cores, the second worklog
↔
Coding session 6 · Shubham
A WMMA tensor-core GEMM
›
The lecture
The GPU's special matrix-math units.
  • Modern GPUs have "tensor cores" that do matrix math ~10× faster than normal cores.
  • How to feed them the exact data layout they demand.
  • The precision menu (FP16 / BF16 / TF32): trading a little accuracy for a lot of speed.
The live build
Program the tensor cores directly.
  • Use the tensor cores (WMMA) and beat your own best regular kernel.
Why it matters Tensor cores are why modern AI is affordable, using them well is a must-have skill.
Where it's used All modern training and inference.
Who uses it NVIDIA and every frontier lab.
Part IIIAttention & profiling2 pairs

Learn to find why a kernel is slow like a pro, then build the operation at the heart of every LLM, FlashAttention, and kick off your capstone.

Lecture 7 · Raj
Profiling & debugging like a pro
↔
Coding session 7 · Shubham
Debug 3 sabotaged kernels
›
The lecture
Find the real problem, don't guess.
  • Read NVIDIA's profiler like a pro and pinpoint the true bottleneck.
  • The professional debugging toolkit, the same workflow the vLLM team uses.
The live build
Fix broken kernels, live.
  • Hunt down and fix the bugs in three sabotaged kernels, under time pressure.
Why it matters Finding why a kernel is slow or wrong is half the job.
Where it's used Daily work at every GPU / inference team.
Who uses it vLLM, NVIDIA, and all inference teams.
Lecture 8 · Raj
Attention: the kernel that ate the world
↔
Coding session 8 · Shubham
Build FlashAttention v1 live
›
The lecture
The operation behind every LLM.
  • "Attention" is what lets a model look back over your whole prompt, and it's the biggest memory hog.
  • The clever trick ("online softmax") that made it fit in fast memory and unlocked long context.
  • Why generating text one word at a time is a different, memory-bound problem.
The live build
Build the famous fast-attention kernel.
  • Write FlashAttention from scratch, and start your capstone.
Why it matters FlashAttention is why long-context, ChatGPT-style models are even possible.
Where it's used Every LLM, everywhere.
Who uses it OpenAI, Anthropic, Meta, Google, DeepSeek.
Part IVThe modern frontier6 deep-dives

The 2025-2026 kernels the best labs ship right now: Hopper, Blackwell, DeepSeek, and Flash Attention 4, each one explained from the ground up, not assumed.

Deep-dive 1FlashAttention from scratchRebuild the famous fast-attention kernel properly, then see what changed in FA2 and FA3.›
Inside
  • Rebuild the full fast-attention kernel, cleanly.
  • The trick that keeps the math correct while streaming through data, plus "masking" so the model only looks backwards.
  • What improved in the newer versions: FA2 (splits the work better) and FA3 (uses the H100's async features).
Why it matters The reference kernel every inference stack quietly depends on.
Where it's used LLM training and serving.
Who uses it OpenAI, Anthropic, Together AI, vLLM, SGLang.
Deep-dive 2Beating cuBLAS on an H100Use the H100's newest features to build a matrix multiply that beats NVIDIA's own library.›
Inside
  • Use the H100's newest features to move data (TMA) and do matrix math (WGMMA) in the background, so nothing waits.
  • "Warp specialization": some threads fetch data while others compute, like an assembly line.
  • Put it together into a matrix multiply that beats NVIDIA's own library.
Why it matters Squeezing the last 10-20% from the most expensive hardware saves millions of dollars.
Where it's used Frontier training clusters.
Who uses it NVIDIA, DeepSeek (DeepGEMM), OpenAI.
Deep-dive 3Triton → CUTLASS → CuTe-DSLThe modern, faster ways to write kernels, including writing them in Python.›
Inside
  • Triton: write a fast kernel in ~40 lines of Python instead of hundreds of lines of C++.
  • CUTLASS: NVIDIA's powerful (but hard) C++ toolkit, and its layout system.
  • The big 2025 shift: top kernels (FA4, DeepGEMM) are now written in Python that compiles down to the metal.
Why it matters The modern way to write kernels, far less code, the same speed.
Where it's used Most new kernel development today.
Who uses it OpenAI (Triton), NVIDIA (CUTLASS), DeepSeek.
Deep-dive 4Inference-serving kernelsThe kernels that make AI products fast and cheap enough to actually ship.›
Inside
  • Two very different jobs: reading your prompt ("prefill") vs writing the answer one word at a time ("decode").
  • PagedAttention: how vLLM packs many users' conversations into memory efficiently.
  • Speculative decoding: a small model drafts, a big model checks, 2-3× faster replies.
  • Running models in lower precision (FP8, 4-bit) to serve more users per GPU.
Why it matters This is what makes AI products fast and cheap enough to ship to millions.
Where it's used Every AI API and chatbot you've used.
Who uses it vLLM, SGLang, OpenAI, Anthropic, NVIDIA (TensorRT-LLM).
Deep-dive 5Blackwell & NVFP4NVIDIA's newest chip and its 4-bit number format that roughly doubles throughput.›
Inside
  • NVIDIA's newest chip (Blackwell) has brand-new tensor cores ("tcgen05") and a new fast on-chip memory ("TMEM").
  • NVFP4: a tiny 4-bit number format that roughly doubles inference throughput.
  • How two chips can even team up on a single operation.
Why it matters The very newest hardware, being early here is a rare, in-demand skill.
Where it's used 2025-26 frontier data centres.
Who uses it NVIDIA and the labs deploying on GB200.
Deep-dive 6Flash Attention 4The fastest attention kernel on Earth right now, explained from the ground up.New in 2026›
Inside: the FlashAttention story, FA1 → FA4
  • Attention has been rewritten four times as GPUs changed. FA4 is the newest, built for Blackwell.
  • The key idea, simply: the matrix units got so fast that the "softmax" step (a normalization the model does) became the slow part, so FA4 rearranges the whole kernel so the fast units never sit idle.
  • Its clever move ("adaptive softmax") is to only redo that normalization when the numbers actually change enough, skipping most of the work. It's the first attention kernel to break a "petaflop" (a thousand trillion operations a second), and it's written in Python.
Why it matters The fastest attention that exists today, you'll understand it before almost anyone.
Where it's used Cutting-edge Blackwell inference (brand-new, still in beta).
Who uses it Tri Dao / Together AI, and NVIDIA (folded into cuDNN).
Part VAI-written kernels2 deep-dives

The newest twist of all: tiny hand-crafted kernels that beat huge libraries, and AI models that now write GPU kernels themselves, and how a kernel engineer guides them.

Deep-dive 7DeepSeek: FlashMLA & DeepGEMMProof that a small, well-crafted kernel can beat the giants.›
Inside
  • DeepSeek open-sourced tiny, super-fast kernels that beat much bigger libraries.
  • FlashMLA: fast attention for their memory-saving design.
  • DeepGEMM: a ~300-line matrix multiply that outperforms huge codebases, the lesson being that smart scheduling beats brute-force code.
Why it matters Proof a small, sharp kernel can beat the giants, a great story for your portfolio.
Where it's used DeepSeek's models, and now widely copied.
Who uses it DeepSeek and the open-source inference community.
Deep-dive 8LLM-generated kernels: the agent + profiler loopAI models are now writing GPU kernels, and your new job is to guide them.New in 2025-26›
Inside
  • The frontier: AI models are starting to write GPU kernels themselves.
  • KernelBench: the standard test for "can an AI write a correct, fast kernel?"
  • AlphaEvolve (Google DeepMind): an AI that discovered a faster matrix multiply and sped up a real attention kernel by ~32%.
  • Your new role: guiding these AI agents with a profiler in the loop, plus an honest look at where the hype breaks down.
Why it matters The future of the job, kernel engineers increasingly supervise AI that writes kernels.
Where it's used Research today, production tomorrow.
Who uses it Google DeepMind, NVIDIA, Stanford, Meta.
Part VIKernel Engineering Capstone Projectswith Crusoe

A real kernel-engineering project, built on capstone project ideas from Crusoe and graded at demo day, the closest thing to an on-site interview before the on-site interview.

CapstoneYour capstone: a real kernel-engineering projectSolve a genuine kernel problem end to end and present it at demo day.›
Inside
  • A real kernel-engineering problem, not a toy exercise. Our partner Crusoe shares capstone project ideas as the cohort starts.
  • You solve it end to end, using everything from Parts I-V.
  • At demo day, the capstones are graded and the standouts are picked.
Why it matters A real, graded project for your portfolio and interviews.
Where it's used Exactly the kind of work frontier companies hire for.
Who uses it The frontier labs hiring GPU engineers.
Who teaches it

Taught live by practitioners

Every session is live, concepts built from first principles, then written into working kernels in front of you.

Dr. Raj Dandekar
Dr. Raj DandekarLectures
MIT PhD · Founder, Vizuara

Has designed and taught Vizuara's 5D Parallelism and Inference Engineering workshops, the first two chapters of this GPU trilogy, and is known for taking hard systems and ML material and making it click from first principles. He teaches the concept lectures.

5D Parallelism WorkshopInference Engineering WorkshopMIT PhD
Shubham Panchal
Shubham PanchalLive coding
ML + on-device engineer · creator of SmolChat

An Android developer since 2017, Shubham specializes in deploying ML models on-device in Android apps, the creator of SmolChat (run LLMs locally, on-device). An active StackOverflow contributor and frequent Medium writer on ML, math and Android, he now works across Rust, C and C++ alongside Java/Kotlin backends. He leads the live-coding, where you write every kernel with him.

SmolChat, on-device LLMsRust / C / C++Kotlin / JavaStackOverflow · Medium
The partnership

In partnership with Crusoe

This cohort runs in partnership with Crusoe, the AI-infrastructure company. The partnership is built into the workshop itself.

Crusoe
Vizuara's Kernel Engineering Workshop, founding cohort 2026, runs in partnership with Crusoe.

A guest lecture from Crusoe

A session with the Crusoe team during the cohort, scheduled around the middle or the end of the nine weeks.

Capstone project ideas

Crusoe shares capstone project ideas as the cohort starts, so your final project tracks problems an AI-infrastructure company actually cares about.

Everything included

What you get when you enroll

One price, the whole system. The live cohort is the centre, and around it sits everything we've built for kernel engineers, the book, the capstone projects, the interview-preparation platform, and a bonus we think you'll love.

25 sessions

Every concept lecture paired with a live-coding session, Mon · Wed · Fri, 7:00–9:00 AM, October 12 → December 7.

Vizuara's Kernel Engineering Book

Full access to our illustrated Kernel Engineering book, the written companion to every session. Unlocked when you register.

Kernel Engineering Capstone Projects

A real kernel-engineering project, with project ideas from Crusoe, solved end to end and graded at demo day.

Interview Preparation website

Access to the Kernel Engineering Interview Preparation website, practice for the exact roles this workshop gears you for.

All code files & recordings

Every kernel you build in the cohort, plus session recordings, yours to keep.

Bonus: Vizz-AI & Vizuara AI Pods

One month of access to Vizz-AI and Vizuara AI Pods, an additional bonus on top of the workshop.

Everything above is included in the single $3,000 launch plan, valid July 22 – August 15, 2026. After August 15 the price rises by 10%.
Be ready for the frontier

This gears you for real GPU-engineering jobs

GPU performance engineering is one of the most sought-after, hardest-to-fill skills in AI. This isn't a toy syllabus, it maps section-for-section onto what companies like Wafer, NVIDIA, and the top labs actually hire for.

YOU BUILD THE JOB ASKS FOR GEMM worklog 1.3% → ~94% of cuBLAS FlashAttention, live FA1 → FA4, serving kernels Triton · CUTLASS · CuTe the modern authoring layer Nsight profiling PTX/SASS, 3 sabotaged kernels Capstone project ideas from Crusoe, graded you, after 9 weeks "CUDA kernel optimization" "FlashAttention / vLLM stack" "Triton or CUTLASS at scale" "profile & ship on H100s" a portfolio piece to defend

Matmul → cuBLAS parity + tensor cores

The core screen for every GPU-perf role, tiling, vectorization, tensor cores, reasoning about speed byte-for-byte.

NVIDIAAnthropic JDWafer

FlashAttention + inference kernels

FlashAttention 1-4, PagedAttention, KV-cache, speculative decoding, the exact stack in frontier inference roles.

vLLM / SGLangTensorRT-LLM

Triton, CUTLASS & CuTe internals

The modern kernel-authoring layer, warp-specialized GEMM, and epilogue fusion.

Kernel Engineer (CUDA/Triton)

Profiling, PTX/SASS & the AI-kernel loop

Nsight/NCU, reading the compiled assembly, and the agent + profiler loop that frontier GPU startups productionize.

WaferKernelBench

These roles are among the highest-paid, most in-demand engineering jobs in the industry, the kind where a single week's pay dwarfs the cost of this workshop. You leave able to point at each line of a job description and say: built that, graded on it, wrote the chapter.

Investment

Worth every bit of it

$3,000
per person · one plan, everything included
Launch price ends August 15, 2026
–days
–hours
–min
–sec
After August 15 the price rises by 10% to $3,300.
Live & graded: every session taught live; a real capstone project graded at demo day.
The book included: full access to Vizuara's Kernel Engineering Book, unlocked at registration.
Job-ready: maps 1:1 to frontier GPU-engineering roles that pay multiples of the fee, plus interview-prep website access.
Explained from first principles: from the CPU up to Flash Attention 4, nothing assumed.

Nobody else sells a single live, graded, book-backed path through all of this. Against what these skills unlock, $3,000 is a fraction of what the job it prepares you for pays in a single week.

When. The cohort begins Monday, October 12, 2026 and runs Mon · Wed · Fri, 7:00–9:00 AM, through December 7, 2026, 25 sessions in all.
Enroll now — $3,000 →

Seats are limited; the cohort is kept small enough for live review of every participant's kernels. Questions before enrolling? Email team@vizuara.com.

Class timetable

Nine weeks, Mon · Wed · Fri, 7:00–9:00 AM

The cohort begins Monday, October 12, 2026 and finishes on Monday, December 7, 2026, 25 sessions of two hours each. Every week runs Monday, Wednesday and Friday, 7:00–9:00 AM, and every session is recorded.

WeekDatesSessionsWhat we cover
Week 1Oct 12 · 14 · 161–3CPU parallelism · the roofline · your first CUDA kernels
Week 2Oct 19 · 21 · 234–6The CUDA programming model · GPU Puzzles · the memory hierarchy
Week 3Oct 26 · 28 · 307–9The transpose ladder · GEMM worklog begins, 1.3% → 36.5%
Week 4Nov 2 · 4 · 610–12GEMM to ~94% of cuBLAS · tensor cores
Week 5Nov 9 · 11 · 1313–15A WMMA tensor-core GEMM · profiling & debugging like a pro
Week 6Nov 16 · 18 · 2016–18Attention · FlashAttention built live · capstone kickoff
Week 7Nov 23 · 25 · 2719–21Beating cuBLAS on an H100 · Triton → CUTLASS → CuTe · inference-serving kernels
Week 8Nov 30 · Dec 2 · 422–24Blackwell & NVFP4 · Flash Attention 4 · DeepSeek's kernels
Week 9Dec 725AI-written kernels · the agent + profiler loop · cohort wrap

Capstone demo day follows the final session; the exact date is announced inside the cohort.

FAQ

Common questions

01When are the classes?›
  • Monday, Wednesday and Friday, 7:00–9:00 AM, from October 12 to December 7, 2026, 25 sessions of two hours each, taught live.
02What if I can't attend live?›
  • Every session is recorded, and recordings plus all code files stay with you. Live attendance is strongly encouraged, the live-coding sessions are where the learning compounds.
03How much does it cost, and until when?›
  • One plan, $3,000, everything included. The launch price is valid July 22 – August 15, 2026; after August 15 it rises by 10% to $3,300.
04What do I get beyond the live sessions?›
  • Full access to Vizuara's Kernel Engineering Book (unlocked at registration), the Kernel Engineering Capstone Projects, the Interview Preparation website, all code files and recordings, and as a bonus, one month of Vizz-AI and Vizuara AI Pods access.
05What are the prerequisites?›
  • Comfort with Python and basic PyTorch. No CUDA or C++ experience is assumed, the curriculum deliberately starts at the CPU and builds every idea from first principles.
06Do I need my own GPU?›
  • No. All the live builds run on cloud GPUs, and we walk you through the exact setup we use in the first week.
07I took 5D Parallelism / the Inference Engineering Workshop. How does this fit?›
  • This is the third and final chapter of the GPU trilogy. It stands fully on its own, but if you've done the earlier workshops, this completes the picture, from distributed training and inference systems down to the kernels themselves.
08I have a question that isn't answered here.›
  • Email team@vizuara.com and we'll get back to you quickly.
Enroll

The cohort begins October 12

One plan, $3,000, everything included, and the launch price holds only until August 15, 2026.

Enroll now — $3,000 →