TPU vs GPU

📜Get repo access at Trelis.com/ADVANCED-inference

Tip: If you subscribe here on YouTube, click the bell to be notified of new vids

🛠️ (NEW) Trelis Benchmarking Seminars: https://trelis.com/workshops-and-seminars/

💡 Need Technical or Market Assistance?

Book a Consult Here: https://forms.gle/wJXVZXwioKMktjyVA

🤝 Are You a Top Developer?

Work for Trelis: https://trelis.com/jobs/

💸 Starting a New Project/Venture?

Apply for a Trelis Grant: https://trelis.com/trelis-ai-grants/

📧 Get Trelis AI Tutorials by Email

Subscribe on Substack: https://trelis.substack.com

Video Links:

- slides: https://docs.google.com/presentation/d/1JH5__AGZRVq_DEpu_Dy_gWh8WR-Y3x2ezK0rnk1GA54/edit?usp=sharing

- one-click-llms repo: https://github.com/TrelisResearch/one-click-llms

TIMESTAMPS:

0:00 Benchmarking Google’s TPUs vs Nvidia GPUs

0:33 Video Overview

1:12 H100 SXM, H200 SXM and v6e hardware specs

4:47 Benchmarking Design with vLLM and llmperf

7:42 Price assumptions per hour

8:47 Tensor Parallel vs Pipeline Parallel

13:45 Pros and Cons of Tensor vs Pipeline Parallel

14:42 Where to test TPUs and GPUs

15:45 Future videos: Blackwell B200 and Amazon Trainium

16:15 Running inference on Nvidia GPUs

19:17 Running inference on Google TPUs

25:51 Running benchmarking with llmperf

28:23 Benchmarking Results: TPU vs GPU

33:21 Conclusion, Resources and Workshop