What's the Difference Between AWS vs. Azure vs. Google Cloud?
October 1, 2024
Article
Recommended experience
Intermediate level
Anyone who wants to learn efficiently serving LLM apps, exploring components, techniques, and tradeoffs. Requires intermediate Python.
Recommended experience
Intermediate level
Anyone who wants to learn efficiently serving LLM apps, exploring components, techniques, and tradeoffs. Requires intermediate Python.
Learn how Large Language Models (LLMs) repeatedly predict the next token, and how techniques like KV caching can greatly speed up text generation.
Code for efficient LLM app serving, balancing model output speed and serving many users at once.
Explore the fundamentals of Low Rank Adapters and see how Predibase builds their framework inference server to serve fine-tuned models at once.
Only available on desktop
Join our new short course, Efficiently Serving Large Language Models, to build a ground-up understanding of how to serve LLM applications from Travis Addair, CTO at Predibase. Whether you’re ready to launch your own application or just getting started building it, the topics you’ll explore in this course will deepen your foundational knowledge of how LLMs work, and help you better understand the performance trade-offs you must consider when building LLM applications that will serve large numbers of users.
You’ll walk through the most important optimizations that allow LLM vendors to efficiently serve models to many customers, including strategies for working with multiple fine-tuned models at once. In this course, you will: 1. Learn how auto-regressive large language models generate text one token at a time. 2. Implement the foundational elements of a modern LLM inference stack in code, including KV caching, continuous batching, and model quantization, and benchmark their impacts on inference throughput and latency. 3. Explore the details of how LoRA adapters work, and learn how batching techniques allow different LoRA adapters to be served to multiple customers simultaneously. 4. Get hands-on with Predibase’s LoRAX framework inference server to see these optimization techniques implemented in a real world LLM inference server. Knowing more about how LLM servers operate under the hood will greatly enhance your understanding of the options you have to increase the performance and efficiency of your LLM-powered applications.
DeepLearning.AI is an education technology company that develops a global community of AI talent. DeepLearning.AI's expert-led educational experiences provide AI practitioners and non-technical professionals with the necessary tools to go all the way from foundational basics to advanced application, empowering them to build an AI-powered future.
Hands-on, project-based learning
Practice new skills by completing job-related tasks with step-by-step instructions.
No downloads or installation required
Access the tools and resources you need in a cloud environment.
Available only on desktop
This project is designed for laptops or desktop computers with a reliable Internet connection, not mobile devices.
DeepLearning.AI
Course
DeepLearning.AI
Course
Duke University
Course
Duke University
Course
Unlimited access to 10,000+ world-class courses, hands-on projects, and job-ready certificate programs - all included in your subscription
Earn a degree from world-class universities - 100% online
Upskill your employees to excel in the digital economy
In Projects, you'll complete an activity or scenario by following a set of instructions in an interactive hands-on environment. Projects are completed in a real cloud environment and within real instances of various products as opposed to a simulation or demo environment.
By purchasing a Project, you'll get everything you need to complete the Project including temporary access to any product required to complete the Project.
Even though Projects are technically available on mobile devices, we highly recommend that you complete Projects on a laptop or desktop only.
Yes, you can download and keep any of your created files from the Project. To do so, please make sure you save any files and work to your device before exiting the product environment.
Projects are not eligible for refunds. See our full refund policy.
Financial aid is not available for Projects.
In rare instances, Projects may be taken down for maintenance or other reasons. If you are experiencing any issues, please contact us.
Auditing is not available for Projects.
At the top of the page, you can view the experience level recommended for this Project.
Yes, everything you need to complete your Project will be available in your browser.