Show notes are at https://stevelitchfield.com/sshow/chat.html
…
continue reading
Вміст надано PyTorch, Edward Yang, and Team PyTorch. Весь вміст подкастів, включаючи епізоди, графіку та описи подкастів, завантажується та надається безпосередньо компанією PyTorch, Edward Yang, and Team PyTorch або його партнером по платформі подкастів. Якщо ви вважаєте, що хтось використовує ваш захищений авторським правом твір без вашого дозволу, ви можете виконати процедуру, описану тут https://uk.player.fm/legal.
Схожі до PyTorch Developer Podcast
Embedded is the show for people who love gadgets. Making them, breaking them, and everything in between. Weekly interviews with engineers, educators, and enthusiasts. Find the show, blog, and more at embedded.fm.
…
continue reading
This is the audio podcast version of Troy Hunt's weekly update video published here: https://www.troyhunt.com/tag/weekly-update/
…
continue reading
Discover a whole new take on Artificial Intelligence with Squirro's educational podcast! Join host Lauren Hawker Zafer, a top voice in Artificial Intelligence on LinkedIn, for insightful chats that unravel the fascinating world of tech innovation, use case exploration and AI knowledge. Dive into candid discussions with accomplished industry experts and established academics. With each episode, you'll expand your grasp of cutting-edge technologies and their incredible impact on society, and y ...
…
continue reading
The director’s commentary track for Daring Fireball. Long digressions on Apple, technology, design, movies, and more.
…
continue reading
Android Backstage, a podcast by and for Android developers. Hosted by developers from the Android engineering team, this show covers topics of interest to Android programmers, with in-depth discussions and interviews with engineers on the Android team at Google. Subscribe to Android Developers YouTube → https://goo.gle/AndroidDevs
…
continue reading
Three nerds discussing tech, Apple, programming, and loosely related matters.
…
continue reading
A podcast featuring panelists of engineers from Netflix, Twitch, & Atlassian talking over drinks about all things software engineering.
…
continue reading
Welcome to Catalyst, the Launch by NTT DATA Podcast. Catalyst is an ongoing discussion for digital leaders dissatisfied with the status quo and optimistic about what’s possible through smart technology and great people. In this studio we believe in shipping software over slideware, that fast will follow smooth, and aiming to create digital experiences that move millions is a worthy pursuit.
…
continue reading
Download This Show is your weekly guide to the world of media, culture, and technology. From social media to gadgets, streaming services to privacy issues. Each week Marc Fennell and a team of people far smarter than him (his words, not ours) take a fun deep dive into how technology is reshaping our lives.
…
continue reading
Player FM - додаток Podcast
Переходьте в офлайн за допомогою програми Player FM !
Переходьте в офлайн за допомогою програми Player FM !
))
Half precision
Manage episode 301973966 series 2921809
Вміст надано PyTorch, Edward Yang, and Team PyTorch. Весь вміст подкастів, включаючи епізоди, графіку та описи подкастів, завантажується та надається безпосередньо компанією PyTorch, Edward Yang, and Team PyTorch або його партнером по платформі подкастів. Якщо ви вважаєте, що хтось використовує ваш захищений авторським правом твір без вашого дозволу, ви можете виконати процедуру, описану тут https://uk.player.fm/legal.
In this episode I talk about reduced precision floating point formats float16 (aka half precision) and bfloat16. I'll discuss what floating point numbers are, how these two formats vary, and some of the practical considerations that arise when you are working with numeric code in PyTorch that also needs to work in reduced precision. Did you know that we do all CUDA computations in float32, even if the source tensors are stored as float16? Now you know!
Further reading.
- The Wikipedia article on IEEE floating point is pretty great https://en.wikipedia.org/wiki/IEEE_754
- How bfloat16 works out when doing training https://arxiv.org/abs/1905.12322
- Definition of acc_type in PyTorch https://github.com/pytorch/pytorch/blob/master/aten/src/ATen/AccumulateType.h
82 епізодів
Поширити
Manage episode 301973966 series 2921809
Вміст надано PyTorch, Edward Yang, and Team PyTorch. Весь вміст подкастів, включаючи епізоди, графіку та описи подкастів, завантажується та надається безпосередньо компанією PyTorch, Edward Yang, and Team PyTorch або його партнером по платформі подкастів. Якщо ви вважаєте, що хтось використовує ваш захищений авторським правом твір без вашого дозволу, ви можете виконати процедуру, описану тут https://uk.player.fm/legal.
In this episode I talk about reduced precision floating point formats float16 (aka half precision) and bfloat16. I'll discuss what floating point numbers are, how these two formats vary, and some of the practical considerations that arise when you are working with numeric code in PyTorch that also needs to work in reduced precision. Did you know that we do all CUDA computations in float32, even if the source tensors are stored as float16? Now you know!
Further reading.
- The Wikipedia article on IEEE floating point is pretty great https://en.wikipedia.org/wiki/IEEE_754
- How bfloat16 works out when doing training https://arxiv.org/abs/1905.12322
- Definition of acc_type in PyTorch https://github.com/pytorch/pytorch/blob/master/aten/src/ATen/AccumulateType.h
82 епізодів
Усі епізоди
×
Ласкаво просимо до Player FM!
Player FM сканує Інтернет для отримання високоякісних подкастів, щоб ви могли насолоджуватися ними зараз. Це найкращий додаток для подкастів, який працює на Android, iPhone і веб-сторінці. Реєстрація для синхронізації підписок між пристроями.
Схожі до PyTorch Developer Podcast
Show notes are at https://stevelitchfield.com/sshow/chat.html
…
continue reading
Embedded is the show for people who love gadgets. Making them, breaking them, and everything in between. Weekly interviews with engineers, educators, and enthusiasts. Find the show, blog, and more at embedded.fm.
…
continue reading
This is the audio podcast version of Troy Hunt's weekly update video published here: https://www.troyhunt.com/tag/weekly-update/
…
continue reading
Big tech is transforming every aspect of our world. But how, and at what cost? This season of Land of the Giants – The Disney Dilemma – focuses on Disney’s ability to weather the ups and downs of the business cycle and changing tastes and explores what has kept it successful for over 100 years. The entertainment giant has leveraged nostalgia and its intellectual property to build a beloved brand, but after an acquisition spree that included Marvel, Lucasfilm, and 20th Century Fox, can it sus ...
…
continue reading
Discover a whole new take on Artificial Intelligence with Squirro's educational podcast! Join host Lauren Hawker Zafer, a top voice in Artificial Intelligence on LinkedIn, for insightful chats that unravel the fascinating world of tech innovation, use case exploration and AI knowledge. Dive into candid discussions with accomplished industry experts and established academics. With each episode, you'll expand your grasp of cutting-edge technologies and their incredible impact on society, and y ...
…
continue reading
The director’s commentary track for Daring Fireball. Long digressions on Apple, technology, design, movies, and more.
…
continue reading
The American healthcare system is one of the most innovative in the world. But it’s also riddled with complex challenges, such as access to affordable medications, inefficiency and administrative burdens, and communication barriers between providers. There’s clearly a better way—and at Surescripts, we have a unique sightline into what that may be. In this series, host Melanie Marcus, Chief Marketing Officer of Surescripts, sits down with today’s most inspiring and innovative leaders in healt ...
…
continue reading
Android Backstage, a podcast by and for Android developers. Hosted by developers from the Android engineering team, this show covers topics of interest to Android programmers, with in-depth discussions and interviews with engineers on the Android team at Google. Subscribe to Android Developers YouTube → https://goo.gle/AndroidDevs
…
continue reading
Three nerds discussing tech, Apple, programming, and loosely related matters.
…
continue reading
A podcast featuring panelists of engineers from Netflix, Twitch, & Atlassian talking over drinks about all things software engineering.
…
continue reading
Player FM - додаток Podcast
Переходьте в офлайн за допомогою програми Player FM !
Переходьте в офлайн за допомогою програми Player FM !
