What is the dif­fer­ence between CUDA, Tensor and RT Cores?

Many modern NVIDIA GeForce and RTX GPUs combine three spe­cial­ised pro­cessing units, or hardware blocks, each designed for specific tasks. CUDA Cores serve as general-purpose computing units for tra­di­tion­al graphics pro­cessing and parallel cal­cu­la­tions. Tensor Cores are dedicated AI ac­cel­er­at­ors that perform matrix op­er­a­tions used in deep learning, while RT Cores are spe­cial­ised hardware for cal­cu­lat­ing light rays in ray tracing, such as realistic shadows and re­flec­tions.

CUDA, Tensor and RT Cores compared

Many modern NVIDIA GPUs no longer rely on just one type of pro­cessing unit. Instead, they combine several spe­cial­ised hardware blocks on a single chip, with each one optimised for specific tasks.

This spe­cial­ised design allows the same GPU to handle a wide range of workloads: rendering games, pro­cessing AI models, sup­port­ing video editing and creating more realistic lighting and shadows through ray tracing. Each pro­cessing unit has its own role rather than handling every task equally.

The key tech­no­lo­gies are CUDA Cores, Tensor Cores and RT Cores:

• CUDA Cores are versatile pro­cessing units designed for a wide range of tra­di­tion­al graphics and computing tasks.
• Tensor Cores spe­cial­ise in matrix cal­cu­la­tions, which are essential for AI and machine learning. They provide the spe­cial­ised pro­cessing power required for modern AI training and inference workloads.
• RT Cores ac­cel­er­ate dedicated ray tracing cal­cu­la­tions, such as de­term­in­ing how light rays interact with objects in a 3D scene.

GPU tech­no­lo­gies from different man­u­fac­tur­ers

CUDA Cores, Tensor Cores and RT Cores are terms specific to NVIDIA GPUs. Other GPU man­u­fac­tur­ers, such as AMD and Intel, use similar concepts but apply different names and technical im­ple­ment­a­tions. AMD, for example, uses terms such as Compute Units, AI Ac­cel­er­at­ors and Ray Ac­cel­er­at­ors (also known as Raytra­cing Ac­cel­er­at­ors). Intel refers to tech­no­lo­gies including Xe-Cores, XMX Engines and Ray Tracing Units. The exact ar­chi­tec­ture varies between man­u­fac­tur­ers, but the general principle of dividing GPU workloads between spe­cial­ised pro­cessing units is similar.

What are CUDA Cores and how do they work?

CUDA Cores are the tra­di­tion­al pro­cessing units found in NVIDIA GPUs. They are grouped together in larger struc­tures known as streaming mul­ti­pro­cessors and perform many of the GPU’s general-purpose cal­cu­la­tions.

They can be seen as the all-rounders of a GPU: flexible pro­cessing units capable of handling a wide range of standard math­em­at­ic­al op­er­a­tions. They calculate shaders in games, process image data and run physics sim­u­la­tions, among other tasks. Their key strength is the parallel pro­cessing of floating-point numbers, par­tic­u­larly in single-precision format (FP32). For this reason, CUDA Cores have tra­di­tion­ally been as­so­ci­ated with FP32 pro­cessing. However, since the RTX Blackwell gen­er­a­tion, this de­scrip­tion is less precise, as shader cores support both FP32 and INT32 op­er­a­tions. They can execute either floating-point or integer cal­cu­la­tions in each clock cycle.

The basis of CUDA’s ar­chi­tec­ture is SIMT (Single In­struc­tion, Multiple Threads). This execution model is similar to SIMD (Single In­struc­tion, Multiple Data), but it is designed for large numbers of threads running sim­ul­tan­eously. In this approach, 32 threads are grouped together into a unit called a warp.

Threads within a warp typically execute the same in­struc­tion while working with their own in­de­pend­ent data and registers. For example, when a GPU needs to colour a large number of pixels, it does not process the in­struc­tion ‘multiply the colour value by 0.5’ one pixel at a time. Instead, it dis­trib­utes the workload across many parallel threads and warps, allowing similar op­er­a­tions to be carried out sim­ul­tan­eously. This makes GPUs par­tic­u­larly efficient when pro­cessing large, con­sist­ent data sets.

Typical tasks for CUDA Cores

CUDA Cores are par­tic­u­larly effective for workloads where many similar cal­cu­la­tions can be performed sim­ul­tan­eously. Typical examples include:

• classic rendering and shading in games
• vertex, pixel and compute shader op­er­a­tions
• image pro­cessing, filters and post-pro­cessing effects
• physics sim­u­la­tions, such as particle, fluid and collision cal­cu­la­tions
• sci­entif­ic cal­cu­la­tions and numerical sim­u­la­tions
• general GPGPU tasks, where GPUs are used for computing beyond tra­di­tion­al graphics workloads
• data pre-pro­cessing for AI workflows
• parts of media pro­cessing pipelines, such as image and video pre-pro­cessing

What are Tensor Cores and how do they ac­cel­er­ate AI?

With the rise of deep learning, GPU re­quire­ments also increased. The reason is that neural networks rely heavily on large numbers of matrix cal­cu­la­tions. When training and running inference on large AI models, enormous sets of numbers must be mul­ti­plied and added together re­peatedly.

CUDA Cores can perform these cal­cu­la­tions in principle. However, while they are highly flexible, they are not spe­cific­ally optimised for pro­cessing large matrix op­er­a­tions at maximum speed. This is where Tensor Cores provide an advantage. These spe­cial­ised pro­cessing units in modern NVIDIA GPUs are designed to handle matrix and tensor op­er­a­tions much more ef­fi­ciently than general-purpose compute units.

Tensor Cores ac­cel­er­ate a process known as GEMM (General Matrix Multiply), or general matrix–matrix mul­ti­plic­a­tion. Instead of pro­cessing in­di­vidu­al cal­cu­la­tions sep­ar­ately, they can handle small matrix blocks at once by combining multiple mul­ti­plic­a­tion and addition op­er­a­tions into dedicated hardware in­struc­tions.

A key factor behind the per­form­ance of Tensor Cores is mixed precision, which involves using different numerical formats for different parts of a cal­cu­la­tion. Depending on the GPU gen­er­a­tion and workload, Tensor Cores support formats such as TF32, BF16, FP16, FP8, FP6, FP4 and INT8. Using lower-precision formats can sig­ni­fic­antly increase through­put and reduce memory re­quire­ments.

The benefit is that smaller number formats require less memory, can be trans­ferred more ef­fi­ciently and allow higher pro­cessing through­put. At the same time, important in­ter­me­di­ate cal­cu­la­tions can be ac­cu­mu­lated at higher precision to reduce the impact of rounding errors. For many AI workloads, this provides an effective balance: cal­cu­la­tions run sig­ni­fic­antly faster and more ef­fi­ciently while main­tain­ing largely com­par­able results.

Typical tasks for Tensor Cores

Tensor Cores are primarily used for workloads that involve large numbers of matrix and tensor cal­cu­la­tions. Typical ap­plic­a­tions include:

• training deep neural networks
• inference for large language models and gen­er­at­ive AI systems
• trans­former models, diffusion models and re­com­mend­a­tion systems
• AI-powered image upscaling and frame gen­er­a­tion, such as DLSS
• speech, image and object re­cog­ni­tion
• low-precision or quantised inference using formats such as INT8, FP8 or FP4
• sci­entif­ic ap­plic­a­tions involving extensive linear algebra cal­cu­la­tions

What are RT Cores and how do they ac­cel­er­ate ray tracing?

The third spe­cial­ised unit, RT Cores, tackles one of the most com­pu­ta­tion­ally demanding graphics tasks: hardware-ac­cel­er­ated ray tracing. With ray tracing, light rays are traced through a scene to simulate shadows, re­flec­tions, re­frac­tions and global il­lu­min­a­tion in a more phys­ic­ally accurate way. The com­pu­ta­tion­al workload is enormous because, for every in­di­vidu­al ray, the system must determine which geometric object it en­coun­ters first.

To make this process more efficient, ray tracing systems use so-called Bounding Volume Hier­arch­ies (BVH). This is a search structure that organises a 3D scene using nested bounding boxes. Large sections of the scene are enclosed in broad boxes first. Only when a ray in­ter­sects one of these boxes does the system examine the smaller areas or in­di­vidu­al triangles inside. This means the GPU does not need to test every triangle sep­ar­ately. Instead, it can quickly eliminate ir­rel­ev­ant parts of the scene and pro­gress­ively narrow the search down to the objects a ray may actually hit.

This is where RT Cores come into play. They primarily ac­cel­er­ate two key tasks:

• BVH traversal: nav­ig­at­ing through the bounding box structure
• ray–triangle in­ter­sec­tion tests: de­term­in­ing whether and where a ray in­ter­sects a specific polygon

The main advantage is that these cal­cu­la­tions are handled directly by dedicated RT Core hardware rather than using CUDA Cores. This reduces the workload on the general-purpose shader units, allowing them to focus on tasks such as shading and material cal­cu­la­tions at the same time. Without this spe­cial­ised pro­cessing, real-time ray tracing at current per­form­ance levels would not be possible.

Typical tasks for RT Cores

RT Cores are used whenever large numbers of rays need to be traced through a 3D scene and tested against its geometry. Typical ap­plic­a­tions include:

• real-time ray tracing in games, such as more realistic re­flec­tions, shadows and ambient occlusion
• path tracing and global il­lu­min­a­tion for highly realistic 3D en­vir­on­ments
• rendering and visu­al­isa­tion in ar­chi­tec­ture, product design, film and animation
• phys­ic­ally based rendering, where light, materials and surfaces are simulated as real­ist­ic­ally as possible
• fast searches through 3D scenes using struc­tures such as BVHs
• sci­entif­ic visu­al­isa­tion involving large datasets rendered with ray tracing

How NVIDIA’s CUDA Cores, Tensor Cores and RT Cores work together

The strength of modern NVIDIA GPUs does not come from in­di­vidu­al pro­cessing units alone, but from their division of labour. CUDA Cores, Tensor Cores and RT Cores perform different tasks and work together to improve per­form­ance. This in­ter­ac­tion can be il­lus­trated using a modern game scene with ray tracing and AI upscaling:

1. First, the CUDA Cores, together with other spe­cial­ised units, handle many tra­di­tion­al graphics tasks. These include geometry pro­cessing, ras­ter­isa­tion, shaders, textures and material effects. They perform much of the core image pro­cessing required to create the final scene.
2. When ray tracing is enabled, the RT Cores take over specific cal­cu­la­tions. They ac­cel­er­ate the process of finding in­ter­sec­tions between light rays and objects in the scene by tra­vers­ing BVH struc­tures and de­term­in­ing which triangles a ray hits. This in­form­a­tion is then used for effects such as realistic shadows, re­flec­tions and lighting.
3. Finally, the Tensor Cores use AI-based methods to enhance the rendered image. With DLSS (Deep Learning Super Sampling), for example, a lower-res­ol­u­tion image is upscaled and improved using a neural network, producing a higher-quality result without requiring every pixel to be cal­cu­lated tra­di­tion­ally. On GeForce RTX 50 Series GPUs, DLSS 4.5 adds Dynamic Multi Frame Gen­er­a­tion and a 6X mode that can generate up to five ad­di­tion­al frames for every rendered frame. DLSS 5 goes further with 3D-Guided Neural Rendering, which uses the Tensor Cores to improve lighting and material detail in real time.

By dividing tasks in this way, each pro­cessing unit handles the workloads it is best suited for, reducing com­pu­ta­tion time and improving overall ef­fi­ciency.

CUDA, Tensor and RT Cores: direct com­par­is­on

The three core types in modern NVIDIA GPUs mainly differ in their areas of re­spons­ib­il­ity, the math­em­at­ic­al op­er­a­tions they are designed to ac­cel­er­ate and the workloads they are optimised for: