Replication does not increase model capacity
Data parallelism puts a separate model copy on each GPU to serve more independent work. Each copy must still fit on its own GPU. Two 24 GiB cards running separate copies cannot each hold a 40 GiB model.
To run a model larger than one GPU, the runtime must partition the model. Layer splitting and tensor parallelism place work differently and have different transfer requirements.
Where the simple sum breaks down
A 48 GiB total across two cards is a possible aggregate budget, not a single contiguous allocation. Some buffers or tensors may be duplicated, layer sizes may be uneven, and the device hosting the output can carry extra work.
The planner labels multi-GPU results conditional and includes a runtime allowance per device. It does not perform per-layer placement. Even a comfortable aggregate estimate requires checking the actual split in your chosen runtime.
Plan the machine as well as the model
Check usable PCIe slots and lane widths, case clearance, airflow, power connectors, the power supply and sustained board power. Two RTX 3090 cards alone can have a combined 700 W graphics-card power rating, before the rest of the system.
An interconnect does not make an unsupported software path work automatically. NVLink use depends on hardware and runtime support; PCIe transfer costs can matter. Benchmark the exact setup before assuming speed scales with GPU count.
See it on your own setup.
Change the model, hardware or context and see the memory budget update.
Try this example