Academic or research source. Check the methodology, sample size, and whether it's been replicated.
[RFC] Add run_dtensor_rng_op HOP to make DTensor RNG traceable
Summary: Fixes: #147757 , #156649 This PR introduces a new higher-order operator ( run_dtensor_rng_op ) that makes DTensor random operations traceable by torch.compile . Previously, DTensor random...
PyTorch Releases··README: ~3 min
2-Minute Brief
According to PyTorch Releases: Summary: Fixes: #147757 , #156649 This PR introduces a new higher-order operator ( run_dtensor_rng_op ) that makes DTensor random operations traceable by torch.compile . Previously, DTensor random ops (dropout, uniform_, normal_, etc.) rely on per-rank RNG state manipulation to produce correct distributed results. This was done via the _distribute_region context manager, which is not torch.compile friendly The new HOP takes pre-computed integer offsets (derived from the DTensorSpec at trace time
[RFC] Add run_dtensor_rng_op HOP to make DTensor RNG traceable
TLDR
Summary: Fixes: #147757 , #156649 This PR introduces a new higher-order operator ( run_dtensor_rng_op ) that makes DTensor random operations traceable by torch.compile . Previously, DTensor random...
According to PyTorch Releases: Summary: Fixes: #147757 , #156649 This PR introduces a new higher-order operator ( run_dtensor_rng_op ) that makes DTensor random operations traceable by torch.compile . Previously, DTensor random ops (dropout, uniform_, normal_, etc.) rely on per-rank RNG state manipulation to produce correct distributed results. This was done via the _distribute_region context manager, which is not torch.compile friendly The new HOP takes pre-computed integer offsets (derived from the DTensorSpec at trace time