PyTorch Bug: `resize_()` Corrupts Tensors On Storage Failures
Have you ever encountered a perplexing crash or unexpected behavior in your PyTorch applications, especially when dealing with data manipulation? You might be hitting a subtle but significant issue related to how PyTorch handles tensor storage resizing. Specifically, there's a critical bug where PyTorch's resize_() method can update tensor shape metadata even when the underlying storage resize fails, creating what we call "corrupted" or "Zombie" tensors. This isn't just an inconvenience; it can lead to serious data integrity problems and application instability, often manifesting as Segmentation Faults or cryptic RuntimeErrors.
Imagine you're working with data from external libraries, like a NumPy array, and you've shared its memory with a PyTorch tensor. This is a common and powerful pattern for efficiency! Now, what happens if you try to resize_() that PyTorch tensor to a new shape, but the original NumPy array's memory isn't actually resizable by PyTorch? Intuitively, you'd expect PyTorch to simply throw an error and leave your tensor in its original, safe state. However, the current behavior can be quite different and, frankly, dangerous. The tensor's metadata – its reported shape and stride – gets updated to the new, desired size even as the system correctly identifies that the storage cannot actually be resized. This leaves you with a tensor that thinks it's big and full of data, but in reality, its underlying storage remains empty (0 bytes). This mismatch is the root of the problem, leading to what we dub a "Zombie" tensor: it looks alive on the surface, but try to interact with it, and it will inevitably crash or produce unpredictable results. Understanding this intricate interaction between resize_() calls, non-resizable storage, and the tensor's internal metadata is crucial for anyone building robust PyTorch applications. We'll dive deep into this issue, explain its mechanics, provide a minimal reproduction, and discuss its broader implications for your development workflow.
Understanding the PyTorch Tensor Bug
At the heart of this issue is the often-used resize_() method in PyTorch, which is designed to change the shape and size of a tensor in-place. While resize_() is incredibly useful for dynamic memory management, it exhibits a critical flaw when paired with non-resizable storage. This situation commonly arises when a PyTorch tensor shares its underlying memory with an external buffer that PyTorch cannot directly reallocate, such as a NumPy array injected using set_(). When resize_() is invoked on such a tensor, PyTorch performs a series of checks. One of these checks determines if the tensor's storage is actually capable of being resized. If it's not, PyTorch correctly throws a RuntimeError, typically stating, "Trying to resize storage that is not resizable." This error message might initially lead you to believe that the operation has failed cleanly and your tensor remains untouched. However, this is where the subtlety and danger of the bug lie.
The core problem is that the resize_() operation is not exception-safe in this specific scenario. Before the storage resizability check even occurs and triggers the RuntimeError, the tensor's shape and stride metadata are already updated to reflect the new target size. This means that by the time the RuntimeError is raised and caught (or even if it's not caught and crashes the program), the tensor's internal representation is already in an inconsistent state. Its shape attribute will now report the large, intended size, but its storage() will still point to the original, empty (0 bytes), non-resizable buffer. This discrepancy between what the tensor reports its size to be and what its underlying storage actually holds creates the
