PyTorch is a deep learning library built on Python and Torch (a Lua-based framework). It provides GPU acceleration, dynamic computation graphs, and an intuitive interface for deep learning researchers and developers. PyTorch follows a "define-by-run" approach, meaning that its computational graphs are constructed on the fly, allowing for better debugging and model customization.
Key Features of PyTorch
- PyTorch uses dynamic graphs, allowing flexibility in model execution and debugging.
- PyTorch provides an automatic differentiation engine that simplifies gradient computation for deep learning.
- PyTorch supports CUDA, allowing computations to be performed efficiently on GPUs.
How to Install PyTorch?
PyTorch can be installed on Windows, macOS, and Linux using pip for CPU (without GPU):
!pip install torch torchvision torchaudio
PyTorch Tensors
Tensors are the fundamental data structures in PyTorch, similar to NumPy arrays but with GPU acceleration capabilities. PyTorch tensors support automatic differentiation, making them suitable for deep learning tasks.
Python import torch # Creating a 1D tensor x = torch.tensor([1.0, 2.0, 3.0]) print('1D Tensor: \n', x) # Creating a 2D tensor y = torch.zeros((3, 3)) print('2D Tensor: \n', y) Output:
1D Tensor:
tensor([1., 2., 3.])
2D Tensor:
tensor([[0., 0., 0.],
[0., 0., 0.],
[0., 0., 0.]])
Operations on Tensors
Python a = torch.tensor([1.0, 2.0]) b = torch.tensor([3.0, 4.0]) # Element-wise addition print('Element Wise Addition of a & b: \n', a + b) # Matrix multiplication print('Matrix Multiplication of a & b: \n', torch.matmul(a.view(2, 1), b.view(1, 2))) Output:
Element Wise Addition of a & b:
tensor([4., 6.])
Matrix Multiplication of a & b:
tensor([[3., 4.],
[6., 8.]])
Reshaping and Transposing Tensors
Python import torch t = torch.tensor([[1, 2, 3, 4], [5, 6, 7, 8], [9, 10, 11, 12]]) # Reshaping print("Reshaping") print(t.reshape(6, 2)) # Resizing (deprecated, use reshape) print("\nResizing") print(t.view(2, 6)) # Transposing print("\nTransposing") print(t.transpose(0, 1)) Output:
Reshaping
tensor([[ 1, 2],
[ 3, 4],
[ 5, 6],
[ 7, 8],
[ 9, 10],
[11, 12]])
Resizing
tensor([[ 1, 2, 3, 4, 5, 6],
[ 7, 8, 9, 10, 11, 12]])
Transposing
tensor([[ 1, 5, 9],
[ 2, 6, 10],
[ 3, 7, 11],
[ 4, 8, 12]])
Autograd and Computational Graphs
The autograd module automates gradient calculation for backpropagation. This is crucia in training deep neural networks.
Python x = torch.tensor(2.0, requires_grad=True) y = x ** 2 y.backward() print(x.grad) #(dy/dx = 2x = 4 when x=2)
Output:
tensor(4.)
PyTorch dynamically creates a computational graph that tracks operations and gradients for backpropagation.
Building Neural Networks in PyTorch
In PyTorch, neural networks are built using the torch.nn module, where:
- nn.Linear(in_features, out_features) defines a fully connected (dense) layer.
- Activation functions like torch.relu, torch.sigmoid, or torch.softmax are applied between layers.
- forward() method defines how data moves through the network.
To build a neural network in PyTorch, we create a class that inherits from torch.nn.Module and defines its layers and forward pass.
Python class NeuralNetwork(nn.Module): def __init__(self): super(NeuralNetwork, self).__init__() self.fc1 = nn.Linear(10, 16) # First layer self.fc2 = nn.Linear(16, 8) # Second layer self.fc3 = nn.Linear(8, 1) # Output layer def forward(self, x): x = torch.relu(self.fc1(x)) x = torch.relu(self.fc2(x)) x = torch.sigmoid(self.fc3(x)) return x model = NeuralNetwork() print(model)
Output:
NeuralNetwork(
(fc1): Linear(in_features=10, out_features=16, bias=True)
(fc2): Linear(in_features=16, out_features=8, bias=True)
(fc3): Linear(in_features=8, out_features=1, bias=True)
)
Define Loss Function and Optimizer
Once we define our model, we need to specify:
- A loss function to measure the error.
- An optimizer to update the weights based on computed gradients.
We use nn.BCELoss() for binary cross-entropy loss and used optim.Adam() for Adam optimizer to combine the benefits of momentum and adaptive learning rates.
Python model = NeuralNetwork() criterion = nn.BCELoss() optimizer = torch.optim.Adam(model.parameters(), lr=0.01)
Train the Model
The training involves:
- Generating dummy data (100 samples, each with 10 features).
- Running a training loop where we:
- optimizer.zero_grad() clears the accumulated gradients from the previous step.
- Forward Pass (model(inputs)) passes inputs through the model to generate predictions.
- Loss Computation (criterion(outputs, targets)) computes the difference between predictions and actual labels.
- Backpropagation (loss.backward()) computes gradients for all weights.
- Optimizer Step (optimizer.step()) updates the weights based on the computed gradients.
Python inputs = torch.randn((100, 10)) targets = torch.randint(0, 2, (100, 1)).float() epochs = 20 for epoch in range(epochs): optimizer.zero_grad() # Reset gradients outputs = model(inputs) # Forward pass loss = criterion(outputs, targets) # Compute loss loss.backward() # Compute gradients optimizer.step() # Update weights if (epoch+1) % 5 == 0: print(f"Epoch [{epoch+1}/{epochs}], Loss: {loss.item():.4f}") Output:
Epoch [5/20], Loss: 0.7014
Epoch [10/20], Loss: 0.6906
Epoch [15/20], Loss: 0.6744
Epoch [20/20], Loss: 0.6483
PyTorch vs TensorFlow
| Feature | PyTorch | TensorFlow |
|---|
| Computational Graph | Dynamic | Static (TF 1.x), Dynamic (TF 2.0) |
| Ease of Use | Pythonic, easy to debug | Steeper learning curve |
| Performance | Fast with eager execution | Optimized for large-scale deployment |
| Deployment | TorchScript & ONNX | TensorFlow Serving & TensorFlow Lite |
| Popularity in Research | Widely used | Also widely used but more in production |
Applications of PyTorch
- Computer Vision: PyTorch is widely used in image classification, object detection, and segmentation using CNNs and Transformers (e.g., ViT).
- Natural Language Processing (NLP): PyTorch supports transformers, recurrent neural networks (RNNs), and LSTMs for applications like text generation and sentiment analysis.
- Reinforcement Learning: PyTorch is used in Deep Q-Networks (DQN), Policy Gradient Methods, and Actor-Critic Algorithms.
- Healthcare and Bioinformatics: Used in medical image analysis, drug discovery, and protein structure prediction.
PyTorch is used in academia and industry for computer vision, NLP, and reinforcement learning applications. With its strong community support and easy-to-use API, PyTorch continues to be one of the leading deep learning frameworks.