Classic GNN Architectures

1. GCN - Graph Convolutional Network

Paper: Kipf & Welling, Semi-Supervised Classification with GCN (2017)

Update rule

Derivation intuition

GCN is a first-order approximation of spectral graph convolution (ChebNet with $K=1$). The factor $\frac{1}{\sqrt{d_i d_j}}$ prevents high-degree nodes from dominating the sum.

Pros and cons

• Pros: Simple, fast, strong baseline on homophilous graphs (Cora, Citeseer)
• Cons: Fixed aggregation weights (no learning which neighbor matters), transductive by default, weak on heterophily

from torch_geometric.nn import GCNConv

class GCN(torch.nn.Module):
    def __init__(self, in_dim, hidden, out_dim):
        super().__init__()
        self.conv1 = GCNConv(in_dim, hidden)
        self.conv2 = GCNConv(hidden, out_dim)

    def forward(self, x, edge_index):
        x = self.conv1(x, edge_index).relu()
        x = F.dropout(x, 0.5, training=self.training)
        return self.conv2(x, edge_index)

2. GraphSAGE

Paper: Hamilton, Ying, Leskovec - Inductive Representation Learning on Large Graphs (2017)

Update rule

Aggregators

• Mean: $\frac{1}{|\mathcal{N}(i)|}\sum_{j} \mathbf{h}_j$
• Max: element-wise max over neighbors
• LSTM: run LSTM over random permutations of neighbors (not strictly permutation invariant but works)

When to use

Large graphs, inductive settings, production systems where new nodes arrive daily (Pinterest, Uber Eats recommendations).

from torch_geometric.nn import SAGEConv

class GraphSAGE(torch.nn.Module):
    def __init__(self, in_dim, hidden, out_dim):
        super().__init__()
        self.conv1 = SAGEConv(in_dim, hidden)
        self.conv2 = SAGEConv(hidden, out_dim)

    def forward(self, x, edge_index):
        x = self.conv1(x, edge_index).relu()
        return self.conv2(x, edge_index)

3. GAT - Graph Attention Network

Paper: Veličković et al. - Graph Attention Networks (2018)

Attention coefficient

Update

Multi-head attention

Run $K$ independent attention heads, concatenate (or average) outputs - same idea as Transformer multi-head.

When to use

When neighbors are heterogeneous in importance. Slightly more expensive than GCN but often more accurate.

from torch_geometric.nn import GATConv

class GAT(torch.nn.Module):
    def __init__(self, in_dim, hidden, out_dim, heads=4):
        super().__init__()
        self.conv1 = GATConv(in_dim, hidden, heads=heads, dropout=0.6)
        self.conv2 = GATConv(hidden * heads, out_dim, heads=1, concat=False)

    def forward(self, x, edge_index):
        x = F.elu(self.conv1(x, edge_index))
        return self.conv2(x, edge_index)

4. GIN - Graph Isomorphism Network

Paper: Xu et al. - How Powerful are Graph Neural Networks? (2019)

Update rule

Connection to Weisfeiler-Lehman (WL) test

The 1-WL test distinguishes graphs by repeatedly replacing each node's label with a hash of its own label + sorted neighbor labels. GIN with sum aggregation + injective MLP matches the power of 1-WL - the strongest expressiveness among standard MPNNs.

When to use

Graph classification (molecules, proteins), when structural discrimination matters. State-of-the-art on many graph-level benchmarks.

from torch_geometric.nn import GINConv

class GIN(torch.nn.Module):
    def __init__(self, in_dim, hidden, out_dim):
        super().__init__()
        nn1 = torch.nn.Sequential(torch.nn.Linear(in_dim, hidden), torch.nn.ReLU(),
                                  torch.nn.Linear(hidden, hidden))
        nn2 = torch.nn.Sequential(torch.nn.Linear(hidden, hidden), torch.nn.ReLU(),
                                  torch.nn.Linear(hidden, out_dim))
        self.conv1 = GINConv(nn1)
        self.conv2 = GINConv(nn2)

    def forward(self, x, edge_index):
        x = self.conv1(x, edge_index).relu()
        return self.conv2(x, edge_index)

5. MPNN

Paper: Gilmer et al. - Neural Message Passing for Quantum Chemistry (2017)

MPNN specializes message passing for molecules:

Edge features (bond type, distance) are first-class citizens. This design powers molecular property prediction - atoms are nodes, bonds are edges with rich features.

6. Comparison Table

7. Full Training Example

# Pick GAT for Cora  -  swap GATConv for GCNConv/SAGEConv/GINConv
model = GAT(dataset.num_features, 8, dataset.num_classes, heads=8)
optimizer = torch.optim.Adam(model.parameters(), lr=0.005, weight_decay=5e-4)

def train():
    model.train()
    optimizer.zero_grad()
    out = model(data.x, data.edge_index)
    loss = F.cross_entropy(out[data.train_mask], data.y[data.train_mask])
    loss.backward()
    optimizer.step()
    return loss.item()

@torch.no_grad()
def test():
    model.eval()
    out = model(data.x, data.edge_index)
    pred = out.argmax(dim=1)
    acc = (pred[data.test_mask] == data.y[data.test_mask]).float().mean()
    return acc.item()