Architecture Lineage

This map is a conceptual learning guide, not a strict genealogy. Arrows mean "this architecture builds on, modifies, or is best understood after" the parent idea.

graph TD FCN["FCN<br/>Dense pixel prediction"] DeepLabV3Plus["DeepLabv3+<br/>ASPP + decoder refinement"] UNet["U-Net<br/>Encoder-decoder + skip connections"] UNet3D["3D U-Net<br/>Direct volumetric extension"] VNet["V-Net<br/>3D volumetric U-Net branch"] ResUNetStyle["Residual U-Net / ResUNet-style<br/>Residual blocks in U-Net stages"] R2UNet["R2U-Net<br/>Recurrent residual blocks"] MultiResUNet["MultiResUNet<br/>Multi-scale blocks + ResPaths"] UNetPP["U-Net++<br/>Nested dense skip pathways"] UNet3Plus["UNet 3+<br/>Full-scale skip connections"] AttUNet["Attention U-Net<br/>Attention-filtered skips"] U2Net["U²-Net<br/>Nested U-structure blocks"] nnUNet["nnU-Net<br/>Self-configuring pipeline"] SegResNet["SegResNet<br/>VAE regularization, MONAI backbone"] TransUNet["TransUNet<br/>CNN/U-Net + Transformer context"] SwinUnet["Swin-Unet<br/>U-shaped Swin Transformer"] UNETR["UNETR<br/>3D Transformer encoder"] SwinUNETR["Swin UNETR<br/>3D shifted-window encoder"] StarDist3D["StarDist-3D<br/>Star-convex polyhedra, instance seg."] cellpose["Cellpose<br/>Flow-field instance seg."] wnet3d["WNet3D<br/>Self-supervised pretraining"] MedSAM["MedSAM<br/>Promptable medical segmentation"] SAMMed2D["SAM-Med2D<br/>2D medical prompting"] SAMMed3D["SAM-Med3D<br/>Native 3D, single-point prompt"] SegVol["SegVol<br/>Text + spatial prompts"] MedSAM2["MedSAM2<br/>3D and video prompting"] FCN --> UNet FCN --> DeepLabV3Plus UNet --> UNet3D UNet --> VNet UNet --> ResUNetStyle UNet --> UNetPP UNet --> AttUNet UNet --> U2Net UNet --> nnUNet UNet --> TransUNet ResUNetStyle --> R2UNet ResUNetStyle --> MultiResUNet ResUNetStyle --> SegResNet UNetPP --> UNet3Plus TransUNet --> SwinUnet TransUNet --> UNETR UNETR --> SwinUNETR UNet3D --> StarDist3D StarDist3D --> cellpose cellpose --> wnet3d MedSAM --> SAMMed3D MedSAM --> SegVol

General SAM and SAM2 are useful learning context for the promptable branch, but they are not tracked as canonical architecture entries in data/architectures.yml. The graph edges above follow the metadata parent field.

Main Branches

Branch	Main Idea	Examples
Dense prediction	Convert classification CNNs into pixel-level predictors.	FCN
General CV context modules	Add multi-scale context and lightweight boundary refinement to dense prediction.	DeepLabv3+
U-Net family	Combine encoder context with decoder localization through skip connections.	U-Net, 3D U-Net, V-Net, Residual U-Net / ResUNet-style variants, R2U-Net, MultiResUNet, SegResNet, U-Net++, UNet 3+, Attention U-Net, U²-Net
Pipeline self-configuration	Improve the whole segmentation pipeline, not only the model block.	nnU-Net
Transformer hybrids	Add attention-based global context to segmentation architectures.	TransUNet, Swin-Unet, UNETR, Swin UNETR
Instance segmentation	Predict object candidates, shape parameters, flow fields, or self-supervised region assignments to separate individual objects.	StarDist-3D, Cellpose, WNet3D
Promptable foundation models	Use prompts and broad pretraining for medical segmentation workflows.	MedSAM, SAM-Med2D, SAM-Med3D, SegVol, MedSAM2

Reading Order

Read FCN first to understand dense prediction. Read DeepLabv3+ when you want general computer-vision context for atrous convolution, ASPP, multi-scale context, and boundary-refining decoders. Then read U-Net because many medical segmentation variants are easier to understand as modifications of its encoder-decoder shape. After that, split into 3D models such as 3D U-Net and V-Net, residual block variants such as Residual U-Net / ResUNet-style variants, R2U-Net, and MultiResUNet, volumetric residual variants such as SegResNet, skip-connection variants such as U-Net++, UNet 3+, and Attention U-Net, block-nesting variants such as U²-Net, pipeline methods such as nnU-Net, and Transformer/foundation-model branches. In the Transformer branch, read TransUNet for the CNN/Transformer hybrid bridge, Swin-Unet for shifted-window U-shaped 2D segmentation, UNETR for 3D Transformer encoding, and Swin UNETR for shifted-window Transformer encoding in 3D. For instance segmentation, read StarDist-3D after 3D U-Net to see how a 3D U-Net-style backbone can predict object-level star-convex polyhedra instead of semantic voxel labels. Then read Cellpose as the flow-field answer to the same instance-segmentation problem: it groups pixels by following predicted flows to object centres instead of assuming star-convex shapes. Read WNet3D next for the self-supervised 3D route: pretraining on unlabelled fluorescence volumes with NCut and reconstruction before optional fine-tuning. In the promptable branch, read MedSAM for the basic medical SAM-style interface, SAM-Med2D for 2D medical adaptation, and SAM-Med3D and SegVol for native volumetric promptable branches before MedSAM2 for 3D image and video-style prompting.