GE-ViP: Graph-enhanced Vision Pipeline for Weakly Supervised Histopathology Slide Analysis

✅ Published in Neurocomputing 679 (2026) 133230 (Elsevier, Q1)

📄 Paper: ScienceDirect
DOI: 10.1016/j.neucom.2026.133230
Received: 26 November 2025 · Revised: 28 January 2026 · Accepted: 4 March 2026

Abstract

Whole Slide Images (WSIs) provide rich structural detail for cancer diagnosis, but their gigapixel scale and the lack of pixel-level annotations make automated analysis challenging. Weakly supervised learning enables slide-level training without exhaustive manual labels. However, many convolutional neural network (CNN) and multiple instance learning (MIL)-based pipelines treat patches independently, overlooking the spatial structure, while existing graph-based methods often rely on large, static graphs that are difficult to scale and sensitive to noise. We introduce GE-ViP, a lightweight Graph-Enhanced Vision Pipeline designed to overcome these limitations through a carefully integrated workflow. GE-ViP first selects diagnostically relevant tissue regions using a combination of SAM-derived tissue masks, entropy-based texture measures, and staining-related cues, effectively removing background regions and low-information patches. A Vision Transformer then encodes the selected patches to capture both fine-grained cellular features and broader tissue patterns. Next, patches are connected using a compact TriFusion-Planar graph that integrates spatial proximity, visual similarity, and semantic relationships, yielding a coherent representation of tissue organization. An edge-conditioned Graph Neural Network (GNN) processes this graph to produce slide-level predictions under weak supervision. Unlike prior pipelines that treat patch selection, feature extraction, and graph construction as independent stages, GE-ViP unifies these components into a single framework, improving efficiency, robustness, and interpretability, supported by feature-level explanations via GraphLIME.

Introduction

Histopathological analysis is a key step in cancer diagnosis, where pathologists visually inspect thin tissue slides under a microscope. Whole-slide imaging (WSI) digitizes glass slides into gigapixel images (~8×10⁴ to 2×10⁵ pixels per side), enabling efficient storage, remote access, and computational analysis. However, constructing reliable ML models requires dense patch-level annotations, which are costly and impractical at WSI scale. While this challenge is commonly addressed using MIL under slide-level supervision, existing pipelines often treat patch selection, feature extraction, and contextual aggregation as separate stages.

Key limitations of prior work:

• CNN+MIL (ABMIL, CLAM): treat patches as orderless set, losing spatial context; attention weights can be noisy
• Transformer-based MIL (TransMIL, HIPT): improve long-range reasoning but suffer from quadratic cost of self-attention on large WSIs
• Graph-based MIL (Patch-GCN, WiKG): introduce relational modeling but rely on fixed/heuristic adjacency, sensitive to tissue heterogeneity
• Post-hoc explainability (GNNExplainer, PGExplainer): computationally expensive and unstable on large heterogeneous WSI graphs

GE-ViP addresses all these limitations through a unified, lightweight pipeline with built-in interpretability.

🎯 Key Contributions

• Efficient weakly supervised pipeline — Semantic-aware patch selection using SAM masks, entropy, and saturation cues retains only 0.5k–1.5k patches per WSI (vs. 200k–300k for dense tiling), reducing preprocessing time to 10–20 min/WSI and memory to <0.2 GB.
• TriFusion-Planar graph with edge-aware GNN — A compact k-NN graph fusing: (1) planar Delaunay spatial neighbors, (2) spatial k-NN in Euclidean space, (3) appearance-based mutual k-NN, (4) semantic-based mutual k-NN. Edge-conditioned GNN with 17-dimensional edge attributes learns tissue structure from slide-level labels only.
• Multi-level interpretability — GraphLIME for node feature importance, slide-level saliency (node dropout effect), patch-level ViT rollout attention maps, subgraph visualization, and t-SNE for embedding analysis.
• Resource efficiency — 1.44M parameters, 5s per epoch (single NVIDIA RTX 4090) — the smallest and fastest among graph-based WSI methods.

Methodology

Pipeline Overview

Figure 2: A WSI is divided into patches and filtered using entropy, saturation, and SAM-based tissue masks to retain informative regions. Selected patches are encoded using a shared Vision Transformer to extract multi-scale visual features, which are combined with patch metadata to form graph nodes. GE-ViP constructs a tissue graph by fusing spatial, geometric, embedding-based, and semantic neighborhood relationships, followed by edge pruning to retain informative connections. A GNN performs message passing to produce slide-level diagnostic predictions.

Stage 1: Patch Selection

A WSI I ∈ R^{H×W×C} is divided into non-overlapping 256×256 patches at 20× magnification (~70–100k patches/slide). Three-stage filtering:

• (a) Entropy filtering: patch entropy φ^(ent) ≥ τ_ent (threshold 4.0); ensures textural richness
• (b) Saturation filtering: mean HSV saturation φ^(sat) ≥ τ_sat (threshold 0.05); eliminates poorly stained regions

• (c) Semantic filtering: SAM tissue coverage score σᵢ =

  Mᵢ

  /

  pᵢ

  ≥ τ_sem (threshold 0.5)

Final selection rule: \(p_i \in \mathcal{P}^* \iff \phi_i^{(\text{ent})} \geq \tau_{\text{ent}} \wedge \phi_i^{(\text{sat})} \geq \tau_{\text{sat}} \wedge \sigma_i \geq \tau_{\text{sem}}\)

Patches are categorized into: (i) low-information benign, (ii) high-information benign, (iii) high-information suspected cancer.

Stage 2: Multi-Scale Feature Extraction

Each selected patch is processed at two spatial scales with shared ViT-Tiny/16 (224) encoder Φ:

• Standard view: Resize(224,224) → f^(1) ∈ R^192
• Context view: Resize(384,384) → CenterCrop(224,224) → f^(2) ∈ R^192

Multi-scale embedding: f_i = [f^(1)_i ‖ f^(2)_i] ∈ R^384

Semantic descriptor: m_i = [aᵢ, ρᵢ, σᵢ] ∈ R^3 (mask area, local density, semantic score)

Node feature: x_i = [f_i ‖ m_i] ∈ R^387

Stage 3: TriFusion-Planar Graph Construction

Four complementary neighborhoods are fused into slide graph G_S = (V, E):

1. Planar Delaunay N^del(i) — spatial adjacency via Delaunay triangulation; bounded average degree; density-independent → geometric stability backbone
2. Spatial k-NN N^xy(i) — Euclidean distance in (x,y) coordinate space
3. Appearance mutual k-NN N^emb(i) — cosine similarity of visual embeddings fᵢ
4. Semantic mutual k-NN N^sem(i) — similarity of tissue descriptors mᵢ

Union: U(i) = N^del ∪ N^xy ∪ N^emb ∪ N^sem, then symmetrized and degree-regularized.

Figure 3: Illustration of TriFusion-Planar graph construction. From left: (a) Delaunay triangulation over patch coordinates forms the geometric backbone; (b) spatial k-NN connects nearby patches in coordinate space; (c) appearance mutual k-NN connects visually similar patches; (d) semantic mutual k-NN connects patches of the same tissue prototype. All four neighborhoods are fused and pruned using the weighted edge scoring function.

Score-based edge pruning: \(s_{ij} = 0.45\, d_{\text{feat}}(i,j) + 0.30\, d_{\text{spat}}(i,j) + 0.25\, d_{\text{sem}}(i,j)\)

Each retained edge carries a 17-dimensional attribute vector g_ij encoding: appearance dissimilarity d^emb, spatial distance r, mask-area difference Δa, orientation (sin θ, cos θ), semantic dissimilarity d^sem, density difference Δρ, 6 RBF encodings of spatial distance, and 4 binary neighborhood source flags (1^del, 1^xy, 1^emb, 1^sem). During training, projected to compact 5-D [z(d^emb), z(r), z(Δa), sin θ, cos θ].

Stage 4: Edge-Conditioned GNN Training

Figure 4: Edge-conditioned GNN architecture. Each layer applies residual GINE message passing conditioned on 5-D edge attributes. After 4 layers, a multi-head readout (Set2Set + attention aggregation + mean pooling) produces a fixed-length slide embedding, which is classified by a temperature-scaled cosine head.

Architecture:

• 4-layer residual edge-conditioned GINE with DropPath and DropFeature
• Hidden dimension: 256
• Multi-head readout: [Set2Set ‖ AttnAgg ‖ Mean] ∈ R^{4d_hid}
• Temperature-scaled cosine classifier: ℓ = τ · (h/‖h‖) · W_c^T

Training setup:

• 120 epochs, AdamW (lr=3×10⁻⁴, weight decay=6×10⁻²), cosine warmup, mixed precision, gradient clipping
• Robustness: node noise σ=0.006, DropEdge 0.18, DropNode 0.14, DropFeature 0.18
• Loss: LDAM-DRW (class imbalance) + R-Drop λ=0.07 (consistency) + Manifold Mixup α=0.35
• EMA weight averaging (μ=0.999)
• Evaluation: 12 test-time augmentation views (node dropout p=0.084, edge dropout p=0.108)
• Early stopping: macro-F1 on validation, patience 30 epochs
• Data splits: 70:30 patient-level stratified + 10-fold patient-level cross-validation

📊 Results

Table 2: TCGA Cohort Characteristics

Table 3: Cancer Type Classification (mean ± std, 10-fold CV)

Bold = best performance; underline = second best.

GE-ViP achieves the highest Accuracy, AUC, and F1-score across all three TCGA cohorts for cancer type classification, surpassing the best prior graph method (WiKG) by +1.76% Acc on ESCA, +0.48% on KIDNEY, +3.53% on LUNG.

Table 4: Cancer Stage Prediction (mean ± std, 10-fold CV)

Staging is inherently harder (subtle visual differences between stages I–IV). GE-ViP achieves the best Accuracy and AUC on ESCA (+5.24% Acc over WiKG) and LUNG (+2.03% AUC over WiKG), with consistent F1 improvements. GE-ViP shows particularly notable gains on the challenging staging problem where most existing methods struggle.

Table 5: Cross-Domain Evaluation — Camelyon-17 & TCGA-COAD

Camelyon-17 (300 WSIs, lymph node breast metastasis): GE-ViP achieves 81.84% Acc and 94.70 AUC despite domain shift (different tissue type, staining, acquisition). TCGA-COAD (465 colon adenocarcinoma WSIs, 4 histological categories): 87.11% Acc with the lowest variance (±0.15).

📊 Ablation Study

Table 6: Patch Extraction Comparison

GE-ViP reduces patch count by >100× vs. dense tiling while maintaining low noise induction through semantic awareness. 10–20 min preprocessing vs. 100–180 min for dense tiling.

Table 7: Statistical Significance — TriFusion vs k-NN (p < 0.05 in bold)

Table 8: Feature Extractor Ablation (AUC %)

ViT-Tiny/16 (224) delivers the strongest overall results across all tasks. ViT-based features consistently outperform CNN variants; ViT-Tiny outperforms even the pathology foundation model UNI, particularly for stage prediction — highlighting the benefit of task-adaptive feature representations integrated with graph-based reasoning.

Table 9: Edge Construction Strategy Ablation (AUC %)

TriFusion-Planar consistently outperforms both single-criterion k-NN baselines across all datasets and tasks. The most significant gains are on staging (+7.20% AUC on ESCA staging vs cosine k-NN).

Memory Efficiency Comparison

GE-ViP achieves the fastest training time (5s/epoch) and smallest parameter count (1.44M) among all graph-based WSI models — 14× fewer parameters than Patch-GCN.

Model Interpretation

GE-ViP provides multi-level interpretability:

Figure 5: Patch-level interpretability via ViT attention rollout. Top-K (K=12) patches selected by semantic score are visualized with attention heatmaps. Highlighted regions correspond to clinically meaningful structures: tumor-stroma boundaries, atypical nuclei clusters, glandular formations, stromal textures, and necrotic rims.

Figure 6: Slide-level saliency maps and subgraph visualization. Left: WSI thumbnail with patches shown as colored points scaled by node importance (confidence drop when zeroed out) — salient nodes cluster over tumor nests, invasive fronts, and gland chains. Right: Top-ranked edges drawn as blue connections, with thick bundles tracing coherent architectural motifs (tumor boundaries, dense stromal regions).

Figure 7: t-SNE visualization of learned slide embeddings. Classes form compact, well-separated clusters. Slides at cluster centers exhibit clear, canonical cancer morphology; boundary slides display ambiguous tissue patterns (useful for clinical triage). GraphLIME feature importance (inset) shows boundary/structure-sensitive ViT channels and texture/nuclear descriptors rank highest; color-only features contribute less.

• Patch-level (ViT rollout): Attention rollout on top-K (K=12) patches by semantic score → highlights tumor-stroma boundaries, atypical nuclei clusters, stromal textures, necrotic rims — clinically meaningful regions
• Slide-level saliency: Each patch shown as a point on WSI thumbnail with color/size encoding node importance (drop in argmax confidence when zeroed) → salient nodes cluster over tumor nests, invasive fronts, gland chains, stromal reaction
• Subgraph view: Top-ranked edges (by endpoint node saliency sum) displayed as blue connections → thick edge bundles trace coherent architectural motifs
• GraphLIME feature importance: Boundary/structure-sensitive ViT channels and texture/nuclear descriptors rank highest; color-only features contribute less (likely due to stain normalization independence)
• t-SNE embeddings: Slides deep inside same-class clusters show clear representative cancer morphology; boundary slides display ambiguous tissue patterns → clinical triage utility

📚 Citation

@article{rahman2026gevip,
  title={GE-ViP: Graph-enhanced vision pipeline for weakly supervised histopathology slide analysis},
  author={Rahman, Maisha and Ahad, Jawad Ibn and Hassan, Md. Mehedi and Moursalin, Golam and Momen, Sifat},
  journal={Neurocomputing},
  volume={679},
  pages={133230},
  year={2026},
  publisher={Elsevier},
  doi={10.1016/j.neucom.2026.133230}
}