Integrating structural and semantic signals in text-attributed graphs with BiGTex

Abstract

Text-attributed graphs (TAGs) pose unique challenges for representation learning by requiring models to capture both the semantic richness of node-associated texts and the structural dependencies of the graph. While graph neural networks (GNNs) effectively model topological information, they are limited in handling unstructured textual data. Conversely, large language models (LLMs) excel in text understanding but lack awareness of graph structure. To address this gap, we propose BiGTex, a hybrid architecture built around a novel, modular component: the Graph-Text Fusion Unit. By stacking these units, BiGTex enables tight, bidirectional interaction between textual and structural representations, allowing text to guide tructural reasoning and graph topology to refine semantic interpretation within each layer. This stands in contrast to prior sequential or loosely coupled approaches. The model employs parameter-efficient fine-tuning (LoRA), preserving the efficiency of the pre-trained LLM. Comprehensive experiments on multiple TAG benchmarks demonstrate that BiGTex achieves state-of-the-art performance in node classification and effectively generalizes to link prediction. An ablation study confirms the critical role of bidirectional graph–text interaction in enhancing representational quality. Overall, this work contributes: (i) a novel hybrid architecture integrating pre-trained language models with GNNs via bidirectional graph–text fusion mechanisms; (ii) a mechanism that injects structural tokens into the LLM and refines representations through learnable graph–text fusion modules; and (iii) empirical evidence showing that BiGTex achieves consistently strong and often superior performance across multiple benchmark datasets compared to state-of-the-art GNN and LLM-enhanced baselines, highlighting its robustness and practical effectiveness in real-world graph-based tasks.