Advancing Legal Text Summarization with Deep Learning Architectures

Abstract

The computational requirements for swiftly and precisely comprehending lengthy legal documents pose a considerable barrier. The advancement of effective automated summarizing methods is essential for resolving these challenges. Extractive summarization, a widely utilized technique, emphasizes the selection of prominent sentences to abridge extensive publications. The intrinsic subjectivity of this work and the difficulties of obtaining contextual information inside extensive legal documents render it problematic, even for human specialists. To tackle these issues, we suggest an innovative method based on hierarchical transformers. Our concept is based on the stacked transformer encoder design of BERT. In light of the quadratic escalation in computing expense linked to the self-attention mechanism of transformer models as sequence length increases, which constrains their use with lengthy documents, we integrate the Longformer. The Longformer's attention mechanism, which scales linearly with sequence length, enables the analysis of texts with thousands of tokens or more. This mechanism substitutes conventional self-attention with a hybrid method that integrates task-specific global attention and local windowed attention. The suggested model was assessed using two recognized benchmark datasets for long-sequence transformers: BillSum and FIRE. Our experimental findings indicate that it surpasses leading approaches on both datasets. On the BillSum dataset, it attains Rouge-1, Rouge-2, and Rouge-L scores of 47.11, 33.02, and 42.19, respectively. In the FIRE dataset, the respective scores are 58.43, 44.31, and 41.54. These results highlight the enhanced efficacy of our suggested approach relative to current leading techniques.