Dual-Level Attention for Multivariate Weather Forecasting: A Hybrid CNN–BiLSTM Architecture with Feature and Temporal Attention

Abstract

Short-term weather forecasting continues to be a difficult problem since it is nonlinear, non-stationary, and dependent on numerous interacting factors. Statistical models such as ARIMA and SVR can provide good and interpretable forecasts; however, their effectiveness in capturing the time-dependent behavior of high-resolution data is rather low. While deep learning models such as LSTM, GRU, and CNN-LSTM combination have greatly increased the performance of forecasting, they still treat all variables and history values equally. In this way, they may neglect the different importance of the meteorological attributes and the time component. For this reason, this paper proposes a Hybrid Attention mechanism that will be used in a CNN-LSTM model for multivariate hourly weather forecasting. This approach combines feature-level and temporal attention mechanisms in the framework of the CNN-LSTM architecture. The proposed approach will be evaluated on a five-attribute weather forecast using a dataset of temperature, humidity, wind speed, atmospheric pressure, and precipitation with a 24-hour history period and a 6-hour prediction period. The empirical results indicate that the proposed model attains MAE of 1.62, RMSE of 2.23, and MAPE of 5.97%, thus outperforming traditional machine learning methods as well as the current state-of-the-art architectures such as TimesNet, PatchTST, and Autoformer. The ablation studies have also revealed that the usage of both types of attention layers decreases the MAE by 14.74% relative to CNN–LSTM architecture alone. Moreover, the attention layer visualisation indicates valuable forecasting behaviour, where there is a high degree of recency effect when it comes to temperature and atmospheric pressure, and at the same time episodic patterns are revealed for humidity and precipitation forecasts.