Publications

MRRI: Memory Retention Mechanism for Robust and Interpretable Time Series Forecasting

Published in Submitted, 2024

Time series forecasting, due to its wide range of applications, has been a hot research topic in recent decades. However, existing studies are mostly dedicated to orchestrating more accurate or efficient deep learning models, while no study has yet focused on exploring the model’s internal mechanism. To address this open issue and inspired by the forgetting mechanism of human brain, in this paper we propose Memory Retention mechanism for Robust and Interpretable time series forecasting, denoted as \textbf{MRRI}. Specifically, we incorporate the forgetting curve function into our designed memory retention mechanism, enabling the model to forget information in an exponential form to explain how itself works. Through this mechanism, MRRI can facilitate the elimination of irrelevant information and enhance the model robustness against noise. Subsequently, we design a memory recall module to retrieve and identify important memories/information for the model interpretability. To further enhance its nonlinear expressiveness, we employ learnable nonlinear activations to learn the nonlinearity of time series data and give theoretical analysis to prove its convergence. Experimental results on nine real-world datasets demonstrate that MRRI achieves state-of-the-art (SOTA) performance, and it degrades less than compared baselines after noise injection, exhibiting substantial robustness. The nature-inspired forgetting mechanism provides a promising idea for building highly interpretable and robust deep learning models.

Recommended citation: Li, X., Zheng, L., Shang, J., Liao, L., & Zhang, J. (2024). MRRI: Memory Retention Mechanism for Robust and Interpretable Time Series Forecasting.

DUVET: Dual View Enhanced Transformer for Multivariate Flight Time series Anomaly Detection

Published in Submitted, 2024

In recent years, the prevalence of modern flight data recording systems such as Quick Access Recorder (QAR), has made it viable to improve flight safety by analyzing large volumes of flight parameters. However, existing studies mostly concentrate on specific safety events and heavily depend on expert-labeled data where the labels are highly scarce and imbalanced. To fully utilize valuable information from the tremendous unlabeled data, in this paper we focus on uncovering abnormal flight patterns and define it as an self-supervised time series anomaly detection problem, enabling us to discover potential and unexpected safety risks. To this end, we propose DUVET, a reconstructionbased Dual View Enhanced Transformer model for multivariate flight time series anomaly detection. Specifically, DUVET adopts Patching and Channel Independence techniques to reduce model parameters and improve Transformer’s computational efficiency of attention mechanism while preserving its semantic representation capacity. Based on the observation that flight safety usually relates to both long-term patterns and short-term deviations, we pioneer a new Bell Attention mechanism, and combine it with Transformer’s self-attention, to capture local and global dependencies respectively, which largely enhances the model reconstruction ability. We first evaluate DUVET on six public real-world datasets and one QAR dataset, demonstrating that DUVET achieves state-of-the-art performance in general time series anomaly detection task, and the ablation study clearly shows the vital role of Bell Attention. Further experiments on the QAR dataset show that DUVET meets key criteria of anomaly detection for flight safety in terms of stability, training efficiency and detection accuracy. Case study shows that DUVET can effectively uncover hidden flight safety risks.

Recommended citation: Li, C., Li, X., Chen, H., Zheng, L., Sun, H., Shang, J. (2024). DUVET: Dual View Enhanced Transformer for MultivariateFlight Time series Anomaly Detection.

ATPF: An Adaptive Temporal Perturbation Framework for Adversarial Attacks on Temporal Knowledge Graph

Published in IEEE Transactions on Knowledge and Data Engineering, 2024

Robustness is paramount for ensuring the reliability of knowledge graph models in safety-sensitive applications. While recent research has delved into adversarial attacks on static knowledge graph models, the exploration of more practical temporal knowledge graphs has been largely overlooked. To fill this gap, we present the Adaptive Temporal Perturbation Framework (ATPF), a novel adversarial attack framework aimed at probing the robustness of temporal knowledge graph (TKG) models. The general idea of ATPF is to inject perturbations into the victim model input to undermine the prediction. Firstly, we propose the Temporal Perturbation Prioritization (TPP) algorithm, which identifies the optimal time sequence for perturbation injection before initiating attacks. Subsequently, we design the Rank-Based Edge Manipulation (RBEM) algorithm, enabling the generation of both edge addition and removal perturbations under black-box setting. With ATPF, we present two adversarial attack methods: the stringent ATPF-hard and the more lenient ATPF-soft, each imposing different perturbation constraints. Our experimental evaluations on the link prediction task for TKGs demonstrate the superior attack performance of our methods compared to baseline methods. Furthermore, we find that strategically placing a single perturbation often suffices to successfully compromise a target link.

Recommended citation: Liao, L., Zheng, L., Shang, J., Li, X., & Chen, F. (2024). ATPF: An Adaptive Temporal Perturbation Framework for Adversarial Attacks on Temporal Knowledge Graph. IEEE Transactions on Knowledge and Data Engineering.
Download Paper

DGAN: Flight sensor data anomaly detection based on dual-view graph attention network

Published in Submitted, 2024

A large amount of flight data is created by various sensors during flight, including pitch, roll, etc. These data often contain anomalies, reflecting potential factors that could lead to unsafe events. Thus, flight data anomaly detection is crucial for aviation safety, enabling the identification of deviations from normal operations. However, existing methods often fail to adequately extract temporal correlations and variable dependencies due to the complexity and high dimensionality of the flight data, resulting in inferior detection performance. This issue is particularly apparent when the anomaly data are scarce. To address these issues, we propose a novel anomaly detection model, named DGAN. Specifically, DGAN employs two parallel graph attention (GAT) layers to create embeddings that effectively capture the complex relationships among temporal correlations and variable dependencies. Additionally, it performs data augmentation on these embeddings to extract two views consistent with the underlying data patterns, thereby revealing broader data characteristics in the original spatial context. Based on these two views, DGAN constructs pairs of positive and negative samples and applies a dual-view contrastive method. This approach enables the model to capture local invariant features, ensuring effective anomaly detection even with scarce anomaly data. We conduct experiments on a dataset comprising 44,808 real-world flight samples from the Airbus A321 aircraft. The evaluation results suggest that the performance of DGAN exceeds state-of-the-art (SOTA) models. Moreover, we examine the effectiveness of DGAN through further visualizations and case studies.

Recommended citation: Yan, W., Li, X., Zheng, L., Shang, J., Wu, L., Lu, J., et al. (2024). DGAN: Flight sensor data anomaly detection based on dual-view graph attention network.

MDGNN: Multiple Flight Safety Incidents Prediction Model Based on Dynamic Graph Neural Networks

Published in Submitted, 2024

Flight safety incidents, such as hard landings and tail strike risks, represent critical concerns during the landing phase. Although Quick Access Recorder (QAR) systems collect extensive multivariate flight data, previous studies have faced challenges in effectively modeling the complex interdependencies between flight parameters, which has limited their ability to predict multiple safety incidents simultaneously. To address this issue, we propose a novel model, named MDGNN, to capture hidden spatio-temporal dependencies and predict both hard landing and tail strike risk incidents. Specifically, we employ temporal convolutional networks (TCNs) to extract both localized representations and long-term temporal trends from multivariate flight data, ensuring the standardization of flight parameters across varying frequencies. Additionally, we are the first to construct a dynamic graph to model temporal relationships, applying a dynamic graph neural network and a temporal convolution module to accurately capture intricate spatial and temporal dependencies. Extensive experiments conducted on 37,904 Airbus A320 flight samples demonstrate that the MDGNN model surpasses state-of-the-art baselines with high prediction accuracy. Furthermore, a case study visualizing key flight parameters highlights the model’s ability to reveal the root causes of safety exceedances, offering valuable insights for flight safety analysis.

Recommended citation: Song, L., Li, X., Liu, H., Wu, L., Sun, H., Zheng, L., Shang, J.(2024). MDGNN: Multiple Flight Safety Incidents Prediction Model Based on Dynamic Graph Neural Networks.

Decomposition with feature attention and graph convolution network for traffic forecasting

Published in Knowledge-Based Systems, 2024

Traffic forecasting is a crucial task for enhancing the quality and efficiency of intelligent transportation systems. In recent years, several neural networks have been proposed to tackle this challenge. However, traffic data itself has complex composition and intertwined temporal patterns. Currently, the absence of effective methods to capture the inherent properties and overall profile of traffic data has become a bottleneck in improving the traffic prediction capabilities of models. To address these issues, we introduce a novel model, Decomposition with Feature Attention and Graph Convolution Network (DFAGCN), for Traffic Forecasting. To highlight the inherent properties of complicated traffic data, we present a unique Decomposition with Feature Attention (DFA) method to decompose the traffic data into seasonality, trend, and remainder. On the one hand, we introduce a new method to filter out noise and possible outliers from time series data. On the other hand, we employ an attention mechanism to make our decomposition method adaptively extract the trend. Subsequently, we utilize spatial–temporal attention to simulate the dynamic spatial–temporal correlations of traffic data. Finally, we use spatial–temporal convolution to extract local dependencies of traffic data. Experiments conducted on four real-world traffic forecasting datasets demonstrate that DFAGCN achieves state-of-the-art performance. The results show that the application of time series decomposition to traffic data significantly enhances the accuracy of traffic prediction.

Recommended citation: Liu, Y., Wu, X., Tang, Y., Li, X., Sun, D., & Zheng, L. (2024). Decomposition with feature attention and graph convolution network for traffic forecasting. Knowledge-Based Systems, 112193.
Download Paper

IMTCN: An Interpretable Flight Safety Analysis and Prediction Model Based on Multi-Scale Temporal Convolutional Networks

Published in IEEE Transactions on Intelligent Transportation Systems, 2023

Flight safety is a key issue in the aviation industry. Recently, with the prevalence of flight data recording systems, some deep learning-based studies have been devoted to predicting safety incidents based on flight data. However, these studies, although they exhibit higher prediction accuracy, have largely neglected the interpretability analysis of safety incidents which is of great concern to airlines and pilots. To address this issue, we define flight safety prediction as a multiscale time series classification problem and propose an interpretable model named IMTCN to provide both accurate predictions and high interpretability of flight safety. First, multiple temporal convolutional networks (TCNs) are utilized to capture local representations and long effective histories from multivariate flight data. Because different flight parameters are collected with diverse sampling frequencies, multiple TCNs are used to handle these parameters separately. Then, we creatively adapt the class activation mapping (CAM) method, which has been used for interpretation in image classification, and combine it with the TCN to provide flight data interpretability. The established model can pinpoint key flight parameters and corresponding moments that contribute most to safety incidents. Experimental results on a real-world dataset with 37,943 Airbus A320 aircraft flights show that our model outperforms the baselines on the task of exceedance classification and prediction 2 seconds and 4 seconds in advance, and case studies demonstrate its superb interpretability for flight safety analysis.

Recommended citation: Li, X., Shang, J., Zheng, L., Wang, Q., Liu, D., Liu, X., ... & Sun, H. (2024). IMTCN: An Interpretable Flight Safety Analysis and Prediction Model Based on Multi-Scale Temporal Convolutional Networks. IEEE Transactions on Intelligent Transportation Systems.
Download Paper

SDTAN: Scalable Deep Time-Aware Attention Network for Interpretable Hard Landing Prediction

Published in IEEE Transactions on Intelligent Transportation Systems, 2023

Hard landing, as one of the most frequent flight safety incidents during the landing stage, is highly concerned by the aviation industry. Recently, the popularization of Quick Access Recorder (QAR), a modern flight data recording system, has made it possible to collect large volume of flight parameters and incorporate state-of-the-art AI technologies to improve flight safety. However, due to the complex, multivariate, and highly specialized nature of QAR data, most existing studies either suffer from information loss caused by rough feature extraction methods, or rely solely on black-box models with no interpretations, making themselves difficult to achieve satisfactory performance in terms of prediction and explainability. To address this issue, we propose a novel attention-driven model named SDTAN (Scalable Deep Time-Aware Attention Network), which can accurately predict hard landing events and provide interpretable insights to help reveal the possible reasons leading to the events. Specifically, SDTAN fully captures information to learn the local representations of parameters, and leverages the time-interval attention mechanism to focus on the entire temporal pattern of flight over the relevant time intervals. It further re-encodes the representations of parameters in a global view and learns the global effect of parameters on the predicted output to uncover the ones which strongly indicate the flight safety status, enabling both high prediction accuracy and qualitative interpretability. We conduct experiments on real-world QAR datasets of 37,920 Airbus A320 flight samples. Experimental results demonstrate that SDTAN outperforms other state-of-the-art baselines and provides effective interpretability by visualizing the importance of parameters.

Recommended citation: Chen, H., Shang, J., Zheng, L., Li, X., Liu, X., Sun, H., ... & Yu, L. (2023). SDTAN: Scalable Deep Time-Aware Attention Network for Interpretable Hard Landing Prediction. IEEE Transactions on Intelligent Transportation Systems, 24(9), 10211-10223.
Download Paper

PathSAGE: Spatial Graph Attention Neural Networks with Random Path Sampling

Published in Neural Information Processing, 2021

Graph Convolutional Networks (GCNs) achieve great success in non-Euclidean structure data processing recently. In existing studies, deeper layers are used in CCNs to extract deeper features of Euclidean structure data. However, for non-Euclidean structure data, too deep GCNs will confront with problems like “neighbor explosion” and “over-smoothing”, it also cannot be applied to large datasets. To address these problems, we propose a model called PathSAGE, which can learn high-order topological information and improve the model’s performance by expanding the receptive field. The model randomly samples paths starting from the central node and aggregates them by Transformer encoder. PathSAGE has only one layer of structure to aggregate nodes which avoid those problems above. The results of evaluation shows that our model achieves comparable performance with the state-of-the-art models in inductive learning tasks.

Recommended citation: Ma, J., Li, J., Li, X., & Li, X. (2021). PathSAGE: Spatial Graph Attention Neural Networks with Random Path Sampling. In Neural Information Processing: 28th International Conference, ICONIP 2021, Sanur, Bali, Indonesia, December 8–12, 2021, Proceedings, Part II 28 (pp. 111-120). Springer International Publishing.
Download Paper

CurveCluster+: Curve Clustering for Hard Landing Pattern Recognition and Risk Evaluation Based on Flight Data

Published in IEEE Transactions on Intelligent Transportation Systems, 2021

Hard landing is a typical flight safety incident, and interpretability plays an important role in flight safety research. However, existing studies failed to provide good interpretability of the reasons for hard landing incidents and suffer from low prediction accuracy. To address the above problems, in this paper we propose CurveCluster+, a curve clustering method based on quick access recorder (QAR) data for hard landing risk evaluation. Specifically, we first conduct an in-depth analysis on hard landing flights by comparing key QAR parameter curves with the group behavior, based on which we establish a two-level hierarchical classification of hard landing incidents according to the hard landing patterns. Then we extract curve-level features from key QAR parameters through interpolation and resampling. After that we turn the classic K-means clustering into a semi-supervised algorithm by incorporating some expert experience and apply it on the curve-level features to automatically recognize the hard landing patterns. Finally, we propose a risk evaluation model based on the clustering results to discover high-risk flights from normal ones. We evaluate our method on a QAR dataset of 37,943 Airbus 320 aircraft flights. The results show that compared with other state-of-the-art data-driven methods, CurveCluster+ provides strong interpretability of hard landing incidents and exhibits good performance in recognizing hard landing patterns (the overall accuracy of our method reaches up to 92.99%). Moreover, it only requires a handful of hard landing samples to discover high-risk flights from tremendous normal landing flights, which is critical for flight safety warnings.

Recommended citation: Li, X., Shang, J., Zheng, L., Wang, Q., Sun, H., & Qi, L. (2021). Curvecluster+: Curve clustering for hard landing pattern recognition and risk evaluation based on flight data. IEEE Transactions on Intelligent Transportation Systems, 23(8), 12811-12821.
Download Paper

A Relation-Guided Attention Mechanism for Relational Triple Extraction

Published in 2021 International Joint Conference on Neural Networks (IJCNN), 2021

Relational triples are the essential parts of knowledge graphs, which can be usually found in natural language sentences. Relational triple extraction aims to extract all entity pairs with semantic relations from sentences. Recent studies on triple extraction focus on the triple overlap problem where multiple relational triples share single entities or entity pairs in a sentence. Besides, we find sentences may contain implicit relations, and it is challenging for most existing methods to extract implicit relational triples whose relations are implicit in the sentence. In this paper, we propose a relation-guided attention mechanism (RGAM) for relational triple extraction. Firstly, we extract subjects of all possible triples from the sentence, and then identify the corresponding objects under target relations with relation guidance. We utilize relations as prior knowledge instead of regarding relations as classification labels, and apply attention mechanism to obtain fine-grained relation representations, which guide extracted subjects to find the corresponding objects. Our approach (RGAM) can not only learn multiple dependencies in each triple, but also be suitable for extracting implicit relational triples and handling the overlapping triple problem. Extensive experiments show that our model achieves state-of-the-art performance on two public datasets NYT and WebNLG, which demonstrates the effectiveness of our approach.

Recommended citation: Yang, Y., Li, X., & Li, X. (2021, July). A relation-guided attention mechanism for relational triple extraction. In 2021 International Joint Conference on Neural Networks (IJCNN) (pp. 1-8). IEEE.
Download Paper

Active instance segmentation with fractional-order network and reinforcement learning

Published in The Visual Computer, 2021

In this paper, a novel model is proposed to segment image instance based on fractional-order chaotic synchronization network and reinforcement learning method. In the proposed model, fractional-order network is used for the preliminary image segmentation, which can obtain fine-grained information to provide a guiding strategy for the exploration of reinforcement learning; afterward, reinforcement learning method is committed to generate high-quality bounding contour curves for the object instances, which can combine the pixel features with local information in the image to improve the overall accuracy. Compared with other fractional-order models, the experimental results show that our proposed model achieves higher accuracy on the datasets of Pascal VOC2007 and Pascal VOC2012.

Recommended citation: Li, X., Wu, G., Zhou, S., Lin, X., & Li, X. (2022). Active instance segmentation with fractional-order network and reinforcement learning. The Visual Computer, 1-14.
Download Paper

A Deep learning Method for Landing Pitch Prediction based on Flight Data

Published in 2020 IEEE 2nd International Conference on Civil Aviation Safety and Information Technology (ICCASIT), 2021

With the development of the aviation industry, aircraft has increasingly become one of the most preferred longdistance transportation tools, and aviation safety incidents have attracted extensive attention. The key to dealing with aviation safety incidents is to accurately predict anomalies and potential hazards in advance and instruct pilots to perform corrective operations. As one of the safety incidents, tail strike may cause damage to the aircraft fuselage which may bring financial losses, or even threaten lives. However, there are few studies on tail strike in depth at present. In order to fill this gap, this paper mainly focuses on the tail strike risk, which is defined as the incident that the maximum pitch angle of the aircraft one second after and before touchdown exceeds a certain threshold. Specifically, we employ the LSTM model to make predictions of the maximum pitch angle with 22 parameters from QAR data. Extensive experiments based on a large-scale data show that the prediction model in this paper achieves the lowest MSE, MAE and the highest fitting coefficient R2-score, as compared to 9 traditional machine learning algorithms, which validates the effectiveness of our model in finding high risk flights.

Recommended citation: Chen, H., Shang, J., Zhao, X., Li, X., Zheng, L., & Chen, F. (2020, October). A deep learning method for landing pitch prediction based on flight data. In 2020 IEEE 2nd International Conference on Civil Aviation Safety and Information Technology (ICCASIT (pp. 199-204). IEEE.
Download Paper

CurveCluster: Automated Recognition of Hard Landing Patterns Based on QAR Curve Clustering

Published in 2019 IEEE Ubiquitous Intelligence & Computing (UIC), 2020

Flight safety is of vital importance to the aviation industry. As one of the most typical security events, hard landing is extremely concerned by airlines and related studies have received extensive attention in recent years. However, existing regression or risk based models either suffers from low prediction accuracy, or cannot provide good interpretability, making themselves impractical in real applications. To solve these problems, in this paper we propose CurveCluster: a curve clustering-based approach which is able to automatically recognize hard landing patterns from quick access recorder (QAR) data. We first provide a two-level hierarchical classification of hard landing events based on different hard landing patterns. Then we extract curve features from several key QAR parameters through interpolation and resampling. Finally, we apply K-means clustering algorithm on the curve features to automatically recognize the hard landing patterns. We test our approach on a dataset of 9,203 A320 flight QAR data samples and the overall recognition accuracy reaches up to 93.1%. Moreover, our results directly reflect the reasons of different types of hard landing events, which show strong interpretability.

Recommended citation: Li, X., Shang, J., Zheng, L., Liu, D., Qi, L., & Liu, L. (2019, August). CurveCluster: Automated recognition of hard landing patterns based on QAR curve clustering. In 2019 IEEE SmartWorld, Ubiquitous Intelligence & Computing, Advanced & Trusted Computing, Scalable Computing & Communications, Cloud & Big Data Computing, Internet of People and Smart City Innovation (SmartWorld/SCALCOM/UIC/ATC/CBDCom/IOP/SCI) (pp. 602-609). IEEE.
Download Paper

Xu Li

Publications