Data Analytics
Massive Computational Data Analytics
iMSEL has developed a framework to extract flow feature from massive dynamic computational data based on proper orthogonal decomposition (POD) and feature mining techniques for targeted visualization and learning. Two key elements in the feature extraction procedure are described including (1) the POD to reduce data dimensions and to decouple the time-averaged and time-varying flow structures in high-fidelity Computational Fluid Dynamics (CFD) data within non-uniform grids; and (2) the feature mining, including clustering-based data mining, filtering, and aggregation to detect both mean and unsteady flow features as well as to identify the region of interest (ROI). The rationale and benefits of combining both elements for fast scalable feature extraction are discussed. Case studies of vortex extraction are undertaken to validate the present approach. The POD accurately captures the characteristic flow structures embedded in the data and provides useful insight into the underlying flow phenomena. The feature mining module in our tool is capable of identifying key features in the ROI (3-10% of the original data) for focused visualization, discovery, and learning.
Real-time Data Analytics
The ability to accurately identify the dependence of flight external loads (forces and moments) on flight condition, dynamic and control states using data from various sources (such as flight test, physics-based models of varying fidelity levels) and to rapidly predict them at test points yet to be flown is essential for developing high-performance, safe, aircrafts and aerospace vehicles with reduced cost and cycle. iMSEL has been developing an integrated software tool based on surrogate models for real-time analysis and prediction of combined static and dynamic external loads using non-proprietary physics-based models and existing flight-test data. Our preliminary study established the proof of concept that the technology can be used to accurately analyze flight test data (5% uncertainty in flight load prediction) for a variety of maneuvers at the real-time rate and to elucidate primary load-driving factors out of complex nonlinear flight load behaviors. The software will be integrated with USAF designated real-time control room environment to automate the entire flight data analysis work flow in AFTC environment.
Machine Learning and Computer Vision
Unsupervised Anomaly Detection and Segmentation of Industrial Product
Industrial visual inspection must identify defective products and pinpoint damaged regions, even when examples of defects are scarce. iMSEL developed USAGI (Unsupervised Segmentation and Anomaly Gradient Interpretation), a transformer-based approach that learns from normal visual features to detect and segment anomalies.
Methodology. A frozen, pretrained ViT-B/16 Feature Extractor captures multi-scale visual features from normal training images. The Equal Transformer aligns and combines these features, while the Memory Transformer learns a representative memory of defect-free patterns. A two-step training strategy balances learning from the current batch with reconstructing earlier samples, helping retain previously learned normal patterns.
During inspection, the Retrieval Transformer compares test-image features with the frozen normal memory and reconstructs the input. Feature distances form an Anomaly Gradient Interpretation (AGI) map that highlights potential defects. Combining this map with reconstruction differences produces a fine-grained binary mask, identifying both the location and shape of abnormal regions.
Results. USAGI achieved pixel-level AUROC scores of 98.2% on MVTec AD, 99.5% on VisA, 98.8% on Real-IAD, and 96.4% on MANTA. Region-overlap scores (AUPRO) ranged from 90.7% to 92.9%, with USAGI leading the compared methods across all three evaluation metrics on VisA. The examples below show anomaly maps, detailed defect masks, and overlays alongside ground truth. Naturally varying textures remain challenging, and the method assumes clean, normal training data.
Deep Learning-Assisted Traffic Decongestion Prediction
When a train blocks a railroad grade crossing, traffic can continue to build even after the train has passed. Predicting when the queue will clear is difficult because recorded congestion events are relatively rare. iMSEL developed a deep learning-assisted framework that combines visual measurements with a mathematical model to estimate decongestion time.
Vehicle detection and counting in traffic video.
Framework: video observations are converted into vehicle counts and train-passing time, then used to estimate decongestion time.
The system analyzes crossing video in two complementary ways. A deep neural network estimates the number of vehicles during normal traffic flow, while a motion-detection algorithm measures how long a train passes through the crossing. A regression model then relates these observations to the time required for traffic to clear. This approach makes use of readily observed traffic conditions when there are too few congestion events to train an end-to-end model reliably.
The team evaluated the framework using 30 congestion events recorded over 200 hours at one grade crossing, with video captured from different camera angles. Vehicle count during normal traffic and train-passing time were both important predictors of decongestion time. The resulting quantitative model enables rapid estimates and provides a basis for further development toward real-world crossing monitoring.
Example results from the crossing video and fitted decongestion-time model.
Paper: A deep learning-assisted mathematical model for decongestion time prediction at railroad grade crossings, Jiang, Z., Guo, F., Qian, Y., Wang, Y., and Pan, W. D., Neural Computing and Applications 34, 4715–4732 (2022).
Remote Sensing Segmentation
Satellite and aerial imagery can reveal how land is used, how cities grow, and where environmental change or disaster damage occurs. Turning these images into useful maps requires assigning a land-cover category to each pixel. Buildings, roads, water, and vegetation can share similar visual patterns, making appearance alone unreliable in complex scenes. iMSEL is developing a segmentation framework that combines image features with automatically generated scene descriptions, bringing language-derived context into the interpretation of remote sensing imagery.
The framework first generates a scene caption with a remote-sensing-adapted BLIP-2 model and encodes its meaning with GeoRSCLIP. A caption-aware graph transformer then connects image patches using both visual features and the caption’s semantic context, allowing related regions to exchange information beyond their immediate surroundings. Caption features are introduced again through cross-attention during decoding, while a lightweight refinement module improves boundaries and reduces patch artifacts. Captions are generated automatically, without manually written prompts, and can be cached as metadata for repeated analysis of static imagery.
Evaluation on three benchmarks achieved mean intersection over union (mIoU) of 54.6% on LoveDA, 84.5% on Vaihingen, and 87.6% on Potsdam, using the study’s data splits and five foreground classes for the two ISPRS datasets. The results were competitive with leading methods, with clearer recognition of small objects and complex class boundaries. Removing caption content from the same architecture lowered mIoU by 1.9–3.1 percentage points across the three datasets. With captions precomputed, single-GPU tests processed a 1024 × 1024 tile in 14.1 ms; generating a fresh caption for every tile increased end-to-end latency to 609.3 ms. Caption quality remains important: omitted scene details can weaken guidance for small or underrepresented classes.
Field-deployable Automated Target Recognition
The need for next-generation multi-int sensing and machine learning (ML) framework for threat target detection and characterization is widely recognized. Innovative solutions rooted in data science and computation based signal processing, including artificial intelligence (AI) and machine learning (ML) offer the promise to overcome many of these critical challenges. Recent advances of AI and ML demonstrated unprecedented potential for automated characterization, monitoring, and identification of objects/targets with distinct behavior from the surroundings. iMSEL has been developing an integrated fieldable ATR system (including both software & hardware) based on deep learning for deploying and testing classifier models obtained by Convolution Neural Network on a low-end computing platform (RaspberryPi 3). CNN-based ATR has been applied to the whole image including a single or multiple vehicles. The classifier is not only able to correctly infer the type of vehicle but detect the presence of a wheel. The automatic target classification within the region of interest (ROI) for video frames has also been developed and demonstrated. Although in a high-clutter environment (graffiti, shrubbery), our prototype ATR system can still fully recognize each visible segment of the passing truck, even with significant target overlap, and the online classification process only takes sub-second. This case study verifies that the optimized detector can produce reasonably fast and accurate recognition even on extremely resource-constrained systems.