Siny, Xingjian Zhang

In the context of Industry 4.0, effective monitoring of multiple targets and states during assembly processes is crucial, particularly when constrained to using only visual sensors. Traditional methods often rely on either multiple sensor types or complex hardware setups to achieve high accuracy in monitoring, which can be cost-prohibitive and difficult to implement in dynamic industrial environments. This study presents a novel approach that leverages multiple machine learning models to achieve precise monitoring under the limitation of using a minimal number of visual sensors. By integrating state information from identical timestamps, our method detects and confirms the current stage of the assembly process with an average accuracy exceeding 92%. Furthermore, our approach surpasses conventional methods by offering enhanced error detection and visualization capabilities, providing real-time, actionable guidance to operators. This not only improves the accuracy and efficiency of assembly monitoring but also reduces dependency on expensive hardware solutions, making it a more practical choice for modern industrial applications.

In this work, we propose an attention-based deep reinforcement learning approach to address the adaptive informative path planning (IPP) problem in 3D space, where an aerial robot equipped with a downward-facing sensor must dynamically adjust its 3D position to balance sensing footprint and accuracy, and finally obtain a high-quality belief of an underlying field of interest over a given domain (e.g., presence of specific plants, hazardous gas, geological structures, etc.). In adaptive IPP tasks, the agent is tasked with maximizing information collected under time/distance constraints, continuously adapting its path based on newly acquired sensor data. To this end, we leverage attention mechanisms for their strong ability to capture global spatial dependencies across large action spaces, allowing the agent to learn an implicit estimation of environmental transitions. Our model builds a contextual belief representation over the entire domain, guiding sequential movement decisions that optimize both short- and long-term search objectives. Comparative evaluations against state-of-the-art planners demonstrate that our approach significantly reduces environmental uncertainty within constrained budgets, thus allowing the agent to effectively balance exploration and exploitation. We further show our model generalizes well to environments of varying sizes, highlighting its potential for many real-world applications.

This work introduces a multi-agent deep reinforcement learning (MARL) framework for persistent monitoring and informative path planning (IPP) of dynamically evolving targets using unmanned agents. Addressing the challenges inherent in real-time multi-target tracking within uncertain environments, we propose a novel spatio-temporal attention network that integrates Gaussian Processes (GPs) for modeling and predicting target distributions. The developed approach discretizes the monitoring environment into graph structures, enabling agents to leverage historical spatial and temporal information efficiently. Our MARL approach employs a Proximal Policy Optimization (PPO) algorithm within a Centralized Training Decentralized Execution (CTDE) paradigm, ensuring effective cooperation among agents while maintaining autonomy in decision-making during execution. We further introduce an adaptive reward mechanism based on uncertainty reduction, explicitly penalizing redundant observations to encourage diversified exploration and cooperative behavior. Experimental results demonstrate that our method significantly improves information gain and reduces uncertainty in dynamic, multi-target scenarios, showcasing superior collaboration capabilities among agents compared to baseline methods.