We build a class of algorithms for hybrid UAV traffic, with special focus on minimum-curvature routing (see video above), congestion relief, multimodal transportation, and path-planning in complex environments. Key applications include human transportation services (e.g., taxis, Ubers, flying ambulances) and remote package deliveries, where fast and efficient trajectory planning, robustness to nonstationary environments, and the ability to traverse harder-to-reach areas (e.g., mountains) are crucial.

We study and improve the tradeoff between model-free (data-driven) methods and model-based methods of stochastic control and estimation.

• Model-free control methods can generalize better to a variety of complex stochastic systems (e.g., via neural networks), at the cost of intensive data consumption and training time.
• Model-based control methods may require less data and training (due to being characterized by explicit models), at the cost of being reliant on simplifying assumptions.

For example, stochastic model-based controllers typically assume the random process can be modeled using Gaussian distributions, and Gaussian white noise models, such as linear quadratic Gaussian (LQG) control and Kalman filtering, have been shown to work well in practice for many applications. However, they cannot be used to model impulsive jump perturbations such as wind gusts, large proprioceptive errors, sudden path deviations due to obstacle collisions, etc. While model-free methods could be used to learn the jump dynamics as a black box, a more efficient process would be to try Poisson and Lévy processes as candidate models, and augment from there.

Motivated by this, we identify and use patterns to develop autonomous control and estimation algorithms that are more efficient in data consumption and computation time.

Representative work(s):

• SooJean Han, Soon-Jo Chung, John C. Doyle, "Predictive Control of Linear Discrete-Time Markovian Jump Systems by Learning Recurrent Patterns." Automatica, 2023.
• SooJean Han, Soon-Jo Chung, "Incremental Nonlinear Stability Analysis for Stochastic Systems Perturbed by Lévy Noise." International Journal of Robust and Nonlinear Control, 2022.

We study and extend reinforcement learning methods by learning patterns in vehicle traffic congestion control. Intuitively, spatial symmetries arise from the rectangular, grid-like structure of the network, while temporal symmetries arise from both the structure and from human routines, e.g., commute times. We develop an extension of episodic control that builds equivalence classes to group together patterns which are considered "similar" to each other, hereby saving memory consumption. Although the specific application of focus so far has been on traffic congestion control, our goal is to develop this framework to be able to be used in other similar network control applications which can be solved using reinforcement learning methods.

Representative work(s):

• SooJean Han, Soon-Jo Chung, Johanna Gustafson, "Congestion Control of Vehicle Traffic Networks by Learning Structural and Temporal Patterns." L4DC, 2023
• Mukul Chodhary, Kevin Octavian, SooJean Han, "Efficient Replay Memory Architectures in Multi-Agent Reinforcement Learning for Network Congestion Control." Under review ICML, Feb 2024.

The goal is to achieve more intelligent network control, including faster communication, more optimal resource allocation, etc. Our approach is motivated by the fact that patterns and simple structures have properties that can be mathematically analyzed. For example, closed-form expressions for quantities (e.g., average hitting time, mean minimum commute time) can be derived for random walks over specific graph topologies (e.g., tree, star, ring, etc.). Applications include the control of large-scale flow networks, where the goal is to maintain low congestion and high throughput. We are particularly interested in vehicle congestion control, air traffic management, and formation-flight of UAVs.

Some common methods of implementing memory for control include experience replay buffers (in reinforcement learning/RL), and neural episodic control. However, many methods build their memories using simple structures, e.g., hash tables. This means each new experience is stored as a new entry, potentially leading to large memory consumption. We study the patterns of a system, then develop smarter rules for determining what to store in memory and how to store them. By taking inspiration from the human brain, we design heterogeneous memory structures that encode information in a diversity of ways (e.g., semantic versus episodic memory). Applications include autonomous decision-making algorithms for robotic systems and other reinforcement learning problems.

Conventional architectures for distributed data-gathering tend to rely on the external sensors to transmit all their observations to the processor, including redundant information. This causes problems with high bandwidth consumption, congestion, latency, and outdated data. We take inspiration from the human nervous system to improve these algorithms: predictions are computed and fed back to the individual sensors. Each sensor now only needs to transmit the unknown part of their recent observations back to the processor. Efficient wireless communication and information-gathering strategies are relevant for applications where multiple rovers and aerial drones must perform environment surveillance, mapping, and target-tracking over the surface of unknown planet terrains.