A large part of achieving this goal requires us to understand the tradeoff between model-free (data-driven) and model-based methods.

• On one hand, it is feasible to rely entirely on model-free control methods for controlling complex stochastic systems, but a well-known issue with these methods is their intensive data consumption and training time.
• On the other hand, model-based control methods require little to almost no data or training, but can only be designed for simpler classes of stochastic systems, i.e., those whose behavior can be written as explicit equations.

We identify and use patterns to develop autonomous control/estimation algorithms that are more efficient in data consumption and computation time.

Related work(s):

• SooJean Han, Soon-Jo Chung, John C. Doyle, "Predictive Control of Linear Discrete-Time Markovian Jump Systems by Learning Recurrent Patterns." Automatica, 2023.

We study and extend reinforcement learning methods by learning patterns in vehicle traffic congestion control. Intuitively, spatial symmetries arise from the rectangular, grid-like structure of the network, while temporal symmetries arise from both the structure and from human routines, e.g., commute times. We develop an extension of episodic control that builds equivalence classes to group together patterns which are considered "similar" to each other, hereby saving memory consumption. Although the specific application of focus so far has been on traffic congestion control, our goal is to develop this framework to be able to be used in other similar network control applications which can be solved using reinforcement learning methods.

Representative work(s):

• SooJean Han, Soon-Jo Chung, Johanna Gustafson, "Congestion Control of Vehicle Traffic Networks by Learning Structural and Temporal Patterns." L4DC, 2023

The goal is to achieve more intelligent network control, including faster communication, more optimal resource allocation, etc. Our approach is motivated by the fact that patterns and simple structures have properties that can be mathematically analyzed. For example, closed-form expressions for quantities (e.g., average hitting time, mean minimum commute time) can be derived for random walks over specific graph topologies (e.g., tree, star, ring, etc.). Applications include the control of large-scale flow networks, where the goal is to maintain low congestion and high throughput. We are particularly interested in vehicle congestion control, air traffic management, and formation-flight of UAVs.

Some common methods of implementing memory for control include experience replay buffers (in reinforcement learning/RL), and neural episodic control. However, many methods build their memories using simple structures, e.g., hash tables. This means each new experience is stored as a new entry, potentially leading to large memory consumption. We study the patterns of a system, then develop smarter rules for determining what to store in memory and how to store them. By taking inspiration from the human brain, we design heterogeneous memory structures that encode information in a diversity of ways (e.g., semantic versus episodic memory). Applications include autonomous decision-making algorithms for robotic systems and other reinforcement learning problems.

Conventional architectures for distributed data-gathering tend to rely on the external sensors to transmit all their observations to the processor, including redundant information. This causes problems with high bandwidth consumption, congestion, latency, and outdated data. We take inspiration from the human nervous system to improve these algorithms: predictions are computed and fed back to the individual sensors. Each sensor now only needs to transmit the unknown part of their recent observations back to the processor. Efficient wireless communication and information-gathering strategies are relevant for applications where multiple rovers and aerial drones must perform environment surveillance, mapping, and target-tracking over the surface of unknown planet terrains.

Representative work(s):

• SooJean Han, "Optimizing Accuracy and Efficiency in Distributed Data Gathering Architectures with Feedback." ArXiv, 2022.