Perception algorithms are ubiquitous in modern autonomy stacks, providing necessary environmental information to operate in the real world. Many of these algorithms depend on the visibility of keypoints, which must remain within the robot’s line-of-sight (LoS), for reliable operation. This paper tackles the challenge of maintaining LoS on such keypoints during robot movement. We propose a novel method that addresses these issues by ensuring applicability to various sensor footprints, adaptability to arbitrary nonlinear dynamics and constantly enforces LoS throughout the robot’s path. We show through our experiments that the proposed approach achieves significantly reduced LoS violation and runtime when compared to existing state-of-the-art methods in several representative and challenging scenarios.
The relative navigation scenario is inspired by the relative navigation challenges encountered in drone racing. In this scenario, the quadrotor must navigate through a predefined sequence of 10 gates while maintaining line-of-sight on 10 static targets in minimal time. The viewcone modeled as a symmetric 2-Norm cone. All of these plots are interactive so you can press play to view the animation, zoom in and out, pan, and rotate the view. Additionally to turn off a trace simply click on the trace in the legend.
Figure 1: CT-LoS Trajectory. Throughout this trajectroy the quadrotor maintains LoS on the keypoints while navigating through the gates, even performaning a flip manuever around frame 242 to do so.
Figure 2: DT-LoS Trajectory. Right at the beginning of the trajectory, the quadrotor imnmeditley pitches down in order to rapidly accelerate while sacrificing visibility on the keypoints in between discretization nodes. This is expected as this is a minimal time scenario and LoS is only enforced at nodes.
These plots animate the positional trajectory of the drone. To make the animation run faster, click the viewcone label in the legend (which will only show up once the animation is playing) to turn off that trace.
Figure 3: CT-LoS Camera View. Note that the full keypoint trajectories are kept within the LoS.
Figure 4: DT-LoS Camera View. As only at the nodes is the LoS enforced on keypoints, the quadrotor is able to move more agressively without regard for LoS violation. For example the agressive pitch down maneuver at the beginning of the trajectory.
These plots show the egocentric or camera view. The keypoint trajectories, resolved in the sensor frame, are shown in different colors. The points correspond to discrete nodes and the lines are the nonlinear propagation of the dynamics.
Figure 5: CT-LoS Conic Constraint View.
Figure 6: DT-LoS Conic Constraint View.
These plots visualize the LoS constraint and show the keypoint trajectories resolved in the sensor frame as well as there projections onto the XZ and YZ planes, with dots corresponding to nodes and lines to nonlinear propagation of dynamics.
Figure 7: CT-LoS SCvx Iterations
Figure 8: DT-LoS SCvx Iterations
These plots show the interations of Alg. 1 until convergence. The initial guess for position and velocity is created by taking a straight-line innterpolation between each gate and boundary condition. The initial guess for attitude is found by pointing the sensor towards the mean location of the keypoints. Note that both methods recieve the same initial guess.
The cinematography scenario is inspired by the applications in which a subject needs to be tracked and filmed. In this scenario, the quadrotor must remain within a minimum and maximum range from a single dynamics keypoint while keeping it within LoS. We assume the trajectory of the keypoint is known. The viewcone modeled as a non-symmetric Infinity-Norm cone.
Figure 9: CT-LoS Trajectory. The trajectory continuously maintains LoS on the dynamics keypoint while reamining within the minimum and maximum ranges.
Figure 10: DT-LoS Trajectory.This trajectory is similar to the CT-LoS towards the beginning but around frame 370, the drone flys vertically upwards, incurring large internodal LoS constraint violation, after penultimate nodes and effectively free-falls into the last node as this is a minimal fuel problem.
These plots animate the positional trajectory of the drone. To make the animation run faster, click the viewcone, minimum range, and maximum range labels in the legend (which will only show up once the animation is playing) to turn off that trace.
Figure 11: CT-LoS Camera View.
Figure 12: DT-LoS Camera View. At around frame 360, the keypoint trajectory is no longer visible in the camera view corresponding to when the drone is effectively free-falling towards the last node.
These plots show the egocentric or camera view. The keypoint trajectories, resolved in the sensor frame, are shown in different colors. The points correspond to discrete nodes and the lines are the nonlinear propagation of the dynamics.
Figure 13: CT-LoS Conic Constraint View.
Figure 14: DT-LoS Conic Constraint View. At around frame 360, the keypoint trajectory is no longer within the viewcone, corresponding to when the drone is effectively free-falling towards the last node.
These plots show the egocentric or camera view. The keypoint trajectories, resolved in the sensor frame, are shown in different colors. The points correspond to discrete nodes and the lines are the nonlinear propagation of the dynamics.
Figure 15: CT-LoS SCvx Iterations.
Figure 16: DT-LoS SCvx Iterations.
These plots show the interations of Alg. 1 until convergence. The initial guess is created by taking the keypoint positional trajectory and shifting towards the -x direction by a constant amount and then fixing the attitude at each discretization node to point the sensor towards the keypoint. Note that both methods recieve the same initial guess.
In our experiments we seek to answer the following questions:
We use the following metrics to analyze and address the above questions:
Figure 17: Relative Navigation LoS Violation. The CT-LoS method has significantly reduced LoS violation compared to the baseline method.
Figure 18: Relative Navigation Runtime.
Figure 19: Cinematography LoS Violation.
Figure 20: Cinematography Runtime.
@misc{hayner2024los,
title={Continuous-Time Line-of-Sight Constrained Trajectory Planning for 6-Degree of Freedom Systems},
author={Christopher R. Hayner and John M. Carson III and Behçet Açıkmeşe and Karen Leung},
year={2024},
eprint={2410.22596},
archivePrefix={arXiv},
primaryClass={math.OC},
url={https://arxiv.org/abs/2410.22596},
}