Probabilistic models for vehicular situation awareness using monocular camera and deep learning | |
| Author | Lianen, Qu |
| Call Number | AIT Diss no.CS-21-02 |
| Subject(s) | Deep learning (Machine learning) Driver assistance systems |
| Note | A dissertation submitted in partial fulfillment of the requirements for the degree of Doctor of Philosophy in Computer Science |
| Publisher | Asian Institute of Technology |
| Abstract | Consumer and commercial vehicles are increasingly utilizing a variety of advanced technologies for autonomous driving, driver assistance, and situation awareness. To provide these capabilities, it is vital that intelligent vehicles perceive, assess, and predict their own state as well as the state of the environment around them, comprehensively and precisely. In this con text, the present dissertation attempts to provide answers to the questions of 1) how can we obtain precise information about the driving environment, 2) how can we compute accurate relative positions and velocities of vehicles surrounding a vehicle in the driving environment, and 3) how can we predict future vehicle trajectories for criticality assessment. The answers to these questions lie, at least in part, in the development of methods for accurate sequential state estimation and object tracking. Sequential state estimation and object tracking traditionally require integration of inform ation from a variety of sensors. In this dissertation, however, I explore the feasibility of performing these tasks while equipping a vehicle with just a single type of sensor: simple monocular cameras. My aim is to push the technology as far as possible, assuming that since humans can drive safety with visual feedback alone, so too could our vehicles’ perceptual systems provide useful information based on visual feedback alone. I therefore assess the efficacy of systems without external tracking aids such as GPS or visual markers. If we can use monocular cameras efficiently, we can develop simple and accurate state estimation and tracking methods that run in real time. My proposed solution synthesizes classic computer vision methods such as optical flow with modern deep learning methods for object detec tion to obtain a novel approach to situation awareness in driving environments. I develop new methods for 3D mapping of the environment around the vehicle, estimating the relative positions of target vehicles around the host vehicle, estimating the linear velocity and an gular velocity of the host vehicle, and probabilistic modeling of the uncertainty in vehicle trajectories in real time. We implement and test the approach quantitatively in simulation and qualitatively on real-world test video. The main contributions of the dissertation are: 1) I propose a new integrated method based on camera calibration, nonlinear optimization, and a state-of-the-art deep learning object detection model that performs 3D mapping of the vehicles around the host vehicle. Projective geometry and nonlinear optimization provide a mathematical foundation for the method. Target vehicles’ relative positions are estimated from image-based detections then refined by a vehicle position optimizer. The method can accurately detect and localize foreground objects and provide detailed information about the relative positions of vehicles in the driving environment.2) I propose a new method for computing the linear and angular velocity of the host vehicle in dynamic road environments based on camera calibration, optical flow, integration of in formation from multiple cameras, and a specialized random sample consensus (RANSAC) algorithm to remove outliers. The method computes host vehicle rotation and translation increments robustly and accurately in the presence of noise in the dynamic environment. 3) I propose a novel dynamical statistical model based on extended Kalman filtering to realize a probabilistic model of uncertainty in vehicle trajectories in real time. The dynamical model enables prediction of the trajectories of vehicles around the host vehicle and assessment of the likelihood of a collision in advance. Simulations and real-world experiments confirm that the new method is effective at gener ating smooth, accurate trajectories for the host vehicle and target vehicles. The algorithm is superior to state-of-the-art sequential state estimation methods such as visual SLAM in performing accurate global localization, with the additional benefit of dynamic trajectory estimation for target vehicles. |
| Year | 2021 |
| Type | Dissertation |
| School | School of Engineering and Technology |
| Department | Department of Information and Communications Technologies (DICT) |
| Academic Program/FoS | Computer Science (CS) |
| Chairperson(s) | Dailey, Mathew N. |
| Examination Committee(s) | Vatcharaporn Esichaikul;Mongkol Ekpanyapong |
| Scholarship Donor(s) | China Scholarship Council (CSC) |
| Degree | Thesis (Ph.D.) - Asian Institute of Technology, 2021 |