Vehicle Interiors with Deep Learning and Computer Vision

Data Science & Business Analytics MS Program: Summer 2019 Capstone Project

Deep Learning in the Auto Industry

Deep learning and computer vision has been leveraged in the auto industry for autonomous vehicles, insurance claims, maintenance, and license plate readers. However, these techniques have yet to be applied to vehicle interiors.

In the summer of 2019, a team of graduate students from Wayne State University undertook the task of evaluating and creating state of the art deep learning techniques to identify the interior features of a vehicle. The goal of this project is to develop a machine learning model that can help identify the content of a vehicle and its features using visual cues from its interior images of the vehicle. This is a multiclass supervised classification problem that will require labeled images to learn the features from curves, edges, and combination of features. Our dataset consist of images collected from the CompCar dataset.

Models have been created with:

Object Detection - Tensorflow 1.X - Utilizing the Object Detection API
Image Classification- Tensorflow 2.0 - Utilizing TF Hub we evaluated base models, retrained and fine tuned architecture.
This will ease transition for future work with a publically maintained codebase.

Data Collection Requirements for Dashboard images

Image Requirements

Dashboard images were collected with a minimum of 3 objects per photo.

Object CLassification & Detection

Detection relies on localization and classification. Using bounding boxes gives us more insight into what the model is learning from. Identifying objects such as the steering wheel, odometer, control center, gear Lever.

Future Work

Mask RCNN Inference identifies Masks from pixel segmentation

METHODOLOGY

Project Steps

Data

With no dataset that is specific to vehicle interiors, our team consolidated images and labeled roughly 300 interior images. The testing photos were collected by the team and with the help of classmates.

Image Labeling

Labeling is done with open source tools, labelImg. Labeling was required for the bounding boxes for the training dataset. LabelImg outputs XMLfiles in the POSVOC format that is used in localization. This tool can be found at LabelImg

Pre-processing

Images and labels are split into 80% training and 20% testing and then the training dataset is further split for the validation set. Each model has a particular configuration required for input.

Modeling

Transfer learning was used as a feature extractor and multiple architectures were experimented with to determine the best tradeoff on speed and accuracy. MobileNet and SSD

Tuning

Tuning parameters such as size, batch size, and fine tuning layer were used in tuning along with a grid search of hyper-parameters

Results

Image Classification is evaluated based on Accuracy - Precision - Recall Object Detection additionally uses IOU - MAP

DEMO

Object Detection Demo

CONTACT

Detroit, MI US

gp5880@wayne.edu