Computer vision systems have applications in surveillance, self-driving automobiles, and robots, and one of the most important components of these systems is the ability to recognize and track objects in real time. The computer vision tasks of object identification and tracking are extremely important and have a wide range of applications in the real world. Some examples of these applications include surveillance, robots, and autonomous driving. In this paper, we provide a proposal for a system that can detect and track objects in real time by utilizing deep learning and OpenCV. Using frame differencing, optical flow, background separation, single-shot detection (SSD), and MobileNets, we propose a system that can recognize and track objects in real time. On many different datasets, the accuracy of the proposed system is very good, and it performs well in real time.