One of the most extensively used techniques in computer vision is object detection. It enables us to recognize and find an object in an image or video. The main aim is to detect the object which was involved in the abnormal activity in public places like Railway Tracks and National Highway Roads. The abnormal activity denotes the harassment, robbery, suicide and accidents which mostly happen during the night time in Roadways and Railways. This detection methodology uses the modified YOLO v4 algorithm to recognize the object, with the kind of identification and localization, it may be used to count the items in a given image and monitor their suitable place. Object detection distinguishes the sorts of objects present in the picture before locating the precise manifestation of the object. For object labelling, a bounding box surrounding the item can be used to accomplish object localization. As a result, the object detection alerts the control room of the appropriate department about the abnormal activity and helps the officers to take immediate action to prevent the illegal activity occurring in the area. This object detection methodology not only detects human beings but also detects the trapped animals in the railway tracks.