Magnetic resonance imaging is the clinically acclaimed imaging modalities which is utilized for the screening of brain abnormalities. It provides the visual interpretation of the abnormalities in terms of tumors, masses, grey matter and clots. However, these readable features of brain are affected due to the presence of inherent Rician noise. Moreover, it also restricts the decision capability of the expert about the brain abnormalities. So, for the restoration and enhancement the brain MR images, a Reshaped Gabor filter based Convolution neural network (RGCNN) method is proposed. In order to develop the proposed RGCNN, a Gabor Layer is employed as the initial layer within a deep convolutional network. This modification provides a better correlation amid the noisy pixel The efficacy of the proposed method has been assessed with respect to the qualitative and quantitative assessment for the brain web dataset. The human visual system, full and no reference image metrics are used to quantitatively measure the performance of the proposed method. Apart from this, a comparative study has been also presented between the proposed and existing method to describe the effectiveness of the proposed method. The obtained results demonstrate that the proposed method is capable of simultaneously reducing Rician noise, preserving edges, restoring fine details, and enhancing anomalies.