The field of gesture recognition, which is a part of computer science and language technology, focuses on using mathematical algorithms to analyze human gestures in non-verbal communication, particularly hand gestures or movements. This research paper presents the development of multi-stream deep transfer learning models, specifically Inception-V3, VGG-16, and ResNet-50, for recognizing signs of south Indian languages using double-hand gestures such as Kannada, Tamil, and Telugu. The classification performance of these models has been enhanced using a dataset of 10,000 double-hand gesture images. Among the models, Inception-V3 achieved the highest test accuracy of 89.5% and validation accuracy of 88.45% in classifying double-hand gesture images into ten categories. The results of this study could be used to create automated systems that help people with speech impairments or other functional limitations and enhance their talents.