Significance To use multimodal data analysis method to mine medical Q&A data in online healthcare platforms and predict whether patients will adopt online doctors’ responses. Method/Process First, numerical, categorical, textual, and visual data related to doctor-patient Q&A are obtained from online healthcare platforms, and three datasets of acute disease, chronic disease and mixed disease are constructed according to disease types. Then, normalization, one-hot encoding, Med-BERT, and convolutional neural network are used respectively to process numerical, categorical, textual, and visual data. Finally, a gradient boosting decision tree is used to predict whether patients will adopt online doctors’ responses. Result/Conclusion Doctors’ profile pictures can improve the prediction effect of online doctor response adoption, and multimodal data mining can effectively predict the response adoption.