Purpose/Significance To analyze the impact of multimodal data on patients’ online physician selection behavior, and to further understand patients’ decision-making in the context of big data in online healthcare community. Method/Process Based on the data of the Haodf.com, feature engineering-based neural network prediction models and end-to-end deep learning prediction models are constructed to predict patients’ physician selection behaviors. SHAP values and feature importance ranking methods are used for interpretability analysis. Result/Conclusion In terms of predictive performance, end-to-end models are generally superior to feature engineering models. The features of system-generated content and patient-generated content in community have a significant positive impact on patients’ physician selection behavior, the impact of doctor-generated content shows considerable difference. The most influential multimodal feature is structured data, followed by patient comment text and doctor images, with the least impact from message text. These findings demonstrate that multimodal information in online healthcare community has a certain influence on patients’ physician selection behavior.