According to the existing problems, the paper proposes an organization and retrieval method of the massive medical images based on hierarchical deep learning in order to obtain more accurate retrieval results, solves the semantic gap problem that exists in the current medical image retrieval at a certain extent.