The paper introduces the basic theory of deep convolutional neural network and expounds on visual feature extraction of brain images based on deep Convolutional Neural Network(CNN), designs a kind of categorizer that is applicable to brain disease image to build brain image disease category characteristics database, making applications like clinical assistant decision-making based on brain image disease category characteristics database possible.