This study conducted a research on the reliability of literature evidence retrieval from an evidence-based perspective, besides this study also explored the replica reliability and test-retest reliability of different retrieval methods. Based on the analysis of the relevant literature, it is found that the retrieval items with "keyword or full text" have higher test-retest reliability in the high correlation data set, and the retrieval items with "keyword and abstract" have the most stable results in the relevant data set; for the replica reliability, "keyword or full text" and "abstract or full text" in the relevant data set is very high, while the reliability of "subject or keyword" in the highly relevant data set is strong. This study shed light on the reliability evaluation index to ensure the stability and consistency of evidence retrieval, provide higher quality evidence for systematic review and meta-analysis, and provide theoretical support for evidence-based quality evaluation research.