In Millikan's oil-drop experiment，if the oil drop is chosen too small，it shows obvious Brownian motion，which causes the measured dropping time to deviate from the ideal dropping time. In this article，through practical experiment and theoretical study，the stochastic differential equation of the movement of the oil drop is established by solving the Langevin equation. By introducing single and multiple time deviation，both the Brownian motion and the time deviation are quantitatively analyzed. As being indicated by the result，when the oil drop is relatively small，the influence of Brownian motion should be taken into account because of the limit of experiment times.