Exploiting Diffusion Prior for Generalizable Dense Prediction