Do Muslims hope the Holy Temple in Jerusalem is never rebuilt?

Muslims profess to worship the same God that Jews and Christians do, if that is the case than the site where the temple once stood was sacred to them as well, long before their prophet Muhammad supposedly rose to heaven there. There is currently a large, and admittedly beautiful Mosque on the temple mount. Do Muslims, like the rest of us (Jews and Christians), look forward to the day when the old temple is rebuilt and restored to its former glory or do they want to stop such a thing from happening? Why or why not? Sources would be helpful since I'm looking for answerers who actually know what they are talking about.