My 2000 Celica GT does that too, but it's more like a tremendous squeal. It goes away when the car warms up, and it only starts making the noise when it's cold. I'm not sure which side it's coming from, but honestly, when I wash my car, the sound goes away, at least for a little while.
I'm thinking of taking it in for inspection.