Is the sound more like a thump?
Maybe it is what they call "power on/off thump."
In what order do you power on/off the components in your system (i.e., cd player, preamp, etc).
If the sound is very slight, I would not worry about it, but if you see the cone of your speakers moving a lot, then you need to have your amp checked.