I started questioning this today actually... kind of seems like I'm not. Its a Rigol DS2202A for reference.Are you using you oscilloscope properly?
What does the trace look like if you connect the ground clip to the probe tip?
What if you connect it directly across a filter cap on your 7805? (you are using filter caps, right?)
I will try connecting the ground clip to the probe tip. I did not put capacitors on the 7805, but there is a 100uF capacitor at the input of the PCB where I spliced the wires in.