That is normal for the old type of adapter that uses a heavy transformer that steps down the AC mains voltage directly. These adapters don't have any regulation, so the output voltage varies quite a lot depending on the load current.
The marked output voltage applies when the maximum rated current is being drawn from the adapter. Even then, it is only approximate, and variations in the AC mains voltage will affect the output voltage. When there is no load on the adapter, the output voltage can be 50% higher than the marked voltage - sometimes even more!
These adapters also have ripple on the output voltage, at twice the AC mains frequency. This won't be a problem for an Arduino though.
Modern switching adapters are fully regulated and are better if you want an accurate output voltage. They generate noise, and this makes them unsuitable for some types of loads - AM radios, for example - but they are a better option. They are also much lighter, and often cheaper, than metal transformer types.