You may have voltage remaining in a battery but with little or no potential current left to drive a load. You want to know how much power is remaining in your battery, not just voltage. Both voltage and current are important factors in a circuit. Multiply them together to get the battery power in voltamps (VA).
Example; I can have a 9volt battery that is almost dead and still read almost 9volts on a meter. If you put a load (resistance) on that same battery the voltage will probably drop to maybe a few volts. That is why you cant get a good reading without loading (adding resistance) the battery. If the resistance is added and the voltage remains high, the battery power is still strong.
A typical multimeter is fine, you just have to add a resistor for testing small batteries..