Most of the time, yes. You have a voltage regulator, in newer vehicles, it is built into the computer. This limits the voltage, and thus the current supplied to the battery.
The alternator is capable of supplying more current at higher RPMs, but in most conditions, it can provide more current than needed to keep the battery fully charged. But there may be conditions where the alternator cannot keep up at idle, for example, running the HVAC fan on its highest setting while the lights are on high beam and listening to a high powered audio system. This could drain your battery at idle.
A voltmeter is pretty useless for assessing the health of your battery. It can only provide the current voltage between the posts. Cars used to have an ammeter that measured the current going into other out of your battery. This gives a lot more information on the health of the battery. Ammeters are hard to wire into the circuits of modern vehicles.