Neither the voltage, e.g., 3V or 12V, nor the size and shape (AA v button cells), has to affect the amount of energy stored, if other things are equal. For a given battery technology, a given volume can store the same amount of energy, regardless of the battery voltage. There may be inefficiencies due to wasted space in packing multiple cylindrical cells, but other shapes are available.
What really matters is how much power the motors need, i.e., how quickly the energy store is depleted.
You could use a couple of 2200mAH AA NiMH cell and use a voltage boost circuit to generate whatever higher voltage you want. Discharging at the 1C rate (i.e. discharged in 1 hour, or 2200mA in this case) with a cell voltage of 1.4V at 90% efficiency (should be achievable) would give almost 1/2 Amp at 12V, for an hours running. In practice the run time will be much longer unless you run your trains flat out and never stop them.