George, Andy
Lime mortar was used a lot in construction with softer materials such as stone, and was very pale grey in colour, whereas cement was used with harder materials such as brick and granite, and had the creamier colour. As you quite rightly mention George, they both would have discoloured with age but lime mortar tends to leave some white deposit over the stonework.
At the end of the day, its your choice, and whichever you choose, I'm sure you'll work your magic and it will look convincing!
Regards, Deano.
Edit ....and of course sand comes in different colours depending on where it comes from, which of course can also affect the colour!!