What is there to use gems on?

I know what you're saying, use them to finish the construction of your building or use them to buy builders, but that's not what I'm asking about specifically. I have five builders and I was wondering what I should use the gems you earn from removing obstacles on. You only get around three a day and it would take weeks to get just 100 gems from this. Should I just continue to save up the small amount of gems I earn from clearing out obstacles to finish construction of some things? Or maybe it would be more beneficial to speed up the training of troops?

What is your guys' take on this?

I prefer using them to speed up upgrades. Gemming troops or boosting barracks (boosting is more efficient) is really an indirect way of gemming for resources. So, unless you frequently find yourself running low on resources, I see more benefit in gemming upgrades.

I lily is gems, if an upgrade will finish say 30 minutes after I would have gone to bed.