The only idea that I have is a "boost" to the builders. Maybe a cost of 50-100 gems to reduce the build time by half. Maybe just a periodic boost like the 1 gem boosts for the resources, using the same higher gem cost since it would be a dramatic help for some buildings.