For this idea, you could add builders to the same project. This would allow you to cut the time it takes to finish the upgrade but you would have less builders to work on other things.
Certain buildings (i.e. town halls) could be exempted from the ability to add builders.
This idea would allow you to complete buildings that you just want to finish or get out of the way quickly and it would also find a use for a builder that you have no use for at that time.