Although tree stumps can be left in the ground, and covered with sprawling plants, such as herbaceous clematis, this can lead to problems with suckering where new shoots arise from the trunk and roots.
Completely dead stumps won't form suckers, but they can play host to root diseases such as honey fungus and, as such, aren't worth leaving in and taking the risk.
Physical removal of the stump is the best solution. Machines known as stump grinders will mechanically grind out the main root plate, leaving a fine sawdust. This can be left to fill in the hole, used as a mulch in other areas of the garden.