Updated Feb 6, 2018
Those are called saprophytes, although there is a wide range, and some fungi also perform this function.
If no decomposition was happening at all, then the nutrients wouldn’t be available for recycling into plants, so my guess is that all other life on earth would eventually starve. It might take awhile.
Also, partly decayed plant and animal matter is among the things consumed by worms, insects, etc, so it would disrupt the food-chain / food-web from that angle, as well.
Edited To Add: Another group of bacteria whose absence would be disastrous are the participants in the nitrogen cycle, because all plants need nitrogen in a form that is generated by those bacteria.
Edited To Add Again: I recall reading an article stating that, plant matter (fallen leaves, etc) near the Chernobyl nuclear disaster site wasn’t breaking down as normal. If true, that suggests that, the radiation has harmed a range of saprophyte bacteria and fungi. This could have a sort of secondary impact on the ecosystems there, for future plants and animals.