There is a limit.. people have had issues with sick fish, malnurished fish etc. Just my opinion, but a lot of issues I attribute to lack of feeding such as fish nipping at corals, aggression, fish jumping, illnesses and the like... There are also many reports of coral fading/pasteling and coral death because there is not enough food to go around.
While its not as common as nitrate and PO4 problems, underfeeding is just as bad long-term if its taken to an extreme.
Ultimately the trick is to find a balance between feeding and the tanks capability to remove nutrients. Reduced feeding is often looked at as a method of making your tank healthier when in fact it will have the opposite effect. Things such as macro algae, better skimming, better water change habits etc should be employed first.
Bristleworm populations will grow/contract with the tanks lifecycle. Most are harmless. The larger worms you need to properly ID as some can be predatory and unseen until they reach a certain size. Provided you don't have losses to things like snails, clams, corals etc you are probably fine, but if you start losing things for no apparent reason, you want to keep an open mind about the larger worms. We see them pop up on a regular basis, Marc had a few not too long ago. So getting a photo of it if possible is a good idea, for peace of mind if nothing else.
Even worms in families considered undesirable can be beneficial as the particular species dietary habits vary. Fireworms are a good example of that. Some are relatively harmless unless you touch them, while others are fish and invert predators and most of them in the family will grow rather large. These are often confused with bristle worms, as are some of the predatory ones, so an ID is important.
A general rule.. if it has antennea or a vice/pincher type mouth, its bad. If the head segment is a very well defined head that varies greatly from the rest of the segmented body, there should be a concern. Black or Dark Brown, like chocolate are usually problem children. Bright colors such as right reds, oranges, yellows etc are always a concern and should be ID'd if possible.
When in doubt, pull it out and isolate it. Its easier to put it back in if it turns out to be harmless than being on a constant search for it after its gone back into the crevices.
fficial&client=firefox-a&um=1&ie=UTF-8&ei=9d2aSfHOJo-Etge10KmgCw&sa=X&oi=image_result_group&resnum=4&ct=title