You'd actually want that stuff going back into the tank or refuge, unless you have clean up crews in the sump. That stuff will just sit there unless you do. If it goes back into the tank, your cleanup crew will have a chance to eat it, or if it stays suspended and goes back down your skimmer has a chance to skim it out.
You should periodically vaccum out the sump if you get a buildup of stuff in the bottom of it.
If it gets bad, then you have an issue with the skimmer or the cleanup crew combined with the feeding of the tank.
It will rarely be pristine, but it shouldn't be a 1/2" thick either.
A simple strainer on the input of the pump should keep algae out, but again, the pump shouldn't be in the same chamber as the pump to begin with so its not supposed to be a stop all, just a shield of last resort.