If you can, remove any spent or dead plant material. The "when" probably isn't as crucial, because it's not an easy task when you might have to literally get into the pond -- but when you can. Eventually it's going to decay and could cause water quality issues. Just be careful about pulling or tugging on lily parts, though, as you might uproot the plant. They make a scissors/grabber combo tool that does an OK job. As far as any other plant material, I scoop it off the surface -- otherwise, it waits until, well, whenever.....