Steve,
I have an 80 gallon tank and I clean my filter once a month. I only rinse the sponges, but I've never noticed anything trapped in them.
When I first saw the stuff floating around I thought they looked like shed skin as well. But there was just too much to be from the little turtles.
Also, when you clean the tubing, you have to remove the tubes from the plastic connectors right? Great...I keep thinking of the initial setup when I had the hardest time connecting the tube to the outlet and inlet pieces.
Well, at least its good to know that I'm not doing something wrong that's causing the growth.