The biggest threat comes from predators, especially reptiles like snakes and large lizards, for example alligators, plus fishing birds.
You can keep off the reptiles by planting repellant plants for example tobacco and lemon grass around the ponds.
Dogs and cats can also reduce incidences of reptiles coming to your ponds to steal fish. You should also keep the pond area well-cleared of over-growing grasses.
To keep off birds, set a wire gauze over the ponds so that birds cannot swoop low into the ponds.