Being attacked by moths and wasps or other pests is a sign of a nutrient deficient tree. A friend of mine did an experiment on her tomatoe plants (she does organic gardening, but wanted to prove how it works to friends/family). The pests attacked the tomatoe plant that she used the normal conventional way (synthetic fertilizers, etc). With the other tomatoe plant she used kelp meal (seaweed has all the trace minerals), soil loaded down with beneficial microorganisms, and I can't remember what else. Nothing bothered this plant and it grew good and strong.
We have 4 fig trees and they say birds and wasps will get to them. We have problems with neither. And, if any fall to the ground they aren't left to rot and attract yellow jackets because the chickens love figs. We have tons of squirrels, and I'm sure they eat some of the figs, but they can only handle so much.