Stop Aphids without Hurting Eggs or Caterpillars?

We don't treat plants with aphids during the season. A better strategy is planting several species of milkweed and spreading them out. Pests like aphids won't discover them all and, over time, you'll attract more aphid predators:

10 Ways To Stop Aphids from Taking Over Milkweed