To prevent your baby from swiping out of the app go to Settings>General and turn off “Multitasking Gestures”. To prevent your baby from exiting the app with the Home Button, use the Guided Access feature. Go to Settings>Accessibility>Guided Access to turn it on.
