To withdraw into oneself — to shut oneself away from others, becoming inward and avoiding social interaction (idiomatic; 'shell' is figurative).
失敗してから彼は殻に閉じこもり、誰とも話さなくなった。
Since his failure he has shut himself away and stopped talking to anyone.
新しい職場に馴染めずに殻に閉じこもる人もいるが、徐々に慣れることが多い。
Some people withdraw into themselves when they can't get used to a new workplace, but they often adapt gradually.
友達は彼女に無理に話しかけるのではなく、殻に閉じこもらないようゆっくり支えた。
Her friends supported her gently rather than forcing conversation, so she wouldn't shut herself away.