The hippocampus is a critical brain structure for memory and goal-directed behavior. When an animal is awake, it represents the animal’s position in the environment and other task-relevant variables and forms new memories. When an animal is asleep, it simulates plausible behavioral trajectories through the environment, often called replay. We are interested to understand how these computational abilities emerge and are shaped by the hippocampus’ circuit structure, and how the interaction between emergent dynamics and plasticity during sleep supports the hippocampus’ role in navigation and memory. Recently, we found that recurrent neural networks trained to predict the sequence of sensory input produce representation and replay similar to that seen in the hippocampus, and are now working to build on this model - by making it more reflective of the hippocampus’ circuit structure and by studying the functional implications of intrinsically generated replay.