Should I be worried about him?

My husband and I recently found out we're having a baby. (7w4d) He told me today that he wants to move to Texas to go back to school. However, it wasn't even brought up for me and the baby to move with him. We would be several states apart for eight months, while he was going to school. I mentioned to him that we don't have enough money right now for us all to move that far away, and his response was "I know." Should I be worried about it or should I just let it go?