Yes, LE, that's and interesting approach. I didn't think at all of a sort of girl's home, but that might perfectly be. And also the the girl might be from earlier days, so the verses are perfectly linked together.
@surferboy: Beautiful thought, except that i think that the line "not that we ever cared" relates only to the fact of beeing broke. - This attitude of young ones: We don't care that we don't have any money.
But this reading does not diminuish the interesting contrast you are talking about and that i feel too, when listening to the song. The decision to search for the girl, is certainly a moment of caring, maybe also a moment of hesitation and doubt to follow the path as a musician.
At the end it is also something he sacrificed for making career/or fulfilling his dreams.
What does he mean with "Later on i picked up the ball / and i took off down the line"?
Because of all that i like the line "I suppose by then I'd realised you'd run into hard times". It is this moment of clarity in a young live when you know you laugh but things are not that easy how you suppose them to be. But it's also the clarity you need to stick to your path. It'is a small moment of self-reflection, and this is so beautiful described here.