I mean, you are doing all of this for a guy that clearly just wants to be friends right now, and you are NOT focusing on yourself to better yourself; you are focusing on HIM and changing for HIM. I know you are going to deny that, but it is very obvious that the motivations behind your ‘changes’ are based around him.

What you need to do is to work on yourself for yourself, make new friends, socialize, and basically just develop a life that does not revolve around him because you don’t know whether or not he is going to want to be with you. It seems as if he isn’t interested in anyone right now; if he was interested in the girl that asked him out- he would have gone out with her. If he was interested in you, well, he would be with you.
If he just so happens to actually meet a girl that he IS interested in, then you are done for. That could absolutely happen. You don’t know.
So you can’t just count on him taking you back, or you will be devastated and you will revert back to your old ways because you weren’t gaining anything for yourself.

You need a backup plan.
And you don’t have one.
He could, very well, meet someone else. I think, actually, that he might be considering meeting someone else, or waiting to see if anyone else who interests him comes along, and if they don’t, then YOU are HIS backup plan, but I can’t be sure of that. It just seems like that is a possibility.
Just find something to do for YOU.
Make a life for yourself so that, if things don’t work out with this guy, you won’t have lost everything.