I'm glad you found this forum and hope you find the answers you are looking for here.
There are alot of reasons to marry someone, but you should never marry someone out of quilt because you slept with them. Talk to God, tell him you are sorry, and don't repeat your mistakes. But do not marry someone for this reason alone. It will never work out if you marry him out of guilt. I know the guilt and shame you are feeling right now. I've been there myself. But marrying this guy will not make those feeling go away. It will only make them worse, because you will be marrying a non Christian. The saying "two wrongs don't make a right" defiantly applies in this case.
You know the decisions you have been making are wrong or you wouldn't be here asking for help. You know in your heart that God loves you, he wouldn't leave you "stuck" with someone for the wrong reasons. He will forgive you for the wrong choices you've made. Trust in him.