The Drifter — Susan Wiggs

He wanted to tell her everything. About the lost years that had changed him from a desperate young boy into a man hardened by life. About the night he'd sold his soul for a woman who wasn't worth the price....

COMES A DRIFTER TO A WINDSWEPT ISLAND...
But Jackson Underhill said nothing. After all, he was an outlaw, clearly on the run--reason enough for silence.

The truth was Dr. Leah Mundy scared him. She made him want to trust again, to share his burden. She made him want a home, a family. And it was dangerous to want such things. Because the past would find him if he stayed -- and there could be no future with a woman who would not leave.