After months of speculation, it looks like Scott Brown is finally pulling up stakes and moving to New Hampshire. Once the former senator announced that he would forgo a run for governor of Massachusetts, the smart money has been on Brown leaving the Bay State and heading north. With Senator Jeanne Shaheen up for reelection in 2014, the prospect of Brown unseating the Democratic incumbent has Republican fundraisers salivating. But before he can take on Shaheen, Brown has a very important obstacle to overcome: a conservative-leaning Republican Party in the Granite State that may not be as enthusiastic about the GOP star as his fans in Washington.

Not even high popularity ratings were enough to reelect Brown last year after he shot to fame in 2011 by winning a special election to fill Teddy Kennedy’s old Senate seat. Deep-blue Massachusetts may have a soft spot for moderate Republicans like Brown and men like Mitt Romney and William Weld, who have won the governorship. But Brown’s decisive defeat at the hands of Elizabeth Warren last year made it clear that a change of address was the only way he was going back to the Senate. But now that a New Hampshire Senate campaign is becoming more likely, Brown and his many fans in the GOP are coming to grips with the same problem faced by Romney once he left liberal Massachusetts and sought the approval of Republicans elsewhere. New Hampshire may be part of the Boston television market and most of its citizens may root for the Red Sox, but its Republican Party is a lot more conservative than the one in Massachusetts. That means the pro-choice and anti-gun candidate who had cross-party appeal in Massachusetts must now convince Republicans who view such stands with disgust that he speaks for them.

Brown’s dilemma is the same as many other Republicans who have come to grief in primaries in the last few years, as grass roots conservatives and Tea Party activists have mobilized in support of more conservative Senate candidates. Seemingly sure Republican wins turned into agonizing losses in Delaware and Nevada in 2010 as Christine O’Donnell and Sharron Angle knocked off more electable candidates in primaries. That wasn’t a factor in Brown’s 2011 and 2012 campaigns for the Senate but in New Hampshire, a state where Republicans have traditionally taken their cues from the conservative editorial page of the Manchester-based Union Leader, Brown seems more like a dread example of a mainstream moderate than the ObamaCare-bashing Tea Party favorite that he was in Massachusetts. His stands on abortion and guns could prove to be a serious impediment to gaining Republican support in New Hampshire.

Of course, those issues aren’t the only problems Brown faces. Owning a vacation home in New Hampshire for many years won’t be enough to convince some voters that he isn’t a carpetbagger driven to their state only by political opportunism. Moreover, as some political pundits have noted, Brown’s move has been more of a whim than a long-term plan. Earlier this year he was flirting with a presidential candidacy by hanging out in Iowa. And so far his campaign in New Hampshire is more a matter of celebrity freelancing than an effort being driven by a clearly thought-out strategic plan.

But that said, Brown’s celebrity as well as his charisma make him a clear favorite in any GOP primary. Former Senator Bob Smith, who has been toying with a return to New Hampshire, wouldn’t present a viable alternative and the other possibilities are unknowns. But conservative unknowns have a way of knocking off GOP celebrities who are moderates. Especially moderates who deviate from consensus conservative positions like abortion and guns. He should also realize that while ObamaCare may make Shaheen vulnerable, she is no Martha Coakley. Even if he wins his party’s nomination, he’ll have the fight of his life on his hands to unseat her.

New Hampshire seems like the solution to Brown’s political problems, but winning a Senate seat there will be a lot harder than just changing the address on his driver’s license.