Supreme Court Justice Samuel Alito considered his Senate Judiciary Committee hearings so painful that for months after his 2006 confirmation he would try to avoid the Hart building where the nomination hearings were held. “I cross to the other side of the street,” he said. “I quicken my step until I’m well past the building.”

It’s likely Justice Alito now feels that way about the Capitol, where his disagreement with President Obama on Citizens United v. FEC during the State of the Union was seen by a national television audience.

I was traveling when most of the criticism (some directed at Alito, some at Obama) played out in news reports and cable chatter, and I won’t echo that. I will only add what I know of Alito from my interviews with him. He has a quiet demeanor yet a definite contrary streak.

While the TV audience saw Alito mouth what appeared to be “not true,” Justice Sonia Sotomayor next to him apparently heard nothing. On the video, she doesn’t respond in any way. He may be a constant grimacer (he didn’t look happy at last year’s Obama Inauguration either), but Alito is simply not a talkative sort. He likes to work at home alone, especially when he is writing opinions. On the bench during arguments, other justices sometimes turn to a colleague to whisper a comment or wisecrack. Not Alito. He is in his own space. Some White House aides who accompanied him on courtesy calls to senators before his hearings said they had trouble making small talk with him.

I’ve found Alito relatively easy to talk to – yet also ready to voice the kind of contrary view the nation saw Wednesday night. When I interviewed him for my Scalia biography, Alito said he admired Scalia but found his originalist approach at times lacking. “To say that you’re an originalist doesn’t really decide the case. Originalism gives you a principle to apply. Very often, particularly in areas where things have changed so much, identifying the principle doesn’t really decide the case.” An example of where their originalism diverges: Alito and Scalia disagree on the Sixth Amendment, specifically on what it dictates about judges’ power in criminal sentencing and on the reach of defendants’ right to be confronted by the witnesses against them.

Alito, who like Scalia is a consistent conservative, took offense at media commentators of years back who dubbed the then-appeals court judge “Sc’Alito.”

“I was insulted by it,” Alito told me, “in the sense that I knew the only reason it was done was because of the fact that we were both Italians. I was flattered if people thought I wrote like him or thought like him. But just sort of lumping the two Italians together, I thought, was not appropriate.”

On the personal side, Alito made clear he is no Scalia, who is known for living large (duck hunting with Dick Cheney, the opera with Ruth Bader Ginsburg) and is definitely not a stay-at-home-alone guy. Justice Alito said he didn’t like to play poker or bridge and tended not to socialize much. “I am a very boring person,” he quipped. Well, as of now, three days after the State of the Union speech, the recorded “views” on YouTube of Alito’s response to Obama are in the hundreds of thousands.