There are actually multiple answers to this question.
On the show, they live in a town called Ossining in Westchester County, New York. It is located about 30 miles away from Madison Avenue in New York City and would take no more than an hour to commute to. The choice of Ossining has raised eyebrows among Mad Men fans and history buffs alike because an ad man of Don's stature and a woman as society conscious as Betty would have most likely lived in Connecticut or the East Side, Chappaqua, Darien, Rye. This has led many to speculate that creator Matthew Weiner chose Ossining because it was the hometown of writer John Cheever who wrote many short stories dealing with suburban angst.
Of course the show does not actually film in Ossining. The home that we have come to call "the Draper Household" is actually located 632 Arden Road in Pasadena, California. Check out a street view on Google maps.
More surprising is the fact that the Drapers actually moved and forgot to tell us! In the Pilot episode, this is the house of Don and Betty:
When it is time for Sally's birthday in Episode 3, this is their home:
Whether their house is in Pasadena or Ossining, I'm sure we would all like to be invited over for dinner. But sorry Don and Betty, we can't bring the scotch.
Links referenced: AMCTV and Iamnotastalker.com.