Before the 20th Amendment to the Constitution the inauguration of the new president took place on March 4th. This was moved to January 20th in 1933 by the 20th Amendment.
Having a four-month gap between the election and inauguration was a practical necessity in the late 18th century. The president-elect needed time to put his personal affairs in order and then embark on the sometimes long and challenging journey from his home state to Washington. In those days, before planes, trains and automobiles, long distance travel took place mostly on horseback.
Under good conditions, a rider in excellent shape on a horse in excellent shape, will travel perhaps 30 miles a day on an ongoing basis. When John Adams traveled from Boston to Philadelphia for the First Congress in 1775, he rode in the middle of a bitter winter, and I doubt he could have covered more than 10 or 15 miles in a day before having to turn in. So the trip between Boston and Washington of 440 miles could take 30 to 45 full travel days. Beyond riding every day outdoors in the winter, the traveler had to have funds for food for the horse for that long, expenses for food and lodging for the himself and his inevitable companion for such a long journey. If the president-elect traveled from a far-away place like Georgia, it took even more time.
As the 20th century came around, it became obvious that the start of the new terms for the president (and congress) in March caused a long lame duck period for the outgoing president and congress, resulting in a lack of good decisions in crisis. This was most obvious in 1861 when Lincoln had to wait before he could deal with the secession of Southern states, and then again in 1933 when Roosevelt needed to take action regarding the Great Depression.
With modern travel and communications tools, this delay is no longer necessary. The president-elect can reach Washington within 12 hours flying time from anywhere in the nation, and he can run his personal affairs from his iPad.
When inauguration day falls on a Sunday, like it does in 2013, the inauguration is done privately in the White House at noon, but the public ceremony takes place on the day after, a Monday. The thinking is that federal offices should be open for the inauguration ceremonies.