All three of Harvard's provosts have come from inside Harvard--two have been deans and one a professor at the business school.
But Summers is known for taking people with him from job to job, so it is possible that he could appoint a trusted friend from the Treasury Department or World Bank to serve as provost.
Summers has not yet commented on his plans for the office of provost.