KAMPALA, Uganda--Thousands of Ugandans rallied in front of the Ugandan Parliament yesterday to celebrate the fall of Idi Amin, the country's defeated dictator. Looters, including some members of the Tanzanian army, joined Kampalans in the continued plunder of shops, offices and residences in the capital city.

For over five hours, the crowd praised the Tanzanian and Ugandan exile forces for overthrowing Amin's regime. They vowed their support for the new provisional government which announced control of the country Wednesday. Top officials in the new government will take office Friday.

But in a radio broadcast from his headquarters in northeastern Uganda, Amin insisted that the country still belongs to him.

"We have got our soldiers controlling the country," he said.

However, numerous eyewitnesses reported that Amin's troops were fleeing toward the Kenyan border. They reportedly harassed local residents, looted, and stole cars along the way.

Sources in Uganda believe that Amin has established a headquarters in Soroti, a town 200 miles northeast of Kampala. Much of Uganda, including all of Amin's home territory in the north, still lies beyond Tanzanian front lines. The invaders have not mounted an immediate military operation to hunt down Amin and his remaining loyal troops.

Sooner or Later

But a Kampala radio broadcast urged Ugandans to find Amin. "He deserves the gallows," the victorious exiles in control of the radio station said yesterday. They also demanded that his officers surrender within two hours, "or be dealt with accordingly."

Tanzanian forces reportedly freed at least 3800 prisoners from a Kampalan jail. International human rights groups say Amin was responsible for the death of tens of thousands of Ugandans and for incarcerating thousands more.