SPLA soldiers. [AFP]
South Sudanese forces captured the oil field last week, accusing Khartoum of using it as a base to launch attacks.
UN chief Ban Ki-moon has said the occupation is illegal and also called on Sudan to stop bombing the South.
The escalating fighting and rhetoric between the two sides over the last week has led to fears of all-out war.
South Sudan seceded last July following a 2005 peace deal which ended a brutal two-decade civil war in which more than 1.5 million people died.
On Thursday, South Sudan issued a statement saying it was not interested in war with its northern neighbour and that it would only withdraw from Heglig if the UN deployed monitors there.
Sudanese President Omar al-Bashir had earlier threatened to bring down the government in Juba following the loss of Heglig, which provided more than half of Sudan's oil.
Main disputes between the two Sudans
- Transit fees the South should pay Sudan to use its oil pipelines
- Demarcating the border
- Both sides claim Abyei
- The rights of each other's citizens now in a foreign country - there are estimated to be 500,000 southerners in Sudan and 80,000 Sudanese in the South
- Each accuses the other of supporting rebel groups on its territory
South Sudan's withdrawal has been ordered to create the environment for talks with Khartoum, Reuters news agency reports.
"An orderly withdrawal will commence immediately and shall be completed within three days," AFP news agency quotes a presidential statement as saying.
According to the Associated Press agency, Mr Kiir said the South still believes that Heglig is a part of South Sudan and that its final status should be determined by international arbitration.
Heglig is internationally accepted to be part of Sudanese territory - although the precise border is yet to be demarcated.
In the latest of intense diplomatic efforts to prevent a wider conflict, US special envoy Princeton Lyman visited Khartoum on Thursday.
The regional body Igad, which mediated the 2005 peace deal, expressed "grave concern about the escalating conflict" and said it would "extend all possible assistance to maintain peace and stability".
“We will be involved having suffered a proxy war by Khartoum”
Gen Aronda Nyakairima Uganda's military chief Uganda, a close ally of South Sudan, also indicated it might become involved if the fighting became a full-scale war.
"We will not sit by and do nothing," Ugandan military chief Gen Aronda Nyakairima is quoted by Uganda's private Daily Monitor newspaper as saying.
A Ugandan army spokesman told the BBC that diplomacy would be exhausted before any military action was ever considered.
During Sudan's civil conflict, Uganda accused the Khartoum government of supporting Lord's Resistance Army, which was fighting in northern Uganda.
Uganda backed the South Sudanese rebels during the civil war and now has extensive economic interests in the newly independent country.
The LRA, led by Joseph Kony who has been subject of a recent viral online campaign highlighting his activities, now roam the jungles of Central African Republic, South Sudan and the Democratic Republic of Congo.
"We will be involved having suffered a proxy war by Khartoum," Gen Nyakairima told the Daily Monitor.
"Our people in northern Uganda suffered and intelligence information also indicates that the LRA, who have an estimated 200 guns, are again in contact with Khartoum," he said.