In 1983, Garang went to Bor, ostensibly to pacify 500 southern government soldiers in battalion 105 who were resisting being rotated to posts in the north. However, Garang was already part of a conspiracy among some officers in the Southern Command arranging for the defection of battalion 105 to the anti-government rebels. When the government attacked Bor in May and the battalion pulled out, Garang went by an alternate route to join them in the rebel stronghold in Ethiopia. By the end of July, Garang had brought over 3000 rebel soldiers under his control through the newly-created Sudan People's Liberation Army/Movement (SPLA/M), which was opposed to military rule and Islamic dominance of the country, and encouraged other army garrisons to mutiny against the Islamic law imposed on the country by the government. This action marked the commonly agreed upon beginning of the Second Sudanese Civil War, which resulted in one and half million deaths over twenty years of conflict. Although Garang was Christian and most of southern Sudan is non-Muslim (mostly animist), he did not initially focus on the religious aspects of the war.
Garang had been fighting for a "New Sudan" since 1983. He was a strong advocate for national unity: minorities together formed a majority and therefore should rule. Together, Garang believed, they could replace President Omar al-Bashir with a government made up of representatives from “all tribes and religions in Sudan." His first real effort for the cause, under his command, occurred in July 1985 with the SPLA’s incursion into Kordofan.
The SPLA gained the backing of Libya, Uganda and Ethiopia. Garang and his army controlled a large part of the southern regions of the country, named New Sudan. He claimed his troops' courage came from "the conviction that we are fighting a just cause. That is something North Sudan and its people don't have." Critics suggested financial motivations to his rebellion, noting that much of Sudan's oil wealth lies in the south of the country.
In the spring of 1991, Mengistu Haile Mariam's regime (in Ethiopia) was overthrown by the Khartoum backed Ethiopian rebels (Ethiopian People's Revolutionary Democratic Front). Upon the rebels’ seizure of the government they closed all SPLA training camps in Ethiopia and cut off the SPLA's arms supply, forcing the SPLA to return hundreds of thousands of Sudanese back to South Sudan. This disrupted military operations and leadership within the SPLA. However, this caused the West to reconsider relations with the SPLA – justifying their providing the SPLA with "non-lethal help."
Shortly after, there was an attempted coup to oust Garang by three senior SPLA commanders in August 1991. The coup turned out to be premature – however, it exposed the deep ethnic divides within the SPLA. Once everything was resolved, Garang resumed power of the SPLA again in 1995.
Garang in a crowd of supporters
Garang refused to participate in the 1985 interim government or 1986 elections, remaining a rebel leader. However, the SPLA and government signed a peace agreement on January 9, 2005 in Nairobi, Kenya. On July 9, 2005, he was sworn in as vice-president, the second most powerful person in the country, following a ceremony in which he and President Omar al-Bashir signed a power-sharing constitution. He also became the administrative head of a southern Sudan with limited autonomy for the six years before a scheduled referendum of possible secession. No Christian or southerner had ever held such a high government post. Commenting after the ceremony, Garang stated, "I congratulate the Sudanese people, this is not my peace or the peace of al-Bashir, it is the peace of the Sudanese people."
In the Hillcrest Hotel in Nairobi on New Year's Day 2003 there was a meeting between the SPLA and the Fur people. Garang asked two associates of Abdul Wahid al Nur (who later formed the Sudan Liberation Movement) to declare that the Fur people were with the SPLA – they refused