Outside the city of Tarhuna, amidst sprawling power lines and rumbling dump trucks, about fifty Libyan rebels are sitting on straw mats nibbling on tuna and smoking cheap cigarettes in a half-completed house. They are biding their time as their commanders negotiate with forces loyal to former leader Muammar al-Qaddafi in the city of Bani Walid some forty miles away. But there is little indication that the loyalists are going to surrender anytime soon.

With the collapse of the Qaddafi regime, the loyalist cause may seem hopeless. But it should come as no surprise that Bani Walid, and other towns like it, such as Sirte and Sebha, have so far refused to embrace the rebels now in control of Libya. Qaddafi meticulously cultivated certain tribes and regions throughout his forty-two years in power: though his regime has collapsed, the loyalty he engendered was strong enough to live on.

The relationship between Qaddafi’s tribe, known as the Qaddadfa, and the Warfalla clan began long before his birth. In the 19th century, the Qaddadfa were expelled from eastern Libya and settled in the region around Sirte. A small clan, they were forced to rely on large tribes such as the Warfalla to protect them. Qaddafi assumed power in the country in 1969, but it was only after a failed coup attempt against him in 1975—a plot hatched by military officers from Misrata, a city toward which Qaddafi, who studied there, previously had a sentimental attachment—that he sought to shore up his regime’s security by leveraging the historical alliance between the Qaddadfa and the Warfalla clans.

The Warfalla were only happy to oblige him. Months after the 1975 coup attempt, when Qaddafi visited Bani Walid, Warfalla elders slaughtered sheep in his honor and presented him with a gold sword. Qaddafi celebrated the occasion by demonstrating his equestrian skills. His pact with the tribe sealed by the ceremony, he recruited its members to staff the most sensitive positions in his security establishment. That is not to say that the relationship between Qaddafi and the Warfalla has always been smooth—Warfalli military officers tried to overthrow him in 1993—but the ties have endured.

Yet ever since the revolution erupted across Libya on February 15th, rebel leaders have suggested that the Warfalla tribe, which resides in Bani Walid, support their cause. They highlighted that their number two official, Mahmud Jibril, is from the tribe. They noted that the first “martyr of the revolution” is a member. And they boasted that one of two fighter pilots who defected to Malta rather than bomb Libyan civilians was a Warafalli.

Today, the rebels continue to propagate this farce of broad-based support in loyalist cities. Abusif Ghynah, a member of the rebel-created Bani Walid council, told journalists outside Tarhuna that the city’s residents no longer support the deposed Libyan leader. He said three Qaddafi loyalists are holding the town in check by providing weapons and cars in exchange for loyalty. Mahmud Abd al-Aziz, another rebel spokesman, said there were only one hundred Qaddafi supporters in the town.

But the reality appears to belie their claim. According to Mustafa Misbah, a rebel fighter standing beyond earshot of Abd al-Aziz, Bani Walid residents believed Qaddafi’s claim that the rebels were foreign terrorists allied with al-Qaeda. And Warfalli leaders remain faithful to Qaddafi despite the rebels’ fervent wishes to the contrary.

When asked who rebel leaders were negotiating with to end the standoff, Abd al-Aziz responded vaguely, saying “city leaders” were leading the discussions. In the Arab world, it is often tribal elders, known as sheikhs, who negotiate such parleys. But when asked whether Warfalla sheikhs such as Abdallah al-Mukhit were involved, Abd al-Aziz was curt. “We don’t negotiate with such people. Their future is for the court to decide.” As a result, it is unclear whom the rebels are dealing with and whom these leaders speak for.

In other Qaddafi strongholds, such as the southern city of Sebha, the story is much the same. The Libyan leader once called the town “the cradle of the revolution” because he studied there from 1957 to 1961. Much like he did with the Warfalla, Qaddafi drew on the Maqhraha tribe that preponderates in the region to fill his security ranks. He lavished the city with investment and modernized what had been a backwater desert outpost.

The city’s residents have not forgotten his largesse. Qaddafi loyalists from Sebha who were captured by rebel forces in Tripoli told me that the former leader retained the town’s support throughout the revolution despite fuel and power shortages. And much like in Bani Walid, Sebha residents were quick to embrace the Qaddafi canard that foreign jihadists had invaded the country. Today the city has still not embraced the revolution that other parts of the country welcomed so eagerly.

Pictures of the rebels celebrating in Tripoli have deceived the world into believing that the six month revolution has come to an end. But with Qaddafi still retaining the loyalty of the regions he favored for four decades, the war will continue for a while to come.

Barak Barfi is a research fellow with the New America Foundation.