Of all the shuttered buildings in Baltimore, there’s one kind that makes April Latimore especially angry.

“We need someone who’s going to open up those recreation centers,” said Latimore, 35, as she stood in Freddie Gray’s old neighborhood during a rally on Saturday. “The money doesn’t trickle down. We’ve been crying out this whole time, and it hasn’t happened.”

Facing serious budget shortfalls, the Baltimore city government has offloaded 14 youth rec centers since Mayor Stephanie Rawlings-Blake took office in 2010. While private groups and the school system have taken over ten of the centers, four have closed entirely—all of them in West Baltimore. During the street protests last week, the rec centers emerged as a focal point for residents who see them as a blatant sign of neglect and misplaced priorities.

But the forces that prompted the city to pull funding from its rec centers also reveal why it will be so challenging for Baltimore to move forward, despite new calls for dramatic change. Both the city of Baltimore and the Maryland state government have been grappling with budget shortfalls that will make it harder for officials to turn their promises of sweeping change into a reality. Rawlings-Blake inherited a massive budget deficit when she came into office, prompting austerity measures that included not only public pension cuts and deferred infrastructure projects, but also reductions to “building upkeep, prisoner re-entry and mentoring programs, Arts and Culture funding” and other city services in Fiscal Year 2013, according to an outside report commissioned by the mayor.

 “Because of budget constraints, we couldn’t fund [the rec centers] at the level that they needed to be funded,” said Kevin Harris, a spokesman for Rawlings-Blake. Though the city’s fiscal health has improved under her watch—earning the city its highest bond rating in years—Baltimore is still struggling to expand its tax base after decades of depopulation, putting serious fiscal constraints on new spending or tax breaks. 

Put another way, Baltimore’s decades-long economic decline has made it more difficult for the city to help itself. And the recent recession only made things worse. 

Officials promise that change is on its way. “What you are going to see is more empathy,” state Senate Majority Leader Catherine Pugh told me. “Youll see more reinvestment in our comunity.” But any public spending increases or tax cuts will require more revenue at a time when both the city and state are still scrounging for cash. Rawlings-Blake has faced a budget shortfall every year that she’s been in office. Maryland’s new GOP Gov. Larry Hogan inherited an estimated budget shortfall of $1.2 billion. Having campaigned on “fiscal responsibility,” Hogan is currently locked in a standoff with the Democratic-controlled legislature over education and health-care spending.

In the wake of last week’s unrest, Baltimore city officials have coalesced around an early step to bring greater economic opportunity to the city’s poorest communities, calling for the city to expand a summer jobs program to 3,000 more young Baltimoreans. But even the most modest calls for change are facing fiscal headwinds: Expanding the jobs program, for example, would take $4.5 million that’s not currently in the city budget. “I will work with the administration to try to identify targeted cuts that could help fund a portion of the additional summer jobs needed,” promised City Council President Bernard Young, appealing for the “Baltimore’s corporate community” to donate money to the project as well.

The mayor’s office insists that fiscal constraints shouldn’t hold back urgent policy reforms. “It’s about priorities—the fact that we are cash-strapped is no reason or no excuse to how to invest in the kind of things that work,” said Harris. Like many cities emerging from the recession, Baltimore has more breathing room these days. The mayor touts the fact that there are no cuts to city services in her latest budget; that property taxes and the structural deficit are lower than when she took office; and that the city has managed to staunch its decades-long loss in population, stabilizing its tax base.

Limited revenue, however, has continued to tie the city’s hands, forcing difficult trade-offs. While it avoids service cuts, the mayor’s latest proposed budget also doesn’t have any property tax relief, frustrating those who believe that the city’s rates—by far the highest in the state—are deterring private investment and growth. “The city needs to add private taxpayers, but it can only do that if it has a competitive tax framework,” said Anirban Basu, who heads an economic and policy consulting firm in Baltimore and served on Hogan’s transition team. And when the city has extended special tax credits to developers, such projects have disproportionately focused largely on revitalized Baltimore’s downtown and Inner Harbor, far from troubled neighborhoods like Sandtown-Winchester. 

Community leaders argue that the money is there if leaders are willing to look for it. “Others have used the excuse of scarcity not to respond. It’s a question of political will. If there is ever a time to answer the call, it’s now,” said Rob English, supervising organizer for BUILD Baltimore, a community development group. “You budget what you value, and if we value uniting Baltimore with Maryland, then corporate leaders and civic leaders will find a way to leverage funding.” Even before last week’s unrest, Rawlings-Blake proposed selling off city properties to provide new funding for the city’s rec centers. Baltimore council members are now calling for less spending on law enforcement as well. (The city’s budget director admitted in December that cutting police overtime would help put a dent in its deficit.)

Conservatives insist that Baltimore’s woes only prove how liberals’ tax-and-spend policies are bankrupt. They point to a $130 million investment project in Sandtown-Winchester that built new homes but ultimately failed to deliver more jobs and opportunities in the neighborhood. Gov. Hogan has yet to lay out any proposal for rebuilding Baltimore, but he’s already rejected the notion that increased spending on education—a top Democratic priority—is the answer. “I can tell you in the week I spent walking about Baltimore City, and talking to more than a thousand people, not a single person mentioned [Geographic Cost of Education Index], or funding formulas,” Hogan said this week, referring to the current education funding fight. “They talked rec centers and more jobs.”

But English, whose group helped build hundreds of homes in Sandtown, believes the neighborhood’s revival fell short not because the money went to waste, but rather because the city failed to invest nearly enough. “This city did not have the political will to invest the type of subsidy that’s needed to crate a market for commercial retail,” said English, comparing the West Baltimore project to the billions that the city has poured into the Inner Harbor.

The tax cuts that Hogan and other Republicans are pushing for carry a price tag, too. Having made tax relief a central part of his campaign, Hogan boasted that the state’s budget this year “broke the streak of 40 consecutive tax hikes” while closing the shortfall. The state legislature, however, rejected most of his proposed tax cuts, arguing that they would take revenue away from critical priorities. “What state roads is he going to 'X' out to send $900 million to the counties?” said Senate President Thomas V. “Mike” Miller, a Democrat.

While the mayor and others are calling for private businesses and foundations to step up, Baltimore is already home to a plethora of major philanthropic groups whose support hasn’t been enough to fill the gap, either. Baltimoreans, moreover, have heard the same promises countless times before, only to see ballyhooed development projects and public-private partnerships fail to turn the city into the next Pittsburgh. Residents felt like they were making history last week, community leaders are now concerned that the new hope and opportunity for change could dissipate through half-measures that may be easier to conceive—and fund. “Yes, young people need summer jobs. But we’re not going to summer-job our way out of the worst unrest since the Sixties in Baltimore,” said English. 

Such constraints have prompted both sides to turn their eyes to Washington. Jay Steinmetz, a Baltimore business owner, rejects the notion that more federal spending is necessary, but believes that federal tax relief would be transformative. “Have the federal government [provide] a subsidized rate for property taxes, and a subsidized rate for business taxes,” said Steinmetz, CEO of supply-chain company Barcoding, calling for Washington to “make the entire city an empowerment zone.” The Baltimore Sun summed up the consensus in an editorial this week.

What's needed is some serious federal investment—whether through direct spending or tax credits—to jump start opportunity, to create jobs, to rebuild schools, to provide affordable housing and child care.

Washington, however, has continued the same austerity that has held Baltimore back. The automatic spending cuts under sequestration have had an outsized impact on Maryland, where the federal government employs about 10 percent of the workforce. Congress has only reversed a portion of the federal cuts so far, and Republican legislators are continuing to resist Democrats’ calls to lift the spending caps. Visiting West Baltimore on Wednesday, Obama cabinet officials promised that federal resources for job training would be coming to their community soon, along the lines of the $5 million grant given to Ferguson, Missouri. But they acknowledged that small-scale interventions wouldn’t be enough. “We need to think about not just a little pilot program, not just small thing[s], but at scale,” said Education Secretary Arne Duncan. But he didn’t elaborate any further as to what that might actually mean. 

The same concerns about cost have swirled around proposed criminal justice reforms like body cameras. But such interventions are relatively inexpensive compared to large-scale economic development projects: A body camera program will cost an estimated $5.5 million to $7.9 million annually, and Baltimore’s mayor has promised to move forward with the initiative by the end of the year. Reducing incarceration and overpolicing will ultimately save everyone money. And some serious changes don’t have any upfront costs at all.

“It doesn’t take money to sit down and look residents in the eye,” said English, “and see who they are and what their story is.”