The refusal by Ralph Northam, the Democratic governor of Virginia, to resign after the revelation of a racist photograph is threatening his party’s political fortunes in Virginia, where Democrats are on the brink of consolidating power after a decade-long rise in the once-conservative state.

With Mr. Northam’s turmoil erupting during a legislative session in an election year, Democrats and Republicans said Sunday that his fragile hold on power risked his party’s policy ambitions and its aspirations for this fall, when control of both the state’s legislative chambers is expected to be bitterly and closely contested.

“You can’t govern without a mandate, and all you’re going to do is make things worse for the state,” said Representative A. Donald McEachin, a Democrat who served alongside Mr. Northam in the Virginia Senate.

Mr. Northam met with some of his staff members on Sunday night, prompting speculation that he might announce his resignation during the Super Bowl. Most of the people he met with told him that resigning was the way to clear his name, according to a state Democrat briefed on the meeting by an attendee.

Both chambers of the Legislature are scheduled to meet on Monday morning for sessions that could bring fresh condemnations of the governor. As of Sunday evening, Lt. Gov. Justin Fairfax, who would succeed Mr. Northam if he resigned, had not been notified that the governor was stepping down.

Mr. Northam’s troubles began on Friday with the surfacing of a photograph on his medical school yearbook page, which showed a person in blackface posing with another in a Ku Klux Klan robe. The governor at first acknowledged that he was one of the figures in the image, and then denied it on Saturday, all while drawing widespread calls for his resignation. Until this episode, Democrats appeared to be on a steady roll in Virginia, a state that had increasingly become a source of strength for the party in major elections.

[Read: Ralph Northam rose to power quickly. His fall might be even faster.]

Since 2008, when Barack Obama became the first Democratic presidential candidate in more than four decades to carry the state, Virginia has shifted steadily leftward. For the last decade, both of the state’s senators in Washington have been Democrats. And more recently, the party has gained greater sway at the Capitol in Richmond.

Two years ago, Democrats picked up 15 seats in the House of Delegates, where they had been locked out of the majority for more than two decades. They are now two seats away from control in both chambers. The biggest prize in controlling the statehouse would be the power, under current law, to draw congressional and legislative districts after the 2020 census.

More power in the Legislature has already translated into significant policy wins for Democrats. Since Mr. Northam was elected in 2017, the party has achieved long-prized goals, like the expansion of Medicaid, and seized new credit for the state’s economic growth.

And this week is arguably among the most crucial of the year’s 46-day legislative session, with an important deadline for bills to advance. The speaker of the House of Delegates, Kirk Cox, and other Republican legislators warned that Mr. Northam’s “ability to lead and govern is permanently impaired.”

Even to his Democratic allies, Mr. Northam now seems hobbled.

“You’ve got to work as one unit to move your commonwealth forward, and he’s just not going to have that ability to do it,” Terry McAuliffe, a Democrat who preceded Mr. Northam as governor, said on CNN’s “State of the Union.”

Mr. Northam’s difficulties can be traced, in part, to his shifting accounts over the photograph, published in a 1984 yearbook for the Eastern Virginia Medical School, which said on Sunday that it would investigate how such “unacceptable photos” came to be published.

First, Mr. Northam apologized Friday night “for the decision I made to appear as I did in this photo.” Then, on Saturday, he declared with certainty that neither person in the photograph was him. But he also acknowledged a separate episode in which he darkened his face with shoe polish for a Michael Jackson costume at a dance contest in 1984.

Mr. Northam maintained over the weekend that, for the time being, he commanded enough influence to govern the state of about 8.5 million people.

“I plan to continue to lead,” he said. “If we get to the point where we feel that we’re not effective, that we’re not efficient, not only for our caucuses, but the Commonwealth of Virginia, then we will revisit this and make decisions.”