In the final scene of Upstairs, Downstairs, broadcast on PBS in the spring of 1977, the head house parlor maid Rose wanders through the empty townhouse at 165 Eaton Place, home to the Bellamy family since 1903. The year is now 1930. Mrs. Bellamy has died on the Lusitania, Mr. Bellamy has remarried, servants have come and gone, and Rose has remained devoted to the house she cleaned and the family she served. 

The home is being sold after the loss of the Bellamy fortune; the son James, killed himself after a catastrophic loss in the 1929 crash. The surviving family members are retrenching to the countryside; they’ve invited Rose to live with them and continue to serve them. 165 Eaton Place will have new owners; the townhouse is a prime piece of real estate for the few who haven’t suffered in the crash. Rose considers each bare room, the ghosts contained by each, and then walks out the front door, the master’s entrance. No one on the street pays her any notice as she walks away.

The final episode of Downton Abbey would have its audience forget any woes, past, present or future: Everyone in the family, Lady Grantham reminds us, is finally, blissfully happy. And of troubles ahead, she considers them easily met: “I think the more adaptable we are, the more chance we have of getting through.”

But the ghosts of Downton Abbey began to weigh on us in its final season. The Crawleys, who we met in the spring of 1912 and leave in the winter of 1926, have suffered in their own way, closing off rooms in the house no longer in use, dismissing servants who are no longer needed. In the final season, when the Abbey is opened for its first charity house tour—a hint of what these grand mansions will serve for future generations—Lady Grantham puzzles over a blank heraldry on the fireplace. She’d never noticed it wasn’t filled in, and it becomes clear the family history the Granthams have been fighting for all these years is known only to those who have served them.

The series, as conceived by Julian Fellowes, originally had a three-season arc, but popular demand stretched it into an ungainly six, and the once-tight plot spun out into tatters. With Mad Men, Matthew Weiner was able to convince us that after seven seasons Don Draper was being led down a fateful path—towards enlightenment, as it turned out. But Downton Abbey lived out its drama after three seasons in a mêlée of death and destruction, culminating with the death of the heir of Grantham, Matthew Crawley, in an untimely car crash. What happened next was extraordinary: Everyone simply went on living.

In his failure to deliver a cohesive six seasons of television, what Fellowes gave us instead was time. The exquisite boredom of the second half of Downton Abbey approaches the provincialism of George Eliot’s Middlemarch. World wars give way to petty battles between mothers and sisters, between servants and their own limited prospects. Policemen come and go with the promise of drama, only to disappoint; suitors arrive and leave, financial problems arise and are cleverly solved, a telephone is installed, a dog dies, a new dog is born. As the years wear on, the characters of Downton Abbey softly bounce around their upholstered rooms, coming together, drifting apart, lightly crashing together when necessary.

But it turns out the dead of Downton don’t really leave us. Scullery maid Daisy sets up the army photograph of William, who she was married to for just a few hours before he died, on a bookshelf in his father’s new farm. She’s much older than William now, preparing to leave service after years as the lowest member of staff. William’s father has become her father, a reminder that the ones who change our lives are often the ones who come in at an angle, dashing in and then dashing out. 

In the final episodes, Tom attempts to reconcile Edith and Mary with the memory of their sister, his dead wife Sybil. But then Tom himself has been little more than ghost in the Downton household, his presence a constant reminder of loss. He drifts from table to table, party to party, and finds a friend in Henry Talbot, who shares an interest in cars. Henry’s best friend has recently died too, killed in a racing accident. Love becomes a form of shared loss for Mary, visiting Matthew’s grave to give herself permission to marry her final, imperfect beau. Fellowes is determined to smash everyone together in the end, in a finale of almost orgiastic happiness. 

As we prepare to leave them, how many people are buried in the Downton churchyard? Sybil is there, with Matthew, and William, and with the coming war, who will join them? The churchyard will soon be more peopled than the Abbey. Matthew’s former fiancée Lavinia is buried there—there was no time to return her body home after her sudden death from influenza in 1918. Her father requested his own ashes should be buried next to his daughter, and Matthew fulfilled that request the following year. Seven years later, as the Crawleys are left to struggle with their future—the house, the title, the finances—their churchyard is home to this forgotten widower father and his daughter, snuffed out by the twentieth century. Lavinia, so perfect and good, so dull, so kind, died a convenient death. But like so many of the deaths in Downton Abbey, her death allowed others to live, and her place in the churchyard is reminiscent of the final lines of Middlemarch:

Her full nature, like that river of which Cyrus broke the strength, spent itself in channels which had no great name on the earth. But the effect of her being on those around her was incalculably diffusive: for the growing good of the world is partly dependent on unhistoric acts; and that things are not so ill with you and me as they might have been, is half owing to the number who lived faithfully a hidden life, and rest in unvisited tombs.