I just read a fic where a character in the Regency period is reflecting on how she'd never imagined she'd be able to have a full-time nursemaid.

Me: Girl, you are and have always been wealthy and you live in an age where labor is CHEAP.

And by "cheap" I mean "so cheap that servants had servants." (Upper level servants in large houses would often have lower-level servants assigned to them as a perk of the job.) So cheap that the gap between "people who were servants" and "people who had servants" was very narrow, and often crossed over the course of a person's lifetime. It was fairly common for working class/poor girls to work as maids for a few years saving up money before they got married, and if they married a reasonably prosperous farmer they would probably be able to afford to hire a maid themselves in good years. (Not "maid" as in "a personal servant to wait on you hand and food," this is "maid" as in "someone to do the nastier/harder bits of cooking and cleaning.")

By 1795, the price of wages for a day laborer was pegged to the price of bread. A gallon loaf weighed 8lbs 11oz, and was theoretically enough to feed a person for a week. Laborers were supposed to make at least three times the cost of a gallon loaf per week, so that if a gallon loaf cost 1 shilling they should be paid at least 3 shillings per week. That is peanuts. For comparison: A pair of wool stockings in the Regency era cost about 2 shillings 6 pence. In other words, a day laborer was paid only a little more per week than the cost of a good pair of socks. Silk stockings--the kind you would wear to a ball--were 12 shillings, or four times the weekly wages of a day laborer.

Combine this with how labor-intensive even the most basic tasks were, and it meant that anybody who could afford servants had them, and anybody above the poverty line could afford them.

Over the course of the 19th Century, the cost of wages relative to the cost of other things rose dramatically, so people had fewer servants and fewer people could afford to have servants. And still, Agatha Christie remembered that when she was young "I couldn’t imagine being too poor to afford servants, nor so rich as to be able to afford a car." She did not grow up wealthy, she grew up middle class. Even in 1900, your average middle-class person in England could not imagine being too poor to afford servants.

This changed radically over the course of the 20th Century; now a middle class person might have a cleaner who comes in once a week, but they definitely will not be able to afford a full-time servant. You have to be wealthy to afford that. So we assume that servants are a mark of huge wealth even in historical periods, when they just ... weren't. This is not helped by the fact that novels set in period times (whether written then or later) rarely mention the servants, so you can read, say, an Austen novel and not have any clue what sort of servants they have. But unless you have researched the issue, it's best to assume they have more servants than you think they had.