When President Donald Trump’s plans for a White House ballroom resurfaced, longtime chief usher Gary Walters offered an account that those entertainment-space constraints are not unique to the current administration. Walters, who served as the executive residence’s top aide for more than two decades and oversaw projects and operations in the residence, said the issue of accommodating large guest lists came up repeatedly with the presidents he supported.

Walters said that when he was chief usher, he made discussions about space for entertaining part of his early interactions with incoming presidents, first ladies and their social secretaries. He described the White House’s biggest public rooms as limited in how many people they could accommodate when set up for events like state dinners, and said the presidents he served all raised concerns about the number of guests the building could handle.

Walters said that Trump’s complaint about the lack of room for entertaining fits a pattern he observed during his own time on the job. In an interview with The Associated Press about his recently published memoir, Walters said he had been told by presidents he served that they wanted “some possibility of an enlarged area” for entertaining, adding that the concern had been raised by multiple administrations before Trump entered politics.

The AP reported that Trump began talking about building a White House ballroom years earlier, and that the plan was publicly detailed before his presidency. In July, the White House announced a 90,000-square-foot space on the east side of the complex intended to accommodate 650 seated guests, at an estimated cost of $200 million. The plan later drew further changes as Trump expanded capacity and increased the price estimate.

As Trump’s proposed ballroom shifted, the AP reported that the capacity was increased to 999 people and that, by October, Trump had demolished the two-story East Wing of the White House as part of the work to build the ballroom there. In December, the AP said Trump updated the ballroom’s price tag to $400 million, doubling the original estimate.

Historians and preservationists objected to images of the East Wing being torn down, but Walters said the demolition was not unlike other construction history on the White House campus. He told the AP that there had been ongoing work across the campus over time, ranging from earlier renovations and new facilities to expansions that added an East Wing during World War II for workspace for the first lady, her staff and other White House offices.

“So there’s always been construction going on around the White House,” Walters said, describing earlier projects and subsequent expansions that came with the loss or replacement of existing buildings. While Walters acknowledged it was “a bit jarring to see the East Wing torn down,” he also said he had personal memories tied to the space, including that he met his wife at the White House and that she worked in the East Wing.

Walters said his career at the White House was shaped by an injury when he was young. He became a chief usher after joining the executive residence staff following an Army discharge and a broken ankle that kept him from patrolling in uniform; he was assigned instead to duties in what is now the U.S. Secret Service’s protective workforce lineage.

In the decades that followed, Walters rose through roles in the executive residence. The AP reported that he was promoted to sergeant in 1975, joined the Usher’s Office as an assistant in early 1976, and was elevated to chief usher by President Ronald Reagan a decade later. Walters retired in 2007 after 37 years at the White House, including 21 years as chief usher, and he described his work as involving close interaction with presidents, first ladies and their families.