In a statement, the star's family said he passed away at home following a short illness.
In a career spanning six decades, Swift was best known for playing Richard Bucket, the long-suffering husband of Hyacinth Bucket, in the classic comedy.
The BBC series mocked British class obsessions and ran for five years in the 1990s, with Swift appearing in 42 episodes.
Dame Patricia Routledge, who played his class-obsessed wife, became known for her pronunciation of her surname (it's Bouquet).
After the show finished Swift continued to have an active career, last being credited with an on-screen appearance in 2017.
He began his career with a number of small roles in the late 1960s, moving into his first television roles in the 1970s.
Swift appeared in the 2007 Doctor Who Christmas special.
He had a daughter Rebecca, who died in 2017. Swift is survived by two sons - Adam, a university academic, and Joe, a garden designer