The actor, 79, who has been married to Fawlty Towers star Prunella for 50 years, said that the woman he loved "doesn't really exist anymore". She has a "slight" condition similar to a mild form of Alzheimers, but still has her sense of humour, he said.
Timothy talks about his wife's condition in a new more More4 documentary, Grand Canal Journeys. The couple have been navigating Britain's canals on a slow boat, after discovering a passion for the hobby in the 1970s.
He tells the programme, during which the pair embark on four canal journeys across Britain, that the outings are perfect for Prunella, because of her struggles with her memory.
"She can't remember things very well, but you don't have to remember things on a canal," he said. "You can just enjoy things as they happen - so it's perfect for her."
Speaking of her condition, he added, "When you think back over that long period and you think this person that I loved and enjoyed this and that and the other with doesn't really exist anymore, it's another person… that's quite painful.
"But you mustn't think like that, you must take it from day to day, you notice how much she's able to do and how much she enjoys life."
Prunella admitted that the condition could be a "nuisance sometimes", but said she was determined to make the most out of life "while we still can".
She told the Radio Times she would eventually like to "die on the eight curtain call," adding, "Eight will mean the show's been rather a success. I just hope I'm somewhere near the middle and have been reasonably good in the part."
The couple will be joined by their two sons, Joe and Sam, later on in the series, which will be broadcast on More 4 on 10 March at 9pm.