Chatting to guests at a reception to mark the monarch's 90th birthday at London's Guildhall, Harry said that "a day off" is what the Queen really needed.
The Prince was speaking to Grenada High Commissioner Karl Hood when he spoke about the monarch.
Prince Harry, Kate and Prince William after the annual thanksgiving service at St Paul's
"He said the best birthday present would be to have a day off, so she could lie about and do nothing," said the High Commissioner.
Acting High Commissioner for Dominica, Janet Charles, added that Harry said the Queen should be relaxing, but that she had wanted to attend the annual thanksgiving service at St Paul's for the public.
"He was talking about his grandmother. He said she should really be putting her feet up but she wanted to be here for the people," said Janet. "He said he hopes there's many more years of that to come."
"He's such a charming person," said a guest at the reception
She added of the 31-year-old Prince: "He's such a charming person. I asked him if I could have a photograph with him, he said he couldn't as he had to obey the rules. But he said 'I'm sure we'll meet again'."
Harry was attending the reception with his brother Prince William and William's wife the Duchess of Cambridge. The trio mingled with guests who had attended the National Service of Thanksgiving at St Paul's earlier that morning.
The ceremony, which was attended by the Queen and a 53-strong royal party, was held to celebrate the monarch's life and featured a series of prayers, hymns, readings and musical performances. It marked the beginning of a weekend of national and Commonwealth celebrations.
The Queen hosted a lunch for visiting Governors-General at Buckingham Palace
After the service the Queen, Prince Philip – who was celebrating his 95th birthday – Prince Charles and the Duchess of Cornwall headed off to a different engagement while William, Harry and Kate attended their reception.
The Queen hosted a lunch for visiting Governors-General at Buckingham Palace, but before sitting down for lunch they gathered for a photograph.