Dressed immaculately in a navy suit and matching blue tie, Harry, 29, was spotted arriving at Westminster Abbey to represent his 87-year-old grandmother.
Almost 2,000 people attended the service, including David Cameron, Archbishop Desmond Tutu and Idris Elba, who played Mr Mandela in the biopic Mandela: Long Walk to Freedom.
Prince Harry and the Dean of Westminster
Harry was pictured walking into the abbey alongside the Dean of Westminster, the Very Reverend Dr John Hall, who conducted the ceremony. Earlier on in the day, Dr Hall announced that a memorial stone to honour Mr Mandela would be placed in the abbey.
The former South African statesman, who was the nation's anti-apartheid hero and first black president, passed away aged 95 last December.
Archbishop Tutu spoke of Mr Mandela's influence and legacy, telling the congregation that the late leader, who had spent 27 years in prison, had gained worldwide respect and admiration.
"He made us believe that all, each one of us, we are made of goodness," said the archbishop. "We are made for caring, we are made for loving, we are made for compassion, we are made for laughter, for peace.
"His remarkable constancy under suffering stands as an example to everyone. In addition, his capacity for forgiveness and his generosity of spirit show what humanity at its best can achieve."
The late Nelson Mandela
Archbishop Tutu then went on to thank those who had supported Mr Mandela's cause.
"Without the anti-apartheid movement, without you extraordinary human beings, Mandela could so easily have died in prison," he said. "Wonderfully, exhilaratingly, the entire world glued to its TV sets watched as this man emerged from 27 years of incarceration."
Mr Mandela visited the abbey in July 1996 during a state visit, when he laid a wreath at the Grave of the Unknown Warrior.
The Duchess and Duke of Cambridge
As news broke of his death last December, Prince William and Kate Middleton were attending the premiere of Mandela: Long Walk to Freedom. The royal couple were overcome with emotion and speaking to reporters in the cinema lobby, William paid tribute to the late statesman.
"It is extremely sad and tragic news," said William. "We were just reminded of what an extraordinary and inspiring man Nelson Mandela was. My thoughts and prayers are with him and his family."