US President Barack Obama and the UN Secretary General Ban Ki-moon will be among those paying tribute.
The service will be held at the 95,000 capacity FNB stadium, where Mr Mandela made his last public appearance. It will also be shown on big screens at three “overflow” stadiums.
The former South African president died aged 95 last Thursday.
‘A mighty life’
The memorial service is one of the biggest gatherings of international dignitaries in recent years.
Many people stood in the rain waiting for several hours to get into the stadium, the BBC’s Pumza Fihlani reports from the scene.
She says the crowds are in high spirits – singing and dancing, stomping their feet – and the stadium is beginning to have the feel of a political rally.
One of those attending, Shahida Rowe, from Johannesburg, told the BBC: “The core of Mandela’s life was humanity. That is why I am here today and the world is celebrating. Thanks to him, I was recognised as a human being.”
The crowds will be addressed by South African President Jacob Zuma and there will be tributes from other heads of state, including President Obama, Brazilian President Dilma Rousseff and Cuban President Raul Castro.
Under Mr Castro’s brother, Fidel, Cuba was a staunch critic of apartheid, and Mr Mandela had expressed gratitude for his support.