The 850-year-old Gothic building's spire and roof have collapsed but the main structure, including the two bell towers, has been saved, officials say.
Firefighters are still working to contain the blaze as teams try to salvage the artwork stored inside.
President Emmanuel Macron called it a "terrible tragedy". The cause of the fire is not yet clear.
Officials say it could be linked to the renovation work that began after cracks appeared in the stone, sparking fears the structure could become unstable.
Paris prosecutor's office said it had opened an inquiry into "accidental destruction by fire". A firefighter was seriously injured while tackling the blaze.
Visibly emotional, Mr Macron said the "worst had been avoided" and vowed to launch an international fundraising scheme to rebuild the cathedral.
The fire began at around 18:30 (16:30 GMT) and quickly reached the roof of the cathedral, destroying its stained-glass windows and the wooden interior before toppling the spire.
Some 500 firefighters worked to prevent one of the bell towers from collapsing. More than four hours later, fire chief Jean-Claude Gallet said the main structure had been "saved and preserved" from total destruction.
Sections of the cathedral were under scaffolding as part of the extensive renovations and 16 copper statues had been removed last week.
Deputy Paris Mayor Emmanuel Gregoire said the building had suffered "colossal damages", and teams were working to save the cathedral's remaining artwork.
Historian Camille Pascal told French broadcaster BFMTV that "invaluable heritage" had been destroyed, adding: "Happy and unfortunate events for centuries have been marked by the bells of Notre-Dame. We can be only horrified by what we see".
Thousands of people gathered in the streets around the cathedral, observing the flames in silence. Some could be seen openly weeping, while others sang hymns or said prayers.
Several churches around Paris rang their bells in response to the blaze, which happened as Catholics celebrate
Holy Week.
Because of the fire, Mr Macron cancelled a speech on TV in which he was due to address the street protests that have rocked France for months.
Visiting the scene, the president said the cathedral was a building "for all French people", including those who had never been there.
"We'll rebuild Notre-Dame together", he said as he praised the "extreme courage" and "professionalism" of the firefighters.
No other site represents France quite like Notre-Dame. Its main rival as a national symbol, the Eiffel Tower, is little more than a century old. Notre-Dame has stood tall above Paris since the 1200s.
It has given its name to one of the country's literary masterpieces. Victor Hugo's The Hunchback of Notre-Dame is known to the French simply as Notre-Dame de Paris.
The last time the cathedral suffered major damage was during the French Revolution. It survived two world wars largely unscathed.
Watching such an embodiment of the permanence of a nation burn and its spire collapse is profoundly shocking to any French person.
The church receives almost 13 million visitors each year, more than the Eiffel Tower
A Unesco World Heritage site, it was built in the 12th and 13th centuries
Several statues of the facade of the Catholic cathedral were removed for renovation
The roof, which has been destroyed by the blaze, was made mostly of wood
The Vatican expressed "shock and sadness," adding that it was praying for the French fire services.
Germany's Chancellor Angela Merkel has offered her support to the people of France, calling Notre-Dame a "symbol of French and European culture".
UK Prime Minister Theresa May said in a tweet: "My thoughts are with the people of France tonight and with the emergency services who are fighting the terrible blaze at Notre-Dame cathedral".
Also on Twitter, US President Donald Trump said it was "horrible to watch" the fire and suggested that "flying water tankers" could be used to extinguish the blaze.
In an apparent response, the French Civil Security service said that was not an option as it might result in the collapse of the entire building.