The non-binding "Florianopolis Declaration" sees whaling as no longer being a necessary economic activity.
The proposal was backed by 40 members with 27 voting against.
Pro-whaling states, including Japan, Norway and Iceland, rejected the resolution.
Instead, they are backing a Japanese counter-proposal that envisages "co-existence" between commercial whaling and conservation.
"Science is clear: there are certain species of whales whose population is healthy enough to be harvested sustainably," the Japanese proposal said.
But environmental campaigners welcomed the commission's decision.
"Instead of the archaic and completely unnecessary hunting of whales, the protection and peaceful and purely non-lethal usage of whales, which includes whale watching, should now be the focus of our efforts," said Nicolas Entrup of Swiss-based NGO OceanCare.
He hailed the decision as "a manifesto for peaceful co-existence between whales and
humans".
Patrick Ramage, whale programme director of the International Fund for Animal Welfare, described the declaration as "a big win for whales".
On Wednesday, pro-whaling nations at the IWC's biennial meeting blocked an attempt to create a whale sanctuary in the South Atlantic.
Brazil's Environment Minister Edson Duarte, whose country proposed the sanctuary, said he was "disappointed" but would not be deterred.
The IWC already recognises two whaling sanctuaries - one in the Indian Ocean and the other in the waters of the Southern Ocean around Antarctica.
In 1986 it also agreed to a moratorium on hunting, which eventually became a quasi-permanent ban.
But by using an exception in the ban that allows whaling for scientific purposes, Japan has still killed between 200 and 1,200 whales every year since, including young and pregnant animals.
Commercial whaling in the 19th and early 20th Century brought the giant mammals to the brink of extinction.