The US-educated veteran diplomat helped craft the pact that curbed Iran's nuclear program in return for sanctions relief.
"I consider you, as put by the leader, to be "trustworthy, courageous and pious" and in the forefront of resistance against America's all-out pressure", he added.
Rouhani and Zarif, who are considered relative moderates, have been facing pressure from Iranian hardliners as the deal they championed has unraveled.
The 59-year old Mr Zarif, who has served as Mr Rouhani's foreign minister since 2013, has faced constant pressure and criticism from hardliners in the country as well as from the worldwide community and the United States. An EU source told Euronews yesterday that the bloc's foreign policy supremo Federica Mogherini hoped that Zarif's resignation would be rejected.
In an Instagram post on Wednesday, Mr Zarif shared an image of the Iranian flag and thanked the people and the leadership for their support in him. "As a modest servant I have never had any concern but elevating the foreign policy and the status of the foreign ministry", he added in an Instagram post. However, it came after Zarif was not present for a meeting with Syrian President Bashar al-Assad on Monday.
After Rouhani's announcement, the semi-official ISNA news agency reported that Zarif had attended a ceremony to welcome Armenia's prime minister to Tehran.
"Mr Zarif is in charge of the foreign policy of the Islamic Republic, and has always been supported by the top officials, especially the supreme leader", Qassem Soleimani, head of the Quds Force, a branch of the Revolutionary Guards, was quoted as saying by Fars news agency.
President Hassan Rouhani hasn't yet accepted Zarif's resignation.
His ready smile and mastery of both the English language and social media has made him a formidable diplomatic rival.
On Tuesday, Israeli Prime Minister Benjamin Netanyahu, long a critic of Iran, welcomed his departure. "Zarif is gone, good riddance". One of his allies said his resignation was motivated by criticism of the nuclear agreement, which has come under increasingly intense fire in Iran since the United States abandoned it past year.