Israeli forces fired tear gas and detained a number of Palestinians on Sunday who were attending prayers after protests erupted in the mosque compound following an announcement that Jews would enter the holy site.
It was the first time in three decades that Jews were allowed into the site during the final days of the fasting month of Ramadan, which coincided this year with the Israeli national holiday commemorating control over the city.
Earlier in the morning, Israeli police had deployed hundreds of their forces around the mosque compound and across the city as hundreds of Jews waited at the compound gates to enter.
Reporting from West Jerusalem, Al Jazeera's Harry Fawcett said: "What had been announced was that Jews would not be allowed into the Al-Aqsa Mosque compound because of the sensitivity coming up to the end of Ramadan."
"But what actually took place was that hundreds of what we would expect to be right-wing settler and religious nationalists assembled at the gate demanding entrance.
"Once the police had apparently decided they would be allowed in, protests by Palestinians started and forces moved in to put down the demonstrations. At that point, the Jewish settlers were allowed to come in," said Fawcett.
Video inside the compound showed Palestinians shouting "Allahu Akbar" (God is great), and throwing chairs, and other objects at a doorway before tear gas and stun bombs were thrown at them.
Thousands of Jews have also flocked to the nearby Western Wall to mark the occasion ahead of celebrations and marches that will be held around the city in the afternoon and evening.
The annual march through the city, including the Muslim quarter of the Old City forcing Palestinians to shutter their businesses, often sparks clashes.
Security measures have been tightened and police are on high alert after a Palestinian teenager was shot dead by Israeli forces as he tried to enter occupied East Jerusalem to offer Friday prayers in the Al-Aqsa Mosque compound.
In a separate incident on Friday, a Palestinian man was shot dead by Israeli police after an alleged stabbing attack on two Israelis in the Old City of occupied East Jerusalem.
The Al-Aqsa Mosque compound, referred to as the Temple Mount by Jews, is the third holiest site for Muslims.
The area has always been closed to Jews during the last 10 days of Ramadan, when large numbers of Muslim worshipers stay at the site.
"The police really aren't giving any explanation as to why they made this decision," Al Jazeera's Fawcett said.
"T he police spokeman is merely saying that access to non-Muslims was allowed today as it is on any normal day" but didn't acknowledge the Israeli holiday or that it was the end of Ramadan.
The last time Jerusalem Day coincided with the end of Ramdan in 1988, the Al-Aqsa Mosque compound was closed to Jews.