The event on Tuesday comes just over a month after the terrorist attacks in Brussels which killed 32 people and those in Paris in November.
In his opening remarks, Antonio Tajani, Parliament's Vice President responsible for inter-religious dialogue, said it is important to conduct work in difficult neighbourhoods of large cities "where social discontent can be exploited by terrorist organisations who wish to proselytise."
He said, "Whoever shoots in the name of God, shoots against God."
The Italian MEP and former Commissioner also highlighted the efforts of Muslims in Europe to end radicalisation: "Many victims of extremist violence and Islamic extremist terrorism are Muslims themselves: we must unite our forces and denounce all forms of violence claiming a religious justification. I am convinced that Muslim communities in Europe and the world share this view. The goal of this conference is to give them a chance to show their opposition to radicalism and hatred."
"Many of these terrorists were born and grew up here in Europe. De-radicalisation is not about cutting roots but rather going back to those roots. Europe needs to build on inter-religious dialogue. We have to fight against preachers who preach hate.
"Being a Muslim is about belonging to a religion which asks followers to respect one another, to respect others. The ISIS criminals are hijacking the religion," he said, adding that de-radicalisation will "help stop the suffering of Muslim mothers."
"Europe and its social values must build on inter-religious dialogue, on coexistence, living together, on common progress to see the flourishing of values. The key word is dialogue.
"Whether we are Christians, Jews or Muslims we need to look in each other's eyes. Unfortunately the ones who kill in the name of God do not look other men and women in the eyes."
Speaking ahead of the conference, Parliament President Martin Schulz said: "Terrorism and radicalisation must be fought through prevention, monitoring, intelligence-gathering and updated rules and sanctions. There is one tool however which beats radicalisation before it even takes place: dialogue.
"We are increasingly witnessing the phenomenon not just of segregated societies, but of more and more segregated and alienated lives. Dialogue helps cure this sad reality."
Further comment came from UK Tory MEP Syed Kamall who said: "I welcome this initiative by the European Parliament to bring together people with expertise in tackling radicalisation at the international level, the national level and at a grassroots level. We can all learn from each other - both Muslims and people of other faiths or no faith.
"The challenge of preventing radicalisation among young people is not one that we can solve overnight, or with a proverbial 'silver bullet'. Instead we need to look at the root causes for most radicalisation, such as identity issues or grievances, and tackle them in our local communities."
Panellist Tokia Saïfi, a French EPP member and Vice-Chair of the delegation for relations with Maghreb countries, opened the first session.
She noted that "radicalisation is a complex phenomenon that can be difficult to understand from the outside."
She added that "it is very important to take note of the role of women when it comes to de-radicalisation" and that "the first combat against radicalisation should be within the immediate family."
Malika Hamidi, director-general of the European Muslim Network, said, "Muslim women are finally being considered today as vital actors in the process of de-radicalisation. It is important to involve them as they are generally the first to notice signs of radicalisation."
She pointed out that "80-90 per cent of young people leaving to Syria only found an interest in religion in the months immediately prior to their departure."
She also noted the importance of working in prisons to combat radicalisation.
Further contribution came from Latifa Ibn Ziaten, founder of the Imad Ibn Ziaten Youth Association for Peace, who told the conference, "Education plays an important role, it has been neglected. Parents are the mainstays. With children playing in the streets, it's important to be vigilant with them. We need to give importance to children, they need to be listened to."
Ending the first panel on the role of women, the Chair of Parliament's gender equality committee, Iratxe García Pérez, said: "The work of de-radicalisation is done in silence but is very precious."
She added: "It is vital to use exactly the same tools as Islamic State in creating counter-arguments. The best weapon to use is exactly the same weapon that Da'esh uses to recruit young people: these tools are Whatsapp, Facebook, Instagram and Twitter. It is essential that security services use the same channels to try to convince hopeless young people not to give up their life in Europe to join IS."