European Parliament President Martin Schulz told the packed conference, "If we do not want Europe to destroy itself, then we all need to stand together, politicians and religious leaders.
"If we win the fight for the hearts of our citizens, if we succeed in pushing back hate, then we still have a chance in saving the soul of Europe."
Parliament Vice-President Antonio Tajani, an Italian member of the EPP group, said he was concerned about the decline in the number of Jews in Europe, from two million in 1991 to 1.4 million in 2010.
He also said he regretted the attacks they are subject to: "Jewish people should be able to live in peace in Europe, respected like anyone else. They should be able to display their faith, their identity, without being attacked."
They were among MEPs and representatives from Jewish communities who gathered in Parliament to debate anti-Semitism and the future of Jews in Europe.
Several participants voiced concern about the future of the Jewish communities in Europe.
"Jews are confronted with insults, discrimination, and harassment, sometimes physical violence, sometimes they are murdered like in Paris, Brussels or Amsterdam," said another speaker, Francis Kalifat.
Another speaker, Pinchas Goldschmidt, President of the European Rabbis Conference, identified two main threats - radical Islam and Islamic terrorism - as well as the rise of extreme right in Europe. The majority of the Jewish participants at the conference also criticised calls to boycott products from Israel, and said anti-Zionism was the new face of anti-Semitism.
"We make a great mistake if we think anti-Semitism is a threat only to Jews. It is a threat, first and foremost, to Europe and to the freedoms it took centuries to achieve," said Jonathan Sacks, the Chief Rabbi of the United Hebrew Congregations of the Commonwealth.
"No society that has fostered anti-Semitism has ever sustained liberty or human rights or religious freedom. Every society driven by hate begins by seeking to destroy its enemies, but ends by destroying itself."
Some cited reasons for cautious optimism, including Brussels Chief Rabbi Albert Guigui, who said, "It is painful to hear that many Jews have the feeling that they can't live any longer, or study, work, pray safely on European soil.
"But yes, there is a future for Jews in Europe. Friendships between different religions are increasing and they are strong and solid."
French philosopher Bernard-Henri Levy agreed, telling the event, "I don't think the situation is as dramatic and tragic as some people make it out to be.
"In 1930 the Jews were alone," he said, "Today Jews do have allies." Levy added: "I don't know of any European county or institutions which display institutional anti-Semitism."