Their move is partly backed by Tibor Szanyi, a Hungarian member of Parliament's S&D group, who said he opposes a boycott and would rather see the public cast an "invalid" vote.
This would send a "strong signal" that people "massively disapprove" of the Hungarian regime's "anti EU and inhumane anti-refugee" policies, he said.
Hungary president Viktor Orbán's administration will hold a referendum on 2 October asking citizens if they wish to take in a set number of refugees imposed upon them by the EU.
On Thursday, Orbán courted yet more controversy when he said the EU should deport all illegal migrants to refugee camps beyond its borders.
"Those who came illegally must be rounded up and shipped out," he told Hungarian website Origo.
He suggested they could be brought to either an "island" or "a coastal area in north Africa."
Opposition groups have called for voters to stay at home to annul the result of the referendum next month, despite the government insisting the referendum is not xenophobic.
A coalition of 22 non-governmental organisations has signed a statement calling on Hungarians to shun the plebiscite, as it "does not allow the promotion of our common values, has no sense and is inhumane."
Szanyi said the message Orbán wants to send is that "it will be Hungarians, not Brussels" who decides on the issue.
He asked, "Is it a step towards Huxit? Orbán likes to play the Brussels card whenever a scapegoat is needed but to actually lead the country out of the EU would be a real political crime."
But opinion polls show just a third of Hungarians still have sympathy with refugees fleeing conflict.
Government spokesperson Zoltan Kovacs said the referendum is about autonomy of deciding who can move to Hungary, adding it was "not xenophobia but common sense".
Orbán has previously criticised the EU scheme as a bid to "redraw Europe's cultural and religious identity."
Under the plans, Hungary would be forced to take in 1294 refugees as part of 160,000 people being resettled across Europe.
But the government has said the vote is the only way in which Hungarians have to express their opinion on such an important issue.
If Hungary refuses to accept the migrants, it must pay a fine of €246,000 for each refugee it refuses.