Jean-Claude Juncker said the lack of a "permanent structure" resulted in money being wasted on missions.
In a showpiece state of the union speech in the European Parliament on Wednesday, he called for closer military cooperation at EU level.
Juncker announced a new European border and coast guard comprising 200 extra border guards and 50 extra vehicles deployed at the Bulgarian external borders from October.
The EU, he told MEPs in Strasbourg, will also propose before the end of the year a European defence fund, "to turbo boost research and innovation."
This would strengthen the European defence industry and help it to innovate, he said.
Europe, said the former Luxembourg Prime Minister, must also do more to defend itself against terrorism.
"Since the Madrid bombing of 2004, there have been more than 30 terrorist attacks in Europe - 14 in the last year alone. More than 600 innocent people died in cities like Paris, Brussels, Nice, or Ansbach," he said.
Over the last decade, the EU, he said, had engaged in over 30 civilian and military missions from Africa to Afghanistan.
"But without a permanent structure we cannot act effectively. Urgent operations are delayed. We have separate headquarters for parallel missions, even when they happen in the same country or city. It is time we had a single headquarters for these operations.
"We should also move towards common military assets, in some cases owned by the EU. And, of course, in full complementarity with Nato."
The Lisbon treaty enables those EU member states to pool their defence capabilities in the form of a permanent structured cooperation, should they wish to do so.
"I think the time to make use of this possibility is now," Juncker declared.
"The business case is clear. The lack of cooperation in defence matters costs Europe between €25bn and €100bn per year, depending on the areas concerned. We could use that money for so much more."
Some MEPs were critical, however, including Geoffrey Van Orden, UK Conservative defence spokesperson.
He commented: "We should be under no illusions - the ayatollahs of European integration see Britain's departure from the EU as their great opportunity. We will no longer be there to put a brake on their ambitions.
"This is particularly true of defence where France, in particular, has long sought to lead an EU army that excludes American influence from Europe.
"This is very dangerous in today's challenging world. An EU army has no credibility with our potential enemies.
"At a time when defence budgets are starved, what is the sense in the EU creating wasteful new military structures that merely duplicate what already exists in Nato? And what will be left of national independence if defence budgets and defence policy are controlled from Brussels?
"EU defence is a political tool to advance European integration. It will not create any additional military capability. This can only realistically be done through national defence regeneration and revitalisation of Nato.
"Britain must do nothing to encourage EU involvement in defence. Instead we should reinforce our armed forces and intelligence agencies - key national assets - demonstrating that we are strong in defence of our national interests and a powerful strategic ally for our friends around the world."