Customers should be informed of actual speeds, the panel said
The Ofcom Consumer Panel said speeds advertised as "up to" a certain level end up being much slower in reality.
The panel said there needed to be a code of practice so broadband customers get proper information about speeds.
The Advertising Standards Authority should look at how companies advertise broadband speeds and encourage them to explain why they can vary, it added.
Research earlier this year found that customers who had signed up for speeds advertised as up to eight megabits a second were often getting something only as third as fast.
The panel said customers should be allowed out of their contract if the actual speed is a lot less than they were promised.
Internet service providers should provide clear information on what can affect speeds, and inform customers of the actual speed after installation, the panel said.
Panel chairman Colette Bowe said: "This code would establish agreed processes to give the customer the best information during and after the sales process, and to give them flexibility to move freely to different packages that reflect the actual speeds with which their ISPs are able to provide them."