Two-thirds of households hit by the bedroom tax are in rent arrears.
The shock findings of the National Housing Federation (NHF), which represents the UK's housing associations, show that more than a third had fallen into debt because of the charge.
It used polling firm Ipsos-Mori to survey 183 associations, and the firm found that thousands could simply not cope with the change.
The bedroom tax takes an average of £14 per week from the benefits of those with a bedroom they don't use, and £25 from those with two.
UnemployedNet has written before about the poverty level welfare paid in the UK - they are among the lowest in Europe - and two weeks ago the Council of Europe accused the government of paying illegally-low benefits.
This means claimants do not have the financial resources to deal with extra shocks including the bedroom tax.
The NHF called the tax "unfair, unworkable" and said it was "heaping misery and hardship on already struggling families".
Chief executive David Orr said: "Now many are at risk of being evicted because they simply can't find the extra money to pay their rent. These people have done nothing wrong."
He said the coalition government had "suddenly changed the rules and given them a false choice - move to a smaller home or pay.
"Yet we know there aren't enough smaller homes in England for these families to move into."
Vulnerable people are bearing the brunt of the changes, particularly disabled people.
The NHF's Ruth Davison told the BBC: "We know that 180,000 people are under-occupying two-bedroom homes, but last year only 85,000 one-bedroom properties became available.
"Two thirds of the people affected by this, by the government's own admission, are disabled."
Despite this growing need the government has reduced the amount of money available next year to help from £190 million to £165 million.