The former Strathclyde Police in Scotland linked it to nearly 6,500 crime reports in the last three years.
But in his first interview on the subject, Abbot David Charlesworth of Buckfast Abbey says the monastery's tonic wine "is not made to be abused".
The monastery, which has been brewing Buckfast for nearly 100 years, opened a new production plant two years ago.
The abbey first started producing Buckfast Tonic Wine in the 1920s, selling it direct to the public with the message: "Three small glasses a day for good health and lively blood."
But in a Freedom of Information answer from Strathclyde Police - which became Police Scotland in April - Buckfast, also known as Buckie, was mentioned in 6,496 crime reports from 2010 to 2012.
Hampshire-based J Chandler and Co, which bottles and sells Buckfast, is taking legal action to stop the force adding its own anti-crime labels to bottles of the tonic wine.
Police Scotland declined to comment on the social impact of the drink because of the ongoing legal action.
Abbot David Charlesworth said, "We don't make a product for it to be abused. That's not the idea.
"We make a product which is a tonic wine.
"It annoys me to think that these problems, all the social deprivation of an area of Scotland, is being put on our doorstep.
"That's not fair. I'm not producing drugs, which I know are going to be used abusively."
He said the abbey had attempted to address problems, for example employing a youth worker in an area where the problems with the tonic wine were occurring.
Abbot Charlesworth said: "If I say I don't feel any responsibility that makes me sound like a heartless so and so. That's not the case.
"Am I upset about how tonic wine is used? Of course I am. It would be ridiculous to say otherwise."
"I don't want Buckfast Abbey to be associated with broken bottles and drunks.
"But is the product bad? No."
He added: "I've heard people say we should ban Buckfast.
"If you ban Buckfast, ban Scottish whisky. It's alcohol, much stronger. But oh no they wouldn't do that.
"So they are picking on a particular thing as a sort of conscience salver."
Buckfast represents 0.5% of Scottish alcohol sales, according to the Scottish government.
The monastery and J Chandler declined to say how much is produced saying it was "commercially sensitive information".
Last year J Chandler's turnover was nearly £40m and the abbey received £6.6m from its business interests, the biggest of which is the tonic wine.
Inside the abbey's winery, opened in 2011, gleaming metal tanks about 100ft (30m) high, each able to hold 125,000 litre, stand like missile silos.
There are also six horizontal metal tanks, each holding 25,000 litres of Buckfast.
Wine imported from France is shipped to the Buckfast plant, where it is fortified and blended by a team of three lay workers.
Vats of ingredients wait to be added to the base wine in a process far removed from when the monks first started making the tonic wine in wooden casks.
From Buckfast Abbey, the finished wine goes in tankers to J Chandler's plant in Andover, Hampshire for bottling and distribution.
Meanwhile, the argument over the social impact of "Buckie" rumbles on.