A survey of UK consumer Internet service providers (ISPs), which ranked BT's broadband service as the best ADSL performer in the market, was met with confusion and scepticism on Monday.
The report, from customer experience analysts Epitiro, was based on performance levels measured around the country between April and June of this year, and ranks ISPs according to criteria such as connection time, HTTP and FTP download/upload speeds, ping and DNS speeds and packet loss.
It put BT significantly ahead of the next four in the top five — sequentially: Virgin, Demon, AOL and Orange — out of a range of what author Gavin Johns said were considered to be the "top 10 consumer-facing ISPs".
However, the results conflict with similar studies, such as those conducted by ADSLguide, which rated Demon as the best-performing ADSL ISP out of the same five contenders during the same period.
Only last week, ADSLguide found Madasafish Broadband to be the fastest and most reliable ADSL service in the UK. ADSLguide based its findings on a combination of technical testing and customer feedback, whereas Epitiro's study concentrated only on the former.
ADSLguide's Sebastien Lahtinen told ZDNet UK that he and "quite a few people in the ISP industry" were "surprised" by Epitiro's findings.
Speaking on Monday, Lahtinen suggested that the testing only 10 major ISPs made Epitiro's conclusions "slightly less useful".
Epitiro's comparison was also made on the basis of the ISPs' 2Mbps offerings, as higher-speed connections currently have too many quality variables, Johns told ZDNet UK on Monday.
Analyst Ian Fogg of Jupiter Research argued that: "Unless you have a large sample size per ISP, the quality of the line may be as significant as the ability of the ISP to deliver."
"In the old days, when people had a flat-rate product… variability of speed was down to what was happening in the ISP's network," Fogg explained.
If advancing technologies, sample sizes and differing testing criteria don't muddy enough waters, it is also worth noting the complaint of one ISP, rated below BT, that Epitiro's results had been skewed because "traffic for BT customers never leaves the BT network whereas if you're with another provider there would be handoff".
Sources within BT — a client of Epitiro — hit back at this suggestion, saying there was no such discrepancy between its services and those offered by ISPs that resell its wholesale services.
"BT Retail's results in respect of the performance indicators measured by Epitiro in their survey can be accounted for by decisions made by BT Retail on equipment and services it offers to consumers downstream of the BT Wholesale network," a BT spokesperson told ZDNet UK. "The opportunity to make these decisions is available to any ISP."
Talkback
The criterion of judgement for 'Best ISP' was very narrow.
My own experience of BT is far less flattering, especially if something goes wrong. Faults go unfixed and fault reports go missing. Being such a large organisation, it is especially difficult to keep continuity through the fault logging / repair process. Hours spent on the phone to off-shore call centres seem futile.
BT's attitude seems arrogant, like they are doing a big favour by allowing you to be their customer. And if you terminate your account (if you are able to get them to do it, that is) they don't seem to care.
It is a pity that the wonderful tenacity and energy that they use when persuing a person whom they feel owes them money cannot be copied in the customer service division.