Globe and Smart Release Opposing PRs Over NTC Test
By now you’ve probably heard about the NTC press release publicizing its head-to-head test between telco heavyweights Globe and Smart. So of course both companies released their own statements yesterday highlighting where they won, while downplaying whatever losses they had.
You can read the National Telecommunications Commission (NTC) press release below, detailing the testing methodology of the government agency, followed by reactive press releases from Globe and Smart placed side-by-side.
2nd Quarter Quality of Service (QoS) Benchmarking Conducted on Mobile Network Service Providers
During the 2nd Quarter of this year, the NTC conducted QoS Benchmarking tests on the networks of Globe Telecom and Smart Communication in all of the 16 cities and 1 municipality of the NCR.
The purpose of the tests is to measure their network performance based on existing NTC prescribed minimum service performance standards.
The tests were conducted using prepaid Globe and Smart SIM cards subscribing to regular services and not unlimited services. Sun SIMs, Talk and Text SIMs, Red Mobile SIMs and Touch Mobile SIMs were not used because these SIMs are using the networks of either Globe or Smart. The NTC monitoring team made sure that the mobile phone numbers used and the locations where the tests were conducted were not known to anyone except to the members of the monitoring team.
The monitoring teams conducted tests during the last week of April, May and June 2012.
The monitoring team initiated a total of 1,506 on-net calls per network. The initiated calls were simultaneously done for Globe and Smart. The monitoring team made the calls using the “drive test” method (i.e. inside a moving vehicle).
The results of the tests on the following parameters were as follows:
- Blocked Calls or Grade of Service refers to the percentage of calls that were not given access by the network.Based on the performance standards on Grade of Service, Smart Communication was measured at 9.95%, while Globe Telecom was at 4.45%. Though both of them did not pass the standard for Grade of Service which is at 4%, Globe Telecom‘s performance has an edge over Smart Communication on this area.
- Dropped Call Rate refers to the percentage of on-going calls that were involuntarily terminated.On Dropped Call Rate, both companies were within the 2% minimum standard. Smart Communications registered a 1.53% dropped call rate while Globe Telecom was at 1.66%. Smart Communication’s performance is better than Globe Telecom in this item.
- Average Receive Signal Level refers to the signal strength that was being provided by the serving cellsite to the mobile handset of the subscriber while the conversation is on-going. This refers to the signal bar of a subscriber’s handset.For Average Receive Signal Level, Smart Communication had an edge over Globe Telecom. Smart Communication registered -62.63 dBm and Globe Telecom registered -69.83 dBm. The minimum acceptable Average Receive Signal Level is -85 dBm.
- Average Signal Quality is the quality of voice transmission while a subscriber is using his mobile phones. It should not be choppy or garbled.For Average Signal Quality, Smart Communications had an edge over Globe Telecom. Smart registered 0.63 and Globe Telecom registered 0.72. The minimum acceptable range for this item is from 0 to 4, the closer to 0, the better. An Average Signal Quality of “0” indicates that there are no errors in transmission.
- Call Set-Up Time refers to the time required for the network to activate the called party. In simple terms, this refers to the period required from the time a subscriber finished dialing to the time of the first ring.Call set-up time for both telecom service providers were within the acceptable industry standard of below 14 seconds. Smart Communications registered at 11.74 seconds, establishing an edge over Globe Telecom by 0.16 seconds. Globe Telecom Call set-up time was at 11.90 seconds.
The NTC has already called the attention of both telcos regarding these results. The telcos informed the NTC that they are already addressing the matter. Globe Telecom said that they are implementing an expansion and upgrading of their network. Smart Communications stated that they are continuously rehabilitating their network. The NTC then mandated the telcos to inform the public of their ongoing efforts to improve their services. The NTC also required the telcos to give a detailed report of these improvements on a per area basis including date of completion per area.
The NTC will continuously monitor the service performance of the telcos to ensure the quality of service that telcos provide to the public.
Now here’s Globe and Smart’s respective press releases on the NTC Test:
NTC results boost Globe
MANILA, Philippines – A recent network benchmark test conducted by the National Telecommunications Commission (NTC), comparing the Grade of Service and overall signal quality of both Globe Telecom and Smart Communications against NTC Cellular Mobile Telephone System (CMTS) standards showed the Globe legacy network outperforming competition’s recently modernized network in terms of grade of service.
The Globe legacy network also performed at parity with the upgraded network of Smart in almost all other metrics.
In the most important benchmark where the NTC has an applied industry standard consisting of Grade of Service or Call Setup Failure Rate (CSFR), Globe Telecom’s legacy network outperformed its competition who has widely announced its network upgrade completion.
With the Grade of Service or CSFR standard set by the NTC at 4%, Globe recorded a nearly identical 4.45% while Smart registered a distant 9.95% (closer to the industry standard is better). This means that call success rates are higher using the Globe legacy network at 95.55%.
For consumers, the experience is that it is easier to call using the Globe legacy network with more calls getting connected even at first try.
For Drop Call Rate, the NTC standard is 2%, and test results showed both telcos performed within the standard with Globe registering 1.6% at parity with Smart’s 1.5%.
Clearly, the results belie the claims made by Smart in its multi-million negative advertising campaigns pointing out drop calls and difficulty in making calls within the Globe legacy network.
In terms of Call Set Up Time, both telcos performed within the acceptable standard of below 14 seconds. The Globe legacy network performed at parity with 11.9 seconds versus the Smart upgraded network at 11.74 seconds. This is a measure of how fast domestic calls get connected from one number to the other.
Other metrics in the study included Average Signal Quality with a minimum acceptable range of 0-4, the closer to 0, the better. The telcos performed at parity, both performing above 0.50.
The final metric is the Average Receive Signal Level with a minimum acceptable range of -85 dBm. Both telcos did not make it to the standard.
The NTC benchmark study was conducted during the second quarter of 2012 in sixteen cities in Metro Manila, through network drive tests using prepaid SIMs of both telcos with a sample size of over 3,000 test calls.
NTC Commissioner Gamaliel Cordoba earlier called the attention of both Globe Telecom and Smart Communications on what he referred to as increasing complaints from customers about drop calls and undelivered text messages, asking both firms to explain the causes for the disruptions and to give a timetable for completion of their network modernization programs.
Globe Telecom President and CEO Ernest Cu in his reaction to the recent NTC benchmark results said, ”It is now clear who is telling the truth and really giving good service to the public even with just our legacy network. Competition has claimed they are done with their upgrade and there’s no denying the official results from NTC quality tests show their upgraded network performs even below NTC standards.
As soon as we fully fire up our brand new network, our subscribers and the entire nation will enjoy a whole new mobile experience that is once again pioneered by Globe Telecom.”
Globe announced recently that its $700-million network modernization program is more than 50% done and is progressively rolling out ahead of schedule for change-out completion nationwide within the first quarter of 2013.
Cu said that unlike other transformation programs that involve merely upgrades, Globe is building a brand-new network that is future-proof and built to provide higher call quality, pervasive 3G and 4G coverage, and faster mobile internet experience.
Smart’s official statement RE: NTC test
[20 September 2012] Yesterday, the National Telecommunications Commission (NTC) confirmed the network superiority of Smart Communications, Inc. (Smart) following the release of the results of its network quality tests for the second quarter of this year which showed Smart posting significantly better test results than Globe Telecom.
Globe, however, issued a statement that attempted to turn the results of the NTC’S Quality of Service Benchmarking Test upside down.
According to that test, Smart’s network rated higher in four of the five parameters that had been defined by the NTC in consultation with all three mobile phone operators. These four important parameters where Smart posted better scores are: 1) Drop Call Rate; 2) Call Set Up Time; 3) Average Signal Quality; and 4) Average Receive Signal Level.
In its statement, however, Globe conveniently downplayed or disregarded these parameters and focused on the “blocked call” parameter, which was the only test that showed better results for Globe. On that basis, Globe said that its “legacy network” was better than that of Smart.
This is the same legacy network that Globe officials have admitted, in several recent occasions, to be congested.
This selective reading of the test results defies arithmetic, and, more importantly, flies in the face of consumer experience.
Globe’s press statement also contains a substantive misinterpretation of the test results in one of the key parameters – the Average Receive Signal Level. Globe said in its statement that: “The final metric is the Average Receive Signal Level with a minimum acceptable range of -85 dBm. Both telcos did not make it to the standard.”
The truth, however, is quite different: Both carriers actually passed the standard. Globe registered a score of -69.83 dBm. But Smart delivered lopsidedly better results: -62.63 dBm. In this parameter, the lower the negative number is, the better the signal. Translated into layman’s terms, what this result means is that Smart’s signal level is up to five times stronger than Globe’s. In terms of customer experience, a stronger signal level means better indoor coverage, resulting in better voice quality, less drop calls, faster and more reliable SMS and higher data speeds.
Though we are encouraged by the NTC test which validates our network superiority, Smart continues to work hard in order to provide our customers superior service. By completing our network transformation in mid-2012, we are now moving on to deliver to our subscribers next generation services such as the Long Term Evolution, or LTE. This is vital because we realize that, in the end, it is the satisfaction and judgment of our customers that really matter. [END]
What’s interesting about the test is that it was conducted on prepaid cards. Perhaps the NTC can run the same kind of tests but with postpaid accounts? Of course this would require a significantly higher investment, as the NTC will have to commit to a two-year monthly plan for at least two numbers. Yet a postpaid test would also paint a more comprehensive picture of telco performance in the Philippines.
This entry was posted on Friday, September 21st, 2012 at 10:57 am and is filed under News. You can follow any responses to this entry through the RSS 2.0 feed. You can leave a response, or trackback from your own site. You can leave a response, or trackback from your own site.