Internet Thought: OECD publishes (my) paper on Fibre

Over at Internet Thought: You can find the links to the OECD paper on fibre networks that I wrote last year. I has finally been published :-) BTW update the RSS-readers, Internet Thought is where it is happening now! :-)

Internet Thought: Moving graphs of OECD broadband penetration

I used the OECD stats on broadband and pc use to make a graph moving through time. I use Google spreadsheet and apps to do this. Look at my new blog http://internetthought.blogspot.com

No Broadband means no Mobile Broadband

over on Internet Thought I posted on why if you don't have Broadband in a region you will not have mobile broadband either.

Two FTTH projects fail and more trouble in The Netherlands

Two projects supported by Reggefiber have had to face defeat. Both were unable to reach the required sign-up rate for the project to commence. See more at my new blog Internet Thought

Free announces stellar numbers, but why is FTTH capex so high?

Go to Internet Thought for the numbers

Internet Thought: Misconceptions on interconnection

Please go here to find my rant on interconnection Update your RSS accordingly

New Post at Internet Thought on submarine fibre

Move your RSS feeds (all fifty of you) I'm now posting at Internet Thought :-)

What Submarine Fibre costs

Pretty soon I will be the Submarine fibre blogger if my posts continue the way they do. However I found a very interesting quote throught the ITU's/Strategic Policy Unit Blog. They link to an article in Business Daily Africa, which shows the costs of running the TEAMs project (wiki entry), to run fibre to the east of Africa. Those joining in the project expect an Internal Rate of Return of 32.71 per cent with a pay back of 2.4 years. Not a bad return for a fibre that's supposed to last 25 years. Current prices of connectivity are $5000 dollar per month (satellite only I believe) and the government hopes to go to $500/month in a couple of years. Though UUNET says this can't be achieved.

Last Post! Moving to Internet Thought

After years of using Lunatic Thought, it just started to sound really lame. I'm no idiot.. no raving lunatic and not really funny either. I just do internet and telecoms on multiple levels. So here it is: Internet Thought! Everything is the same, just the name is different, so update your RSS feeds! (I'll crosspost for a while and take some interesting posts here.)

OECD Workshop on Fibre Investment and Policy Challenges, Stavanger, Norway, 10-11 April 2008

The aim of the Workshop is to examine fibre investment across the OECD and look for best practices across a range of investment scenarios. The Draft agenda of the Workshop is available in PDF format. Yours truly will also give a presentation with the title: Dig or wait: Is now the best time to roll out fibre? (Funny thing; they already have the title. I haven't even started making the presentation yet) Oh well, I don't need to find a catchy title ;-)

My session will be: Investment opportunities and challenges
The investment session will look at the opportunities and challenges facing existing communication operators and new market entrants (e.g. utilities) as they work to expand fibre coverage. The session will evaluate situations where investment makes economic sense for private firms and other situations where governments may need to play a more active role. It will look at the track record of various public/private sector partnerships and attempt to find some effective recommendations.

OECD on Global Opportunities for Internet Access Development

The OECD published a great interview with its senior economist Dr. Sam Paltridge on Global Opportunities for Internet Access Development. He also wrote a paper on the subject which is a great read. Some snippets that I liked alot

Q.What role do Internet Exchange Points (IXPs) play in the creation of Internet service?
Name withheld

A. IXPs are places where different Internet networks can physically interconnect to send and receive traffic between their networks. Following the commercialization of the Internet they rapidly spread around the world to enable service providers to economically and efficiently exchange traffic locally.

In the absence of an IXP, in any country, local traffic between two service providers will by and large be exchanged internationally. In these cases an email sent from one user to another, in the same country but using different service providers, may be routed via an IXP in New York or Paris rather exchanged domestically. By way of contrast, if that traffic is exchanged locally it can be far less expensive (i.e. avoiding expensive international circuits) and provide better performance for users (e.g. in some countries avoiding satellite circuits with their inherent delays).

Some 90 countries don’t have IXP’s today. There is a map created by the Packet Clearing House which keeps count of countries with and without an IXP: https://prefix.pch.net/applications/ixpdir/summary/
IXPs are relatively inexpensive (e.g. less than USD 40 000 per IXP) to establish and can benefit all stakeholders. For less than USD 4 million each country of the world without an IXP could have one.

Such expenditure, for example, in the form of development co-operation, however, would only make sense if the conditions are in place to enable the IXP to operate efficiently and become industry driven. The main challenge is often is creating awareness of the benefits of IXPs.

and from the report
The next several billion Internet users represent a commercial opportunity rather than a burden and this should be reflected in policy approaches.

Liberalisation of communication markets focuses competitive forces on the expansion of access and affordability for the poor as well as promoting innovation applicable to local circumstances as highlighted by the recent experience of some Asian and African developing countries. Improved access and lower communication costs generate general economic and social benefits. Historically, in markets typified by monopolies and little momentum for access growth, the cost of reaching and maintaining service to low income users was not sustainable.

And on Page 12 there is a explanation of Jipp's Law which used to mean that countries could over and underinvest in telecommunications and which was used to keep investment low and prices high in accordance with the development of the economy in that region.

and on interconnection it sais:

MPP and CPP markets have both exhibited strengths and weaknesses relative to each other in thedevelopment of wireless markets. Both have shown capabilities for innovation and growth. A number of open questions for the future relate to how will both models will deal with convergence with the Internet.Will one model, for example, prove more adept at fostering innovation at the interface with the Internet? How will financial flows between countries be affected by the different termination models? Will one system lead to greater integration in service offerings between countries or networks with the same termination model irrespective of geography? Finally, if termination rates are about compensating networks for costs what explains the vast differences between charges and how will this impact on the competitiveness of countries with high termination rates?

All in all it is a must read on development of networks as it covers many angles on the subject from submarine fibre, to IXP's, to interconnection

idea #1880: Graphical network configuration by drawing visio like drawings

Just posted this to the Ubuntu Idea Brainstorm site. You can dump all your brilliant ideas for how an Operatings system (like Linux of Windows) should work and they will see if they can build it into Ubuntu Linux. My idea is number 1880. This is really where open innovation starts. Listening to all the good ideas of the users.

I want to be able to graphically design and/or check how my computer connects to the internet and to other pc's devices in range/on the same network.

Say I'm sitting at home, small home network 4 pc's two wired, two wireless, one pc has printer attached, printer on wireless etc. I would like to be able to draw a visio diagram of what the network looks like and later on be able to check the diagram how it is functioning and if something changed.

So literally I would like to be able to draw a diagram showing my laptop, wireless link to wireless access point, access point wired via 100mbit ethernet to DSL modem. DSL modem is four port switch. Connects to 2 other pc's. One of these pc's is connected via USB to a printer. The wireless access point has a USB port, which has a USB storage attached. Than the system should just be able to figure out where I do my backups, where it can print, how it can connect to the internet etc.

This was something I discussed with Colleagues in an all windows environment this morning. One of my colleagues had trouble configuring the printer. He wanted to tell the computer to connect to "that" printer "there". This required a string like \\foo-bar-boo\baf\boom\bang-123. All he wanted to see was a diagram of the building we were in, go to the sixth floor, see the printers on the lay out and click the left printer on the east wall.

Oh yeah, not even Ubuntu user... just wished all networks could be configured like this.

T-Mobile to sell Orange Broadband, no FMC then.

Interesting news. Telecompaper is reporting that T-Mobile will sell Orange Broadband in the Netherlands. They said so in their SEC-filing. They got the broadband division with the purchase of Orange Mobile and there was an idea in the market that this would give T-Mobile a presence in both the fixed and mobile market. It turns out that Fixed Mobile Convergence is of no interest to T-Mobile and they are shedding the department.

Maybe Free can buy it and add some 'joi de vivre' to the Dutch market.

Terminate terminating fees: EC Interconnection report released

The WIK institute has released a paper on "The Future of IP Interconnection" written for the European Commission, by Scott Marcus, Dieter Elixmann and Kenneth Carter. I know Scott and Kenneth as some of the smartest people in the business (Dieter by association must be as well :-)

I just read the Executive Summary and it is a joy to read. I can hear the gnashing of teeth in the boardrooms of the GSM companies already and they would do right to oppose it everywhere they can. (The GSMA already has the a very low quality report on the subject) In my previous job I tried to set this on the agenda and it was interesting to see how NRA's and competition authorities haven't yet thought about the problem. This will give them much to think about .

Currently interconnection with a mobile network is an expensive affair with extremely high terminating costs. These costs in no way reflect an economic or technical reality, but they are very beneficial to the industry. They stop anybody from competing on price, by setting a limit to the price per minute. (Equal to termination costs *(1-market share)) If you price below that limit every call the customer makes will loose the telco money. This is a great way to keep prices high, competition limited to non-price items and since termination fees are set by the regulators telco's can say that it's not their fault. It's a great model from a telco point of view: No price competition, regulators that have to take the blame for setting the fee, difficult economic models that can be attacked on all sides and best of all when you stand to loose money you can try and manipulate the system to bolster your business.

The internet world works with a different model based on peering and transit agreements and no requirements to pay terminating fees. It has some very interesting effects. The price setting of one telco has no influence on the price setting of another. It also means that there is no terminating monopoly, therefore no need for regulators to step in. There is a dynamic market for transit and because of that prices continuously drop. From the point of view of a telco company this is not a very good model, since it introduces competition on price. This might lead to losses, bankruptcies and all that stuff. In the internet world you actually have to know what you're doing or you might lose your business.

The report "The Future of IP Interconnection" has the following edited for size conclusions and recommendations:

• For reasons explained more fully in the report, it is clear that current CPNP arrangements already have a substantial negative effect on welfare, especially for the mobile network, in a number of respects:
  • They tend to lead to inefficiently high wholesale termination fees, even when the fees are regulated;
  • High wholesale mobile termination fees in effect create collusory incentives to maintain high per-minute (mobile) retail prices;
  • The high retail prices depress use of the service to levels far below those that are efficient, and may in some cases depress use below the level of efficient monopoly price for the operators as well. CPNP systems with high mobile termination fees may tend to drive faster take-up of mobile services, which can be positive in developing countries; however, with minor exceptions, further stimulus to mobile adoption for the EU27 is not needed.
  We conclude that CPNP arrangements are already problematic today.
• We do not advocate an interconnection obligation as regards IP data traffic in general, and we do not see a need to mandate any-to-any peering; however, NRAs must be able to intervene if interconnection breaks down, especially where this is a manifestation of some form of market power.
• Independent of the migration to NGN, we think that societal welfare would be substantially enhanced if mobile termination rates were much lower than they are today, and possibly no higher than the rates that prevail today for fixed termination rates. This could be implemented by (1) accelerating the speed with which the maximum call termination rate declines from year to year under existing CPNP arrangements, so as to reasonably quickly achieve levels much lower than those that pertain today; or (2) by requiring all fixed and mobile operators to eliminate call termination fees altogether; or (3) by permitting negotiated termination fees subject to an obligation that the fees be reciprocal (the same in both directions) between each pair of interconnected (fixed or mobile) networks.
• What we concretely recommend instead is that the Commission mandate that fixed and mobile call termination rates “fast glide” to prespecified target levels over a predefined number of years (somewhere between three and five).
• What we concretely recommend instead is that the Commission mandate that fixed and mobile call termination rates “fast glide” to prespecified target levels over a predefined number of years (somewhere between three and five).

One thing I partially disagree with is what they say on Net Neutrality.
"In regard to network neutrality, we do not advocate major regulatory initiatives at this time. We see merit in the use of Articles 20 and 22 of the Universal Service Directive to require ECSPs to document their practices as regards blocking access to services or degrading the quality of access to services. There may also be merit in enabling NRAs to mandate a minimum quality of service, as the Commission has proposed. NRAs and NCAs need to be prepared to address wilful deviations from network neutrality, especially where an element of economic foreclosure appears to be present; however, the existing regulatory framework for electronic communications probably provides adequate tools, and competition law provides additional mechanisms."

I agree with them that their should be no heavy handed etc. But I do think that the current regulatory framework needs to be tweaked in order to deal effectively with violations of Net Neurtrality. Currently only telco's can appeal to an NRA when something nefarious is happening on the networks. Furthermore it's unclear if the NRA has a stick to hit with, since it can only require parties to negotiate. I therefore belief that we should have a rule that gives NRA's a stick that they can use in case something bad happens and the market can't solve it itself.

All in all, great read and let's see what kind of opposition it will draw.

Google buys trans-Pacific submarine cable Unity

Gigaom is reporting on Google buying a share into the Unity submarine cable. Many people will read into this an attempt by Google to become a telco or do anything out of its current layer 7 service and application business. I don't belief it is, it's just simple economics. Google now buys wholesale capacity instead of retail. My reaction on Gigaom was:

One of the main drivers for wanting your own fibre on certain submarine routes is the pricing strategy of the owners of the submarine fiber. Traditionally these fibres have been owned by incumbent national monopolists. Their pricing was set at a fixed price per Mbit/s. If your banndwidth utilisation grew, their income grew too, though their costs didn’t, leading to excess profits. On the Transatlantic route this problem has been solved by having an oversupply of commercial competitive fiber. The oversupply resulted in a situation I call mutually assured destruction, where everybody went bankrupt and whole networks were sold for pennies.

On the Pacific route it’s mostly incumbent national monopolists owning fibre and they probably have learned from the Atlantic disaster. This means prices don’t drop (or not as quickly as traffic growth) and that means that some parties see an increase in their traffic costs. Google now has solved this by joining a club of submarine fiber owners and not having to worry anymore about the cost of a megabit/s. Google just has to worry about when they will fill up their terabit chunk and when someone will slice through the fibre.

BTW I’m willing to bet Google will join another club on this route to add some much needed redundancy.

An Anchor cut Flag Falcon between Dubai and Oman

There are a million conspiracy theories around about the submarine fibre cuts in the Mediterranean an Middle East. Flag Networks has a status update on the repairs on their website. The status update for the Falcon cable between Oman and UAE shows a picture of the 5-6 tonnes anchor that cut the cable. No pictures of Navy Seals, dark submarines, Osama bin Laden or Ninja's. It all may just be bad luck and stupidity.

Both cables will be repaired by Sunday!

What I don't get about Google

Google is one of my favorite webcompanies. And why not, they make brilliant products. Yes privacy is a concern, but they have proven to be true to their motto of don't be evil. However there are several things I don't get about them.

- Why have Blogger, Google Analytics, Adsense, Adwords and Feedburner and not integrate them into one convenient package? Why do I have to configure different things at different locations. Integrate and make it easy, so that uptake is high etc. --> why can't I see which pages/topics yield the most money in Adsense? (Though in my case the Agpo Nev 124 is still very popular and I will not write about it again)

- Why is it so hard for Google to understand that I search and read in two languages: English and Dutch. Fix it so that I can get a mix of Dutch and English results in search and especially News, mixed with really high ranking other language results... At the moment either the first ten are English or Dutch and that means I have to do two searches instead of one.

- Why don't all Blogger posts have automagic trackback links or something of the kind. Google knows who links to my blog, please fix that

- Why doesn't Google integrate its feedreader with a Delicious like system. I share items already and that should be enough... (tagging and starring is not for me)

- Why hasn't Jotspot come back unto the market? You bought it... now give it back, it was a great wiki system.. though Socialtext and Confluence are excellent too!

- Why hasn't Google become the default yellow pages yet... They have the technology, but searching for businesses on Google Maps still yields funky results. (at least here in .nl)

- Why do several Google products feel rough around the edges? they are brilliant, but just need hat extra 20% to be finished. (sounds like some of my papers and ideas, I know its not fun to do the last 20 percent, but it does make it better for the user)

- Why doesn't Google build more office productivity applications... Please help the workplace to move forward. There is so much data in an organisation. A couple of Google boxes with cheap apps that help us organize could make a real difference.

- Why haven't the implemented my idea for Google Adsense for Charities yet?

- What is the status on my last wish list for Google?

- Why are there no blogs/forums/trackers where we can freely dump our feature requests and see status updates?

What do you miss?

Any points I have forgotten?

KPN, 3 and Play drop data roaming charges. It's cheaper to use VOIP

KPN, 3 and Play have dropped the data roaming charges between their networks to 25 cents/Megabyte. That's great for their customers roaming in the UK, Germany, Belgium, Poland etc. So how many seconds of VoIP does this buy you? A VoIP stream is between 10 kilobit per second and 90Kb/s. This translates to between 90 and 800 seconds. Given that the price of a roaming call on the mobile network has been set at 49 cents per 60 seconds and receiving at 24 cents/minute, it seems that it is now officially cheaper to use VoIP for mobile roaming!

Put differently: any (VoIP like) stream of 266 Kbit/s or 133 Kbit/s is now cheaper than a GSM call of 9600 bit/s.

There must be a catch here. Telco's are never this afraid of mrs. Reding and this is not a competitive market yet!

Submarine fibre cables and public policy

Three cable breaks in submarine cables in one region in one week. It's enough to get the conspiracy theorists going. For me it's the trigger to write down some ideas I've been having on submarine fibre optic cables. I'm going to argue that nations need more/different connectivity than the market can provide, that therefore it will need to be treated as a public good and the pricing will have to be set accordingly.

In an information society fibre optic cables are the nerves that keep the country moving. You need them in your country to every extremity (house), but you also need them from the country to the rest of the world. No fibre, no information society. The flow of information enables the flow of goods, money, people. For a significant part of the world this flow of information is a Single Point of Failure (SPoF). Most countries have only one cable coming into the country. The only backup is satellite, which is due to latency and a limited bandwidth is so 1970. If your country is not connected to the world via fibre, you're still living in the seventies.

The dependency on a handful of fibres for international connectivity (and trade) should make fibre as important as oil and gas lines or airports. So how many do you need. Given the risks involved one would want at least 3 different lines on different routes coming in and out of the country. The routes should be undersea, because these tend to be less prone to damage than the ones over land. There are more idiots with backhoes than boats. (Yet another reason not to want to be a landlocked country) And that is where the first problem lies: How do you get three international routes of the more expensive kind to almost every country in the world Currently most lines are Trans North Atlantic or Trans North Pacific. They run from rich country to rich country, poor country to the north and never south-south. A redundant network would look much more like a web, instead of a like spoke and hub system it now is. Unfortunately fibre follows money. South America is connected to the world via the US. Africa via Europe. Asia- Europe mostly goes via the US. In an ideal situation South America would have multiple direct links with Europe, Africa and Asia via both the Atlantic and the Pacific. And Asia and the Middle East wouldn't send all their traffic over the US or the Suez Canal.

The trouble is money. Long distance undersea fibre is expensive: hundreds of millions and even billions for the fibre, but only a couple of million a year for operational expenditure. But making money when multiple submarine fibres compete is almost impossible. A submarine fibre has the capacity to carry 7 terabits/s these days and more is becoming possible. So you've got an instant oversupply of capacity. Multiple competing lines lead to competition on marginal costs and mutually assured destruction. This has happened on the Trans Atlantic route where there were several new entrants in a few years, while one fibre was more than able to carry (almost) all Europe-US traffic. The price of traffic dropped to whatever sales could get per mbit/s instead of how much they needed to pay the banks. Bankruptcy was assured. Bankrupt networks were bought for almost nothing and kept prices low, which killed the business for the initial winner, which led to the winner becoming a loser too. Mutually assured destruction it is.

The high investment in northern routes has made the US, Europe and South East Asia the natural hubs for traffic to less connected regions. A telephone call to Europe from South America is cheaper when it runs from South America to the US and then towards Europe. A more direct connection might be logical and redundant, but the low prices on the north Atlantic route are a cheaper alternative than going direct. At the moment there is only one direct connection from Brazil to Europe. If you look at the fibre map: a lack of fibre is clear in South America, the Indian Ocean and Africa.

When it doesn't make economic sense to have fibre going southern routes, because its cheaper to route it over existing northern routes, there might be a reason for the government to step in. Security in the supply of connectivity can be regarded as a public value. The trouble is: how do you price it. If one fibre route can substitute others, the result might be that all traffic will flow over the cheapest one, leaving the others empty. Another problem may be that governments (and their monopolist telco's) tend to collect monopolist rents from international connectivity. Africa's lack of connectivity is as much the result of the general poverty as it is of its monopolist telco's, who keep international connectivity expensive to finance corruption and inefficiency. The price per mbit/s stays much the same in many countries, but demand increases every year, leading to ever increasing profits and a much lower economic growth than would have been possible. It would be preferable that the monopolist is paid a fixed amount for the fibre and every month or every quarter set the price on the basis of the volume in the preceding period. Since traffic is growing at a rate of 50% in most countries, prices would drop with 33% per year accordingly (see graph for price (green) vs volume (red).

There are two solutions to the connectivity problem. One is outright government financing of submarine fibres. Pay for them the way you do with bridges and roads and write it off in one year. (If necessary work together with neighboring countries) The cost of connectivity would then only be in the costs of operation. Since operational costs are only a small fraction of the total costs, the price of connectivity could be very low. Another option is to use a revolving fund where the income from writing off one line is used to finance a second one. It would then be possible to combine the costs of submarine fibre and to have the more popular routes subsidize the less popular ones by adding the total costs of the fibres and spreading the costs equally over the users of the fibres. Neither is perfect. They may lead to adverse effects like, suboptimal routing, freeriders, bankruptcies in of commercial networks etc, because the price might be below the market price in other regions. The latter one might also suffer from being too expensive at first and therefore stifle growth. All in all it's not an easy problem and I don't have a perfect solution. However from a public policy perspective it can't be a good idea to have your nation dependent upon only one piece of glass with the thickness of a hair.

BTW for nice pictures of a Cable ship see my blog entry of a week ago.

Structural Separation?: CAIW cable bought by Rabo real estate/infra fund and management

Interesting news. The Dutch cable company CAIW has been sold to its management and Bouwfonds Asset Management (Rabobank's real estate division). The commercial part has been sold to the management and the network has been sold to Bouwfonds. This sounds like a structural separation between the network and the commercial part, where the commercial part will rent hte lines from Bouwfonds and Bouwfonds sees this as a long term investment with fixed margins. It's not clear from the press release how the structural separation will be achieved, ie will Bouwfonds just own the passive cables or also part of the active infrastructure and/or will this be only be a legal separation and will the operation remain the exact same.

From what I can gather from the financial documents the sale price was around 190 million Euro for 150k subs or 135000 houses (110 million in net profit for the municipalities, but there was 83 million in long term debt), which would translate to about €1250 per sub. (Not Cheap! with a profit of 4.3 million) Unfortunately it is unclear to me how much was paid for the network and for the commercial part separately.

I still find it very interesting because this is a clear example that there is money in infrastructure and investment firms do like long term investments in network infrastructure. CAIW has also shown itself to be innovative and has already invested in fibre networks for both business and consumers (Naaldwijk)

Fibre optic cable laying vessel Tyco Responder in Curacao

In December I was on Curacao for my wedding. In the city center I got this telecommunications treat: The Tyco Responder floated past us into the harbour of Curacao. This ship lays and repairs fibre optic cable in the Caribbean. These ships have made the internet possible. Imagine if you would still connect to most of the world via satellite..... If you ever want to know about the business of laying cables, read Mother Earth, Motherboard by Neal Stephenson. (Unfortunately the image is a bit blue because of a wrong setting on my camera)

Reding: Blame it on the National Regulators

Ai Frau Reding has the audicity to blame the national regulators that they haven't brought mobile data roaming tariffs down!

Wasn't the problem that national regulators were incapable of bringing down these rates, because the rates were high at the wholesale level and therefore out of their jurisdiction (a foreign mobile operator was the problem). Wasn't it the case that our friends from the Commission therefore had to set the roaming rates? Wasn't it the same Commission that didn't want to regulate data roaming, but saved it for a later date?

Come on Frau Reding, you could have fixed this last year, don't blame it on the national regulators! Or is this all a scheme to get the Euro Uber Regulator (EECMA) in place?
Even better fix mobile roaming by introducing competition. Give all operators including MVNO's the option to offer access to anybody roaming in the country and require that the home operator is only allowed to charge a small percentage for the billing. For this see my previous entry.