European Telecom Market Authority (ETMA aka EECMA) a threat to checks and balances in the EU

The proposed European Telecom Market Authority (aka EECMA or European Electronic Communications Market Authority) is an utter disaster for the democratic functioning of the EU and the balance of powers in the EU. It will:

• not address the fundamental problem of an imbalance in power,
  
• increase the lack of democratic oversight of telecommunications decisions at the EU level,
• seriously devalue the role of national regulators ,
• ruin the quality of rules and regulations and finally,
• there was no demand for it.

I will try to sketch the context and the formal situation first and than explain how it works in practice, then why the new European Telecom Market Authority will have the negative effects that I suspect and lastly what I think needs to be done.

Context
As you may know, the EU proposed changes to the current telecommunications regulatory framework. This framework is the basis for the telecommunication laws in each of the EU member-states. The current framework is quite good, certainly compared to eg. the US telecommunication laws, but a review has been held and some proposals have come from the Commission. Most of it is evolutionary and not revolutionary. There is however one controversial proposal, which is the European Telecom Market Authority.

The proposal
The EC propose to establish an organization of about a 120 people reporting to the European Parliament and tasked with:

• "ensuring that the 27 national regulators work as an efficient team on the basis of common guiding principles;
• delivering opinions and assisting in preparing single market measures of the Commission for the telecoms sector;
• improving the accessibility of telecoms services and equipment for users with disabilities;
• monitoring closely the use of the single European emergency phone number, 112, and identifying remaining obstacles;
• facilitating cross-border EU services in relation to rights-of-use for scarce resources such as spectrum and numbers, and enabling operators wishing to do so to use a single European area code for their services;
• addressing network and information security issues."

The Commission is of the opinion that we need such an organization, since the national regulators (NRA) in the member states haven't all been swift, consistent and effective. This is undeniably the case and some in cases nations have just been downright horrible and wrong, like Poland, where the deputy minister was head of the independent regulator and Germany with its Regulatory Holiday for its poor incumbent Deutsche Telekom, which is building something that uses VDSL2, but is a service instead of a network and therefore according to the German regulator not subject to regulation. However, member states getting it wrong doesn't mean the Commission/ETMA should take over and even less that it would be good at it.

The Current Situation
A longer description can be found here. Currently when a national telecoms regulator issues a ruling it has to notify this with the Commission and the Commission can then raise "Serious Doubts" or even Veto a ruling. The Commission has nothing to say on the actual remedies, though it would love so. Now when serious doubts are raised the NRA will have to consider those and take them into account into its ruling, if the Commission is still not happy, than it can veto the measure. If the NRA is unhappy, it can go the European Court of Justice and wait 4 years for a solution. Also the European Regulators Group (ERG) can be asked for an opinion to advice in a dispute between NRA and Commission.

Reality
Reality has a tendency to be different that what you read in the law. I've taken to game theory quite alot lately and laws are the rules of the game, but they don't explain how the game is played in reality. The current game is unbalanced and completely tilted towards the Commission. What happens is the following:

• The Commission comes asking what your proposed regulations are. It will then informally and verbally tell you what is wrong and that you need to fix it. At that moment civil servants NRA's and ministries internally will have a serious problem already. Directors want something fixed, ministers are wondering what parliament will think.
• The Commission will also want to see the proposed remedies, regardless of the fact that they are not allowed to rule on it. If they don't like the remedies, they will not like the ruling. <\
• If the Commission raises 'Serious Doubts', the press will be informed. There might be an official point of view, but informally it's a spin war. The Commission answers to no one. The NRA and national government will have to fight of the press and parliament. The press figures that the government is wrong from the get go and in parliament the opposition is having a field day. At this point most political figures will buckle and give in to the Commission.
• If the Commission vetoes the governments decision than the government is in big problems. The minister clearly has failed, the NRA is incompetent. This will result in debates in parliament and again politicians will cave in.
• The internal discussions within the Commission on the subject can reflect national and EU-level political discussions to such an extend that it is hard to distinguish between reason and politics.
  
• The ERG can be asked for an opinion, but the Commission is not bound by it and it caries little political weight. It has a tendency to side with the Commission when the Commission is very clearly correct (eg. Germany Regulierungs Ferien) and only sometimes against the Commission, when it is clear the Commission is wrong. In the former case the Commission will cite everywhere that it is right and cite the opinion widely. In the latter case the Commission will brush of those incompetent NRA's.
• A government will not have any place to go until a final ruling by the Commission. When there is a ruling it can stand in line at the European Court of Justice, which is a group of intelligent but very slow judges. It currently takes 4 years to get a ruling.

So there you have it. A NRA can be right, but in order to be acknowledged as right it takes 4 years. At which point the ruling is irrelevant. In the mean time there is a stale mate or worse the country issues the ruling according to the decision of the Commission and just continues the lawsuit out of principle. So should you wonder why NRA's and Commission often agree, there you have it.

The fundamental problem
The fundamental problem of telecoms regulation in the EU is not incompetence at the national regulator as the EC claims, its the inbalance in power, partially caused by the long time it takes to appeal a decision of a nation and the difference in context a NRA operates in compared to the EC. The ETMA will not solve this problem, but only will make it worse.

ETMA will not be under the oversight of the Commission, but will report to Parliament. There is however no mechanism to prevent the Commission meddling heavily into the operations of ETMA. ETMA can now deflect any direct attention away from the Commission, while the Commission can still run it from a distance. This gives the EC more power in a subtle way. It will not improve the position of the nations, because for an appeal they still can go to the ECJ in Luxembourg. So you got a perpetrator, a fall guy and forever to wait for justice.

Increasing Democratic oversight
As said in the previous paragraph, ETMA will deflect attention away from the Commission, giving the Commission more power in a more subtle way.Parliament is toothless vehicle, because it cannot dictate what ETMA will do, just oversee that it is doing something. The European Parliament just isn't equipped to deal with direct intervention of the Commission in ETMA. Normal ways of indirect democratic oversight of the Commission is also out of the question. These ways are normally barterring and bribing, because the Commission needs the Member States on other issues. The Commission is now impervious to barterring and bribing, because it can deflect any just criticism towards ETMA. Now barterring and bribing are already bad, but this doesn't add oversight, just an extra layer of confusion.

Devalue the role of national regulators
The Commission will use ETMA to lay down dictates of how to regulate. It will require each nation to do exactly the same as elsewhere, regardless of the local situation. Yes the EC will tell you differently, but the Commissions actions against The Netherlands on the cable sector will show you differently. The Dutch cable sector is different than anywere else because of having 98% homes passed and 95% of the people subscribed to analogue and/or digital tv. Knowing this, national regulators will be wiser than to "Think Different". They will auto-conform without ever considering a different option.

Injure the quality of rules and regulations
When conformance becomes the rule, regulators will loose the appetite to properly research their national markets and identify proper actions in line with the national situation. This will make both the Commission and the NRA complacent. The Commission will argue that it's always right because everybody follows it, through ETMA of course. The NRA's will go grow complacent by just copying ETMA.

Lack of demand for ETMA
There is no clear demand for establishing an ETMA with 120 people. These people are going to do stuff already done in the Commission and at eg. ENISA at the moment. Chances are that it will be all new people, who will duplicate the Commission's work. 120 people doing nothing will want to do something and that will lead to more meddling, bright ideas, windows dressing, useless reports and infighting.

Solution
A solution is not easy, but would need to consist of these elements:
- Quick dispute resoluton at the ECJ (6 months). This will make the ECJ relevant and remove time as a weapon from the EC.
- Strengthening of the ERG. If a majority of the ERG agrees with the Commission, than the Commission must be right and the other way round. It's hard for any party to broker a deal with 14 nations to get a favorable ERG ruling.
- Strengthening ENISA to tackle security problems, where they are. The problem is not in the network and the cause is not the telco. So establishing a CTO at ETMA will not help.
- Accessibillity and 112 emergency are already done at the Commission. They are not purely problems of the IT sector and could be handled through normal channels of the Commission.
- Cross Border problems should be dealt with by a proposal of the Commission and a decision by the Member states and not by ETMA

In the coming days I will work through the papers more. I hope to have a look at what a provider of electronic communications networks and services is, Net Neutrality and functional separation.

Update November 21st 2007: For a moment I thought I had misjudged the Commission and that ETMA would actually have something real to say. I thought the 27 regulators would have full voting power and that the Commission would be subject to the opinion of the Board of Regulators. But I've read the regulation and it turns out the Board of Regulators can only issue a non-binding opinion, much like the ERG now. So they are toothless fluffy paper tigers, no chance that he Commission might hurt themselves on the Board of Regulators. Oh well, it's good to know the world is still a sphere and pigs don't fly.

Nokia E51 Released yesterday, NO on November 22nd or December 6th?!?!?!

Well, what can I say... I'm sad. November 12th came and went and there has been no official release of the E51 neither in The Netherlands, nor in the UK. It seems however that you can buy it on Ebay and somewhere in Europe. The guy behind e-series.org has his already and blogs about it.The people of Symbian Review have a review up and love it.

German sites, like Conrad, were reporting that they had it in stock, but now they are reporting that it will be available on the 22 of november. Amazon.co.uk has moved the date up to December 6th. My mr. Fix-it, whose got a good reputation of getting the cool stuff when it's still hot, hasn't gotten his hands on it either. He still has it in backorder. I haven't found a Nokia site yet that sais that it is shipping. There are no press releases either. Nokia Europe has moved the phone to its list of available phones and off the list of phones to be released, but Nokia Germany and the Netherlands both have it listed as "available soon". Most Nokia sites do show a nice promo for the phone, but I don't want pretty pictures, I want press releases and shipped models.

Meanwhile it seems that Hungarians can buy the E51 on the black market already... I interpret the Hungarian euphemisms as: A container full of these somehow got lost and ended up in our shop... that's not stealing is it :-)

FTTH, CCTV and a safer society go hand in hand

Brilliant article by The Register: Residents of Shoreditch in the UK got access to the footage of CCTV surveilance camera's through digital TV. The pictures were in a grainy resolution as to not allow people to identify individuals, so privacy was kind of protected. The results were: IT'S MORE POPULAR THAN BIG BROTHER! Better still, people wanted more of it and in a higher resolution. The focus group response was: "Focus group feedback indicates the CCTV is helping address fear of crime and... generating major new community vigilance resource."

Now imagine, FTTH everywhere and grannies are working from their home as CCTV camera operators. We'll get those nice, spiffy AXIS cams installed, that work in the daylight and nighttime and can deliver HDTV quality. You could install it as a scheme where those watching get a bonus for every crime they report, or as a new work from home scheme. Whole groups of inactives could be crowdsourced. Actually you could get two or three people watching the same scene independently. If they don't know the others who are watching it could be a great scheme to keep CCTV camera operators honest. If two are reporting something happening and number three isn't that person not doing his job.

At high def 5mbit/s or more this will require some nice FTTH or VDSL2 type of connections. (ofcourse there are ways of conserving bandwidth until such a moment when the operator feels it necessary to get a good luck) The traffic is therefore best kept locally, but that should be no problem, as locally there should be no lack of bandwidth in a VDSL2 world. (Note to the Brittish: this doesn't include you. BT keeps you at ADSL2+ which just isn't good enough to hook all those camera's up to at HDTV and most of you will not be able to watch it in HDTV, since you live too far from the exchange)

Combine it with Google maps and let people annotate the events. Even better, store all the information readily available and searchable for the public. Let them annotate it and they will make everything even more clear. Pretty soon you'll have a complete record of every car that drove by a road. A description of every person that ever glanced at the cam. Let them combine their own pictures and you'll have a transparant society! What the German Democratic Republic (commies) never achieved, we can achieve by using camera's and grannies!

Now just think of it, the possibilities are endless and the great thing is peoples curiosity/loneliness will bring Big Brother upon us, without as much as a complaint. Onwards to the Transparent Society and Big Brother be dammed. If you've got nothing to hide, you've go nothing against this idea! (/sarcasm)

There is no economic basis for QoS

This was the outline of a paper I planned to write, but for which I just have too little time to get it finished. My main point is that QoS mechanisms in a network are a bad idea (tm) This is generally examined from a technical point of view. The arguments are either generally that we tried it and it didn't work. There is little research on that evaluates the economical side. The little research that there is, generally argues that QoS mechanisms could work if all parties in a communication chain just work together and the reason they don't is because of the lack of incentives. I belief there are several reasons why QoS can't work and why it is a failure of logic.

Introduction

The internet is broken, so we’re told by scientists and standardization bodies. We need a new internet and research at Stanford, Berkeley, Fraunhofer Institute and various European Union programs will fix it. One of the main points of criticism is the internet’s lack of Quality of Service mechanisms to shape and prioritize traffic and to make sure that unimportant traffic doesn’t hurt unimportant traffic. All this in order to give the end-user an optimal Quality of Experience. The ITU has made end-to-end Quality of Service a central element of the design of its specifications for a Next Generation Network.

There is a great deal of attention in academic research on telecommunications networks for Quality of Service mechanisms. It is often stated that without such mechanisms telecommunications networks will not be able to deliver a stable and reliable service. Both from the technical side as from the economical side there is a considerable body of literature on how these mechanisms will work out in the network and in the business models that sustain the network. In order to realize QoS we invest large sums of money in research programs to fix the dreaded problem. Given the amount of scientific papers and research proposals mentioning the absense of QoS as a major problem for the roll-out of all kinds of advanced and mission critical services, how could we not. Everybody says it's important, so it must be important. Except for one minor detail, despite over twenty years of research and various standards and implementations of standards, nobody is using it.

The idea of QoS mechanisms as being essential to the stability and reliability of the network has been at the heart of the Net Neutrality debate. It also rears it head in debates on how to make a sustainable investment in networks and services. The notion of QoS mechanisms has therefore passed the realm of the purely technical and academic and entered policy debates, where policy makers will have to value the various claims.

This paper will examine in a multidisciplinary way the basis for QoS-mechanisms in telecommunications networks from both a network engineering as an economical point of view. Quality of Experience for the end-user is the end-goal of any network architecture and that is where QoS-mechanisms are supposed to deliver. We will show that the use of QoS-mechanisms to deliver QoE is bound to result in failure right from the start, since QoS through shaping and prioritizing is a logically and conceptually flawed concept. It’s a holy grail and a pipedream. These mechanisms cannot work and therefore building either networks, business models or policy on it will result in failure. There is however a simple solution to QoS-problems and that is to over-engineer the network and all the active equipment (servers, routers etc)

What is Quality of Service and Quality of Experience?

Just look it up on the wikipedia. Also look up jitter and lag etc.

What kind of QoS mechanisms are there?

These mechanisms in general take three forms:

- Prioritizing systems, that let packets move ahead of the que based on how high their priority bit is. (like sirens and lights on an ambulance)

- Bandwidth Reservation systems, that guarantee a certain amount of bandwidth over (part of) a link between two points. (like the telephone network that reserved a line between two points)

- QoS enabled routing systems: Routing systems that try to route traffic on knowledge of the state of the network. (like a driver learning of a traffic jam on the route to work and therefore taking another route).

These systems have seen various implementations and all have failed. There are alot of explanations in literature that explain why QoS is not a success. They can be divided into three different classes:

- it's failure of the previous technology, but we will think up a new one that will get it right

- it's failure of economy, bandwidth is too cheap, implementation is too hard (but this will all change, just you wait and see.)

- it's failure of timing, currently we don't need it, because nobody uses the internet for business critical stuff, but the status quo has to change or we cannot do telesurgery etc.

There is very little literature available on whether QoS is actually necessary and whether QoS is actually possible. If all these mechanisms have any chance of working at all.

Many engineers that design the protocols and networks that have build the internet explicitely and implicitely accept that QoS-mechanisms will not work in actual networks. Most sensible engineers don't even want to get into a debate anymore about it.

Actual implementations of Quality of Service Mechanisms

There are currently several QoS mechanisms standardized for use with the Internet protocol.

Diffserv, Intserv, RSVP, MPLS

Categorisering

The conceptual errors that underlie the failure in implementation of Qos mechanisms can be either technical or economical. Technical errors are those errors that make it impossible to design a technical system that answers to all the demands of a QoS-mechanism for it to be technically functional, stable and reliable. Economical errors are those errors that make it impossible to properly implement and operate a network with QoS-mechanisms. The economical errors and technical errors feed into eachother, strengthening each others effects.

Technical errors:

• Scarcity in the network is a layer 1 problem. QoS mechanisms are operating in layer 2 to 7. We’re trying to stuff more bits into a pipe than can properly fit. Like trying to put marbles through a funnel. Or put differently, trying to fix a layer 1 problem in layer two or three, by making assumptions on layer 7 traffic and on the real world making use of it.

• QoS routing is NP hard

• QoS tries to make the pipe more efficient to allow for more traffic. This only works when the pipe is almost full, but not when it’s fully full. In a dynamic system the difference between empty, almost full and fully full is a couple of percentage points. This leaves very little room to manoeuvre

• QoS prioritization works on the switches that are in between the two users. On a modern system the time advantage that can be achieved by prioritizing a packet through a switch is x millionth of a second. This is less than x% of the one way time of a route of 100km. We’re trying to solve the lines problem in the switch.

• Qos only works if the switches can derive an order in which to treat applications. If all streams have top-priority there is no way to determine which ones should get priority over other ones.

• Bandwidth reservation mechanisms are binary. There is capacity that can be reserved, or there is no capacity that can be reserved. This regardless of whether there is capacity available.

• It may be a straw that breaks the camels back, but there is a lot more weight wearing it down. Removing the straw or one other object might save the back, but the camel remains heavily burdened. Same in networks. Both big and small flows can break a network. It’s the total that counts.

Economical errors:

• A QoS system will have to weigh the demands of all users in order to weigh the highest utility for all. This will require an insight into the utility function of each user and an overarching utility function to weigh the utilities of all users against each other.
  

• There is an implicit assumption that both sender and receiver will value a stream equally high. In any communication, there are senders and receivers. Both have a value for that communication and a value for other communications flows on the same connection. When watching a movie online, the company that broadcasts the movie values the QoE of its customer very highly. It doesn't want the customer to receive a jagged and jittery movie. The customer however is not only watching a movie, but might also be expecting an important phone call or communicating otherwise.
  

• QoS works in a static setting (see technical). However the market is dynamic if its healthy. This will reflect itself in the network as one the main platforms over which market forces exert themselves. One cannot assume a static situation for a QoS mechanism if the data flows will follow market dynamics and grow with growth in population and prosperity (when bandwidth usage decreases, there is no need for QoS mechanisms)

• (variation on above) If market is stable (no growth or declining) there is no reason to ration traffic. If traffic is declining the use of QoS mechanisms after a while is unnescessary. if it is growing than after a foreseeable period there is too much traffic for QoS mechanisms to add QoE.
  

• In many business cases surrounding QoS mechanisms there is an assumption that QoS enabled traffic that has been paid for, has a higher value to the user than data that has not been paid for. This sounds logical from an economical point of view if money is an adequate proxy. However it isn’t. Compare a VoIP call that clashes with a pay per view movie. If the VoIP call is about an important subject (birth of child) than it has priority for the receiver, regardless of the QoS level paid for.

Solution
Overengineer the network, so you don't have a situation where QoS mechanisms are appropriate
On end-user connections let the end-user prioritize the traffic to and from him/her. It's the only one that has an accurate view of its utillity function.