tl;dr :Patents provide juridical and commercial protection and monopoly in counterparty of publications and explanations of scientific inventions to everybody. APIs provide juridical and commercial protection and monopoly in counterparty of opening data, services and resources to third-parties. So, APIs are the enhancer of the business driven numerical revolution as patents have been the enhancer of the business driven industrial revolution.
1) Private property, Intellectual property and patents.
In th last centuries, we had a lot of inventions everywhere, lots of new products but the global innovation and science was slowed by secret.
Each inventor of a technology or a product was not sure to keep its competitive advantage by explaining how the product worked and it was the kingdom of secret to avoid copy or retro-engineerin It was the beautiful days of industrial spy legends.
Everybody was re-inventing the wheel and was wasting time and resources to make what others have already made, and couldn’t build on top ot existing technologies.
"For examples, scientists often kept their discoveries secret. Isaac Newton and Gottfried Leibniz argued over which of them first invented calculus because Isaac Newton did not publish his invention for decades. Robert Hooke, Leonardo da Vinci, and Galileo Galilei published only encoded messages proving their discoveries. Scientists gained little by sharing their research other than claiming their spot in history. As a result, they preferred to keep their discoveries secret and build off their findings, only revealing how to decode their message when the next man or woman made the same discovery. “
The intellectual property was not protected contrary to private property
Private property is simpler to define because it is easy to understand limits on concrete lands and goods. This my land , my house, my food , simple to separate from each other because physical. If I give you a part of it, I have less for me.
If human could put limits on water and air, they would have put it but it is technically harder to separate fluids than solids (They however decided to give to countries 300 km long from coast the ownership of seas, but not for individuals, for the moment) So Humans have always tried to protect their material property, and they had to find a way to protect the immaterial one, in a economy of knowledge.
Then some countries, leaded by Italyand England decided to organize better intellectual property
How protect something which is immaterial was the main question? Where would be the limit of ideas? Because If I share with you a knowledge, I still have the knowledge, so why would we define an intellectual property on something that is not splitable? They decided to not protect ideas, but market exploitations of these ideas, concepts or inventions.
They have put rules on trademarks and industrials inventions by protecting “human actions or processes that modify matter”.
So theories could not be patented but only specific actions enabling new phenomenon or scientific results on matter or material for a concrete use.
Specifically, the right of patent, that interests us here, tells that you can protect an invention if and only if you are :
- Inventing something new that it was not obvious for a average knowledge engineer of your activity - increasing significatively a technology and its results by an inventive process - potentially creating a technology or process with an industrial prospective and business opportunity (This is really why theories can not be protected by patents or even mathematics)
If you respect these 3 cases you would be able to have a juridical and commercial protection on your invention on an geographical aera of your choice during 20 years, as long as you pay annual fees.
The deal is that you have to reveal your knowledge, reveal your know-how and secret to everybody making the science move forward. So companies decided to dive into this protection and have freed the knowledge which was locked inside their company, as long as the business monopoly is guaratneed.
2) The numerical industrial revolution: data is the asset
In the industrial revolution the asset was physical materials. Either Gold, Coal, Copper, Oil, Iron, wood, stone, then product made with them, as cars, buildings, goods, fridges… the asset of the industrial world was physical. Then with the appearance of Internet and the digital world the asset of the 21th century is the data. Consulting companies often call it the oil of the next century.
So the question of lots of companies now is "how to make money with our data or platforms, but not loosing control on it" The same question as before "how to make money with my industrial invention, not to loose the secret and know-how on it"
We have seen lots of companies which didn’t want to go into the open data initiative because they were afraid of loosing control on their data, despite of possibility to make money of it.
I’ve made a scpeific article about it, where I explain that they want to keep control for secure future investments, protect their data juridically for making business and money.
Exactly the same way an industrial wants to protect its potential market and business by writing a patent on its invention, today companies are looking for securing their data openness, and are launching APIs to explore potential markets.
3) Why APIs are the new patents?
Patents provide juridical and commercial protection and market monopoly in counterparty of publications and explanations of scientific inventions
APIs, which provides juridical and commercial protection and market monopoly in counterparty of opening data and resources to third parties.
But for APIs :
- juridically is not defined by patent cooperation treaty but by custom Terms of Service
- commercial protection is enabled by API management solution, by controlling of the openness of the asset with possibility to limit or revoke access to a competitor to your data, so protecting your business.
- the asset is the data, where for patents the asset was the technique and the process
These juridical and technical restrictions and business dependance with specific licensing defined by ToS are a new kind of patent.
Contrary to open source, these open resources keep the control and the monopoly to the provider which has the only right to really make money on top of its ecosystem.
Even if today they are not adapted anymore to the fast technology changes, patents have historically made science and the industrial revolution move forward by enabing safe invention revelation in securing business in counterparty, by juridical and commercial protection during a limited time and area.
Now we are in the numerical revolution, in an economy of knowledge and the real gold or asset of companies is the data.
APIs enable to share more valuable data with juridical and commercial protection and so are the enhancer of the numerical revolution, making the web programmable in the 21th century, as patents have been the enhancer of the industrial revolution of the 19th century, making the world a globalized in the real life economy.
Why big companies didn't jump into open data but will open APIs?
tl;dr : Contrary to public authorities, companies are driven by business and ROI, and the classical open data movement obliged them to loose control on their data, with free licence and free of charge policy. So they decided to wait for technical soluton to secure , protect and monetize ther data opening, and they now launch open APIs which provide this 24/7 control with API management solutions.
1) Open data : first an academic and politically movement
The Open data movement is the promise that the data has to be open, so according to opendefinition.org, it means :
"A piece of data or content is open if anyone is free to use, reuse, and redistribute it — subject only, at most, to the requirement to attribute and/or share-alike. ”
In the philosophy of open source, the open data movement began with academic research in the 50’, with climatology and geology, because the data has to be shared to understand global changes and ecosystem that only your country can provide.
Today the open data movement has been globalized with Internet and this is the case I want to talk here.
Since 5-10 years, lots of public authorities have launched open data to everybody, with many ideas :
- the data has to be free for the people, because it is collected with people taxes
- the data has to be completely open, with a free licence as Odbl, which respect the open definition upper
- the data has to be fisrtly openned for political transparency for political evaluation and Return on Investement calculation by the society.
- then for economical reasons because opening these assets, entrepreneurs will build applications and create ecosystems on top of this data.
Lots of things can be told about the global success of open data, what we can say is that the movement is growing, but the raw material and the lack of data has really put brakes into developer community.
Some platforms like Socrata or Opendatasoft transforms the raw materialin XLS or CSV into REST/JSON API and make developers more confortable working with public open data, and will make the civic open data more and more usable.
2) Why big companies did not jump into open data?
Because companies want to :
- keep control
- evaluate ROI and make short term money
As National laureate of National Open data Contest (with my start-up), and co-organizer of the Datapero (Data&Drinks in french) we have talked to numerous big companies and they are really afraid about open data. Why?
They are afraid to loose control on the data and their re-use and loose money or competitive advantage.
They don’t want to :
- be obliged to have an open data licence (Odbl for example)
- lose control , by not knowing who is using the data and for what kind of application.
So they are unanimous :
“Why we would give to anybody, with a free licence and for free our data that we have collected with our investments? "No open data, we are not politically engaged, we are not activist, we are businessmen and businesswomen"
3) When open APIs answers to all their questions and fears.
By making lots of events on APIs, meetups or conferences, I’ve seen them very interested in APIs. Even Gartner predicts that 75% of Fortune1000 will have a public API in 2014.
By explaining what are APIs and how to manage them with API management solutions, they have found all the answers they were wainting for opening their data.
Actually, with API management solution, companies :
- keep control on who is using the data, with an APIkey registration form which can be revoked at any time, depending on terms and conditions
- keep control on how the data is used , with rate limiting or quota, and by looking at the domain where the API is called
- keep control on the re-usability with a non open data licence but a custom API Terms of Service, defining a contract between a “customer” and a “supplier”
- can better monetize the service, by finding a business model that which is impossible to think with the mainsream open data movement philosophy
- creating a community around the API, with all the apikey delivered and personnal informations into the developer registration form.
- calculating with all these management features upper the global ROI of the opening intitiative (which with the classical open data is not really possible)
So we are seeing companies which were saying never to open data, opening these data through an API !
It is also very unexpected that because APIs are the way to open data and resources in a secured and controlled way, companies now expose more data/resources and more valuable data/resources. They were opening no valuable data just because of the lack of control! And it can explain a lot of things about the delayed success of open data.
To my mind, the Open API movement is included inside the global Open data movement, more restrictive, but more business and easily monetizable.
The real question is does the open data movement will benefit of open APIs, or does open APIs will kill the open data movement as free to use, free of charge?
After all, a data opened with a very restricitive licence, or access is already an opened data? All the data will not be as open as the definition of opennness, you can read my previous article "What is Open?" but may be APIs can be the first step, more secure to open “closed data”, then to free the data as a global asset for everybody.
Lots of companies have understood why they need to be open, for evolving with their ecosystem and benefit of all the external work, expertise and ideas of the “crowd”.
These companies now say that they are open, with open APIs, but what is to “be open” for them? They often don’t have the same definition.
In IT and logic doors for example, this is binary. You are 0 or 1, you are closed or open. If you are not closed, you are open. No middle. After all, a door a little open is an open door right?
Bur for business as for APIs , the notion of open is more tricky. Let’s take the example of my house. My house has an entrance door with 4 stairs before, a kitchen, a bathroom and many rooms on 2 floors.
Would you consider my house open if : - I let only few selected people come into and let the other outside? this the the question of neutrality - I let the door open to everybody, they can go in the 1st floor but nowhere else in the house? this the question of exposed ressources and transparency - I make it $1000 for entrance fee? Or if people with wheelchairs have to go up with 4 stairs?Or if I ask a secret password? this is the question of access ans authentication - I let people enter but they have to follow my rules, by avoiding specific debates or touching nothing? The question of freedom, hackability and re-usability
So, what is an Open API?
- Transparency :
Transparency is how far the company exposes its internal assets, services and APIs to developers.
To be open in this case is :
- exposing the same ressources (as far as possible) you use inside the company to the third party - notice about the API strategy, changes and explaining to developers their roadmap
This for example how Amazon exposes its EC2 or S3, Salesforce expose its APIs or Google exposes Appengine or Bigquery.technology. For example, Governments in the open data movement have to expose the same ressources as they use internally.
- Access and authentication :
Access is the fact that the API is accessible to a user. It is also about the work you make for : - a developer portal to make it easy to register - API design to make it easy to use - Authentication protocol to access to a data. In this case open is to make always the lightest protocol needed to access to a ressource. - The business model and the price of the APIkey. In this case open is to have at least a free version to test and build already stuff that enables a good user experience for few users
- Freedom and re-usabilty :
This is the question about Terms of Service and Developer Policy. It is here about how developers have the right to hack on your API and use it for their own purpose with their own model.
To be open in this case is :
- having Terms of Service well explained and simple to understand - letting it hackable with no restrictions to specific use cases or business model restrictions - not to say that you have the right to revoke anybody for any reason (often without notice !)
- Neutrality :
This is the fact that how far an API provider will have the same behaviour concerning :
- Access, - Transparency - Policy, rate limits and Tos - Quality of service without tiering
with every API user.
So open is not a binary logical choice as we can know in computer science. It is more some time philosophical , or even juridical than only technical and it can be represented as a patchwok of transparency, access, freedom and neutrality
Have you any thought to add about how to define better what is open?
Edit : I found a Openness definition here, from opendefinition.org, and it follows the upper principles, but more detailed about general data.
The web is developer centric : Marketing to developers
1) How the web has become developer centric ?
- The web content centric
In the 1990’ the web, aka web 1.0, was readable with only text, pictures and pages. In a word, static or content centric. The biggest difficulty for users was to find the data we needed into this growing “library”. Some ones tried to make directories but game changers came with search engine in late 90’. After execution , business model, and pertinence trust, the one which won the battle was the next billionar company. Google has won the game by making good choices, neutral politic on search results, and nice execution with great technology. - The web user centric
Then came the Web 2.0, the web dynamic and social. The biggest difficulty for users was to follow people interactions and identity to know with who we are interacting, because this web would come user centric. Lots of companies too tried to surf on this wave, like Friendster or Myspace but only a few had understood before others the importance of really connections between users, dynamic feeds and user generated content and databases. This is where Facebook came and ruled them all into this social web. In a second phase, Twitter came with its microblogging service confirm this trend. They are today respectively King and Prince of social web, but far behind the Lord of the web Google which still dominates the content web (Yes the web2,0 in on top and still dependant on web 1,0).
But Twitter and Facebook with their APIs opened a door that is not ready to close for a long time.
- The programmable web , developer centric
Web 3.0 ? What does it mean? To my mind nothing. But what is the web on the 2010’ ? I may have an idea on it. This is what we call the programmable web. With a lot APIs out there, now more than 8000, 100 new ones per week and 1 million expected for 2017, we assist at a revolution which is developer centric, because only developers can us these APIs.
In the 21th century, data is the new goods, produced and stored is servers, distributed on information highway through APIs, transported by developers into application promoted in numerical supermarkets, where end user will come to download them.
This web is still on top of content and dynamic and social, but is growing even faster than precedent one. Now developers are making the new Kings of the web, using their APIs for making lots of applications and building new models around these Extended Enterprises. Now thanks APIs, company services and data are ubiquitous for perpetual innovation.
Each new web trend is included inside the precedent one but growing faster than the precedent and making the old one growing too. The reason why IBM explained that the 90% all data produced in all human history was in the last 2 years.
But who will be the giant of this programmable web? The one which has understood that this web is application centric so developer centric.
Companies have to change the way they acquire users and developers and change their marketing of vaporware or bullshit into a marketing of utility, to a maker or do-er generation.
This is the case for example of Twilio or Sendgrid which invests a lot in Developer Evangelism and focus on makers for their API.
The new question that companies have to manage now is what developers expects from an API usage?
2) Marketing API to developers
Skills : Developers want to discover new possibilities and improve their skills when building application. For example Twilio enable developers to make their application text or talk by SMS or calls so enabling a new user experience to users that the developer know how to do.
Tools : Developers want to ease their life when making application or website, so they look for tools for avoiding boring stuff. For example Sendgrid helps them to monitor e-mailing easily and rapidly to focus on their application development and users or Github enables them to better manage versionning
Money : Developers are like artists and they love live and earn money by doing what they love : coding. So APIs like Google Adwords enable them to earn money when used, or for example Amazon.
Cause : Developer are makers and do-ers and often are great contributors on internet. So Causes like Openstreetmap or Wikipedia, or open source on Github can often leverage them to build better, faster and stronger things and projects.
So the marketing has to evolve to be focused on do-ers in this programmable web completely developer centric and companies has to understand it if they want to benefit of developer communities.
On the API rating agency blog, I said many times that the programmable web is behaving like the brick-and-mortar business, where APIs are like companies and where we have to make webservices supply chain management as we make industrial supply chain management, and inter-webservices connections like inter-industries business.
I wanted to share with you an interesting case about how, Today the 18th of January, French Government (following my old comparison between how webservices works and how goods are distributed), wants to adapt the traditional intercompany VAT system to inter-webservices Numerical VAT system.
TL;DR : If you give back the data you collect to others, via an API, no taxes for your company. If you keep it for your own purpose for any reason, taxes.
How does the classical Value Added Tax system work?
France is the country which has invented in 1954 the Value Added Tax (V.A.T.) which is now the most widespread tax in the world.
The idea behind this tax is that only the final user, the end user, who doesn’t add value to the product, is the payer. So, as soon as you add value to a material or a product, you can sell it without taxes to a 3rd party.
Guess you are a company and you buy a product, you have to pay the VAT in advance, but if you add value to it you’ll sell it with VAT too. Then, the government will give you back the VAT you paid, and collect the VAT on your customer. This is mainly the case between inter companies sales.
But if you don’t add any value to a product and you use it as the end user (often BtoC), then you will have to pay the whole VAT from all the companies which have transformed the product.
The spirit of this tax is that If you make it better and give it back to others, no taxes. If you want to benefit from the product for your own purpose, taxes.
French government asked two tax law specialists about how to avoid that Google, Facebook, Apple , Amazon, Ebay and even Industrials companies etc…skip taxes with the traditional system, in USA, UK, and France) The report was publicly annouced today.
The French Prime Minister asked in Novemebr a study to 2 specialists, Colin and Collin, for proposing solutions about
- collecting value from webcompanies which are avoiding taxes and
- make a better web
Their conclusions : a Numerical TVA for company which doesn’t free/open their data.
Apply VAT to numerical content
The study is long, (and in french) but to put in a nutshell, it plans to tax companies which don’t open the data they collect, and the main recommendation is to enable then open APIs. You can access the whole study here.
The main idea is that the most a company open the data they have collected on customers or users, the less they are taxed. And not only Google, Facebook or others, all industries are in : Banks, Energy, Food, Cars manufacturers etc… The user would have access to all his data at anytime, following movement of the customer empowerment like the “smart disclosure in Us” or the MiData in UK”
You can see as a Connect-ification of the web (i.e Facebook connect , Twitter connect, Github connect) where each company will have to give back all the data they have to the user and would have the possibility to bring them with them in each application. Let’s go for a MyBank connect, MyEnergyProvider connect, MyCity connect etc…you will be able to allow any 3rd-party service to access the level of data you want to share in order to benefit from the User Experience of each application. It’s the spirit, today, of the Scope of APIs when you are a developer and you ask yous users permissions to fully access to your application.
What would be future applications?
After the “Quantified Self” with Fitbit API for the data I produce every day. For example, we may have a “Quantified Me” with all the Data I produce in the Real Life Economy with my financial, banking, environmental, commercial data.
Don’t fool ourselves. The first goal of this tax is to collect money. But maybe in a smarter way than just “pay because you make money” but more “pay because you don’t give back and share data you’ve earned for free”. The philosophy is that the end-user is the data and the content creator. The aim here is to free/open innovation and web services on the web where the data owner will have the right to bring it with him to any 3rd party application. These Open APIs will respect neutrality because everybody will have access to his own data from company which want to avoid taxes, following my proposition of API neutrality and following the Net neutrality. If a company like Twitter want to avoid taxes, they won’t be able to choose which one they accept to share data with. Everybody, or nobody, so it’s Neutral.
The solution would be probably a graph database scheme applied to personal data, but can Government and tax law incentives can really make it happen?
Lot of questions pops up about how to follow and check all the data that are shared and how to collect taxes on bits, that are not in application yet. More to come with the debate that just begins in France and will be for sure continued in UK and US which have already these questions about how to fairly tax Giants of Internet.
According to savetheinternet.com, the Net Neutrality means that Internet service providers or governments may not discriminate between different kinds of online content and apps and users. It guarantees a level playing field for all websites and Internet technologies without any intervention.
Following mainly 2 technical principles:
Absolute non-discrimination : all content, sites, and platforms equally distributed on the network First come first served : No enqueuing of data packets because of fees.
You have to add also no censorship, affordable access etc…
Because of business we today have more a Limited discrimination without tiering
That means if you don’t pay specific fees for quality of service, you cannot have a better network than someone else on the same network. Also, if you pay for a high level Quality of service, so you’ll benefit of this high level quality of service, but in the same condition than an other customer paying the same fee.
Net Neutrality is the reason the Internet has driven online economic innovation, democratic participation and free speech. It protects our right to use any equipment, content, application or service without interference from the network provider. With Net Neutrality, the network’s only job is to move data — not choose which data to privilege with higher-quality service and which to demote to a slower lane.
Now we have a programmable web, through more than 8500 open APIs and a lot more partner and private ones. And through combined applications using plenty of APIs, now the final user quality of service depend of API providers.
"The principles and factual assumptions that animate network neutrality— that the network has been operated in a particular socially beneficial way and that, especially in the absence of effective competition, it should stay that way—can also apply to Internet services that solicit mash-ups from third-party programmers like Google Maps or Facebook”
And when twitter has changed its API policies for all developers, it was a second kind of non respect of API neutrality.
Because twitter was providing an open API, when 3rd-party developers made a business which becoming competitive with their platform and business model, they decided to restrict it, enforcing their position on their own data and ecosystem.
The issue is that it exactly looks like an anti-trust behaviour. Twitter has a dominant position on Twitter data (obvious), but also on all the Twitter ecosystem itself enabled. This is the same as ine the case of USA against Microsoft case for Windows and internet explorer anti-trust issue in 1998. Microsoft enforced its dominant position on Windows by providing a web browser without giving enough access to other compnaies to build one , so less possibility to users to install an other one, and so enfrenging the sherman anti-trust act. (Funny, that the case obliged Microsoft to give access to Windows API to third party companies in a fair way)
Jonhathan Zittrain in its “The Future of internet” takes the following examples :
Those who offer open APIs on the Net in an attempt to harness the generative cycle ought to remain application-neutral after their efforts have succeeded, so all those who have built on top of their interfaces can continue to do so on equal terms. If Microsoft retroactively changed Windows to prevent WordPerfect or Firefox from running, it would answer under the antitrust laws and perhaps also in tort for intentional interference with the relationship between the independent software makers and their consumers. Similarly, providers of open APIs to their services can be required to commit to neutral offerings of them, at least when they have reached a position of market dominance for that particular service”
But it is what Twitter does with its API !
Here is Microsoft = Twitter Windows = Twitter API Windows app = Twitter official apps Competitor’s windows app = Twiiter API 3rd party app
And working these days of API Terms of service, a lot of API providers don’t give public all their rate limits or quotas, and say openfully that they can up to their own discretion make differences between third party developers. As you serve me, i let you quiet, even if you pass the limit i said. Even Google, Facebook, Foursquare, Klout, ask for contact of you think you’ll pass through the limit and examine your business. This is not neutral.
What could be API neutrality?
Absolute data to 3rd party non-discrimination : all content, data, and views equally distributed on the third party ecosystem. Even a competitor could use an API in the same conditions than all others, with not restricted re-use of the data.
Limited discrimination without tiering : If you don’t pay specific fees for quality of service, you cannot have a better quality of service, as rate limit, quotas, SLA than someone else in the API ecosystem. If you pay for a high level Quality of service, so you’ll benefit of this high level quality of service, but in the same condition than an other customer paying the same fee.
First come first served : No enqueuing API calls from paying third party applications, as the free 3rd-party are in the rate limits.
There is a solution. Pay-as you go or other paying model insures you API neutrality. Because every $1 has the same value.
Even, some freeAPI from companies which are profitable are trustable, but when it is free, you are the not the consumer, you are the product. So the provider has a control on you, and if you are not a suffisant profitable or usccessful third party app, you may be cut at each time.
Never forget that free openAPIs provided by companies may be dangerous for your business, because they are governed by opportunity and if the API is not enough profitable, it will change in a not neutral way.
As Johnathan Zittrain says too that
"Skeptics may object that these relations can be governed by market forces, and if an open API is advertised as contingent, then those who build on it are on notice and can choose to ignore the invitation if they do not like the prospect that it can be withdrawn at any moment. The claim and counterclaim follow the essential pattern of the network neutrality debate."
Only free APIs from NGO, like openstreetmaps or wikipedia could be trust and considered are neutral ones. Because the data is already free.
Who is ready for defining better these rules and may be make with us a developer petition?
Microsft Popfly stopped maintaining the platform not because of a lack of success but because no monetization or products could be made on it.
Google transformed its Mashup Editor made for everyone, into App Engine that was designed specifically for developers.
Yahoo Pipes is still alive, but it is based on the Yahoo Query Language which was made, guess what, for developers ;)
To my mind, monetization of old automated API and mashups platforms failed for one main reason.
The main reason is that programming is for programmers.
Non-developers made cool mashups fast, but without enough value to pay for it. Programmers were too restricted for making smart applications. No enough added value, no business, no money, no future.
But Everything can be learned. Programming is not some secret rocket science. It is like any foreign language: you just have to practice and learn from your mistakes.
Does Chinese seem difficult to learn and speak? Well, more than 1 billion people on the planet knows Chinese! Even children ! ;)
This would of course take some time and work, and anybody does not become a Chinese poet or receive a Nobel Prize in Literature but still, you can get a decent level by learning and practicing.
Do you really think you can learn a foreign language by drag-and-dropping words together? You will be able to make word associations, sentences, even a little text but nothing very smart or subtile.
Speaking is not the only association of words.
This is the same for programming and combined APIplications !
I concede that you can do more with a programming language because most of them are logical and with automated tools you can assemble more advanced bricks and make more advanced constructions than in natural languages.
IFTTT, Elastic.io and Whenauser and other API automated platforms are wonderful services to make fun and useful mashups (they call it for example recipes on IFFT) but will be always limited in all the different inputs and outputs possible, which needs to be converted in human language, and this for 7000 APIs and more everyday.
Maybe with a billion input/ouput bricks it would be possible… but it would be faster to learn how to program and learn how to read an API documentation.
Building an API rating agency, I strongly think that APIs are made for developers, because it is a professional tool and if you want to exploit all degrees of liberty with APIs, you need to understand programming paradigms.
If we want a good API industry, we need to have API providers make good designs and do good practices in order to ease developers. In such a ecosystem based on trust, APIs would be much more stable and if they happen to change, changes would be done in a respectful and preventive way.
Hence the API rating agency made for helping developers.
When an API changes, all your business, application or website can be weakened, perturbed or even broken. Although you really are aware of the risk, you somehow blindly trust your API provider and unconsciously hope that it will find a solution in time, for you.
However, you may be using an API that has dozens, hundreds or even thousands of developers in its ecosystem and you may be the last person to know that the API is broken and your app doesn’t work anymore!
And it is even worse when the company outsources software development.
Let’s see what are the solutions today to monitor API changes and manage efficiently your composite application supply chain?
I identified 6 ways for that:
1) Watch the API’s official blog
You will find information about changes, versions, their roadmap… maybe there is even a changelog to help you see the differences between versions.
Some providers have a newsletter, subscribe to it, it is important to stay in touch with your API supplier.
2) Following the Official API Twitter account
If you want to get instantaneously informed about API changes, you can follow the official API’s Twitter account, if there exists one. I hope that the API status Twitter account is managed by the API team inside the company and not an badly managed outsourced one, otherwise there might be some hours of delay for status updates.
3) Programmableweb blog
Programmableweb.com is the main directory of APIs, they register APIs and mashups since more than 5 years, and blog often on API enconomy and API changes/breaks. It has been founded by John musser an official twitter are @programableweb @jmusser @adamd (editor)
4) Hacker News
Hacker News is the forum of Y-combinator (the start-up accelerator founded by Paul Graham). Here you’ll find discussions about programming and business topics. Discussions about APIs are often very interesting and start a whole thread of comments. You can check few minutes per day to check what’s hot in the developer’s ecosystem and API ecosystem.
5) Other Twitter accounts talking about the API industry
Twitter enables you to find information from others tweetos! Don’t hesitate to follow the accounts of blogs or companies dealing with the API ecosystem such as @progammableweb @kinlane @3scale @Mashery @Apigee @Layer7 @SOAsoftwareinc @Alu_cloud @mashaper @Webshell_ @Temboo for example.
News often come from articles on Gigaom, Venturebeat, TechCrunch and so on, so you might get the information via other tweetos as well.
6) When your application breaks :(
The worst way to know that an API has changed, it is the hard way: when you receive claims that your application doesn’t work anymore.
So every now and then check your application and be sure that all members of your API team receive your customers’, users’ alerts by email at your firstname.lastname@example.org or email@example.com address.
But the most important piece of advice we could share is: count only on yourself to find the information!
Trust in an API provider and its API is exactly the same thing as in real life trust in an classic industry supplier and its production and factory.
Is the company consciensous with involved employees? = API team trust
Is the company have good support and take care of customers? = API team trust
Do the provider notice for prices and commercial conditions changes? = API policy and business model trust
Do the revenue model is a viable one and give trust in the future? = API policy and business model trust
Do the company has live user applications and developers involvment? = API community trust
Is the company has revenues, so the API can be viable in time? = Trust in the company
and a lot of other comparisons examples with production supply chain.
So, we need to understand what is important for developping a business using an API, and rate each of these elements in a nutshell to give fastly a picture.
Here are the 5 criteria which compose, to my mind the global trust rating of an API.
Each criterion (design and documentation, policy, API team, API ecosystem and company) has a rating from A (best) to D (worst).
The following simple rules will yield the final rating in terms of those more specific ratings.
The final rating is necessarily the worst of the ratings of all the criteria. This is then weighted with +/- signs according to the other ratings. Let’s take the example when the worst rating is C. If there is exactly one C rating, then the final rating will be C+. If there are exactly two C ratings, then the final rating will be C. If there are three or more C ratings, then the final rating will be C-.
If one criterion has a substantial risk then the whole API is considered risky for your business because criteria are all interdependant.
So what’s the rating of your API or the APIs you use today? What criterias would you add? or change?
Webservices supply chain management : the SIRI case.
Mashup and webservices supply chain management and API risk management will be a full time job tomorrow. Let me explain how , finishing by a concrete example on SIRI.
Today, as developer, you may use several APIs in your application, in an advanced mashup or combined application.
And then you’ll have to manage many providers, data sources, performances, quota, rate limit, terms and conditions….this is not so easy.
Also, many APIs may mean multiple API relations and API risks to manage for your application.
This is what I call Webservices supply chain and risk mangagment.
Exactly in the same way as a automobile industrial has to manage different sources of supply for car construction (tires, wheels, lights, mechanical piece etc…). In this case, this factory has to manage competition, nomenclature, purchase conditions, performance, delivery, financial risk that the company fail on each component ! It is a huge an important work.
In a mashup or a combined application , this is the same, you’ll have to manage each, each brick provider as each API provider this way. Even more because in the cloud and public APIs it is a lean supply chain !
For the mashup supply chain, each developer or start-up needs to behave as a cloud purchase and supply manager.
Once the idea of the application or the mashup is ready, this manager will go on the cloud marketplace to find the best suppliers for its project.
These differents missions, will be :
Sourcing of all the API providers to find the best partners which provide an reliable API to build this mashup
Exploration of the API and the specificity and the quality of datas inside. In the same time, a risk analysis needs one each API provider you see to be sure it is reliable.
Check for performance in a free trial if possible and look for policy and terms and conditions which are in phase with your project.
Contact the company, or direct apply for an API key (even purchasing)to access to the API and negociate premium services depending on your need.
APIs responses aggregation, data treatment to make them interoperable
Make previsions on your probable traction and usage for scaling with yours different API suppliers
Optimise your delivery conditions, negociate a SLA of your different provider depending on your user performance needs, and find solution to manage performance , caching for example.
Don’t forget to stay aware of news API versions from your providers and keep contact with API providers to know their roadmap and avoid obsolescence of your application, and risk survey for be sure that your today supplier can supply you in the future.
This a way to manage efficiently different APIs from differents API providers in an application or a mashup.
Now some clues to understand the SIRI case !
Please imagine a API supply chain management for SIRI which gather many dozens of APIs ! It is a full work you don’t have to minimize if you want a full time good user experience and maintain your mashup for long term.
I recently had a discussion in June with the director of SIRI at Apple and SRI (Stanford Research Institute) International members, which are at the base of the AI of SIRI.
"Siri will be building on an ecosystem of Backend Cloud APIs. In its simplest form the API would declare the meaning of the data going in and out via ontologies that have been pre specified reachable by Siri on the Internet. Siri will than build a response on the Fly from the API data. The concept of ontologies-as-specification is the hallmark of Tom Gruber (http://tomgruber.org/) CTO and founder of Siri Inc and now at Apple working on Siri. Tom’s approach to the challenges of reaching out to the data on the Internet and getting back something useful is quite revolutionary in that it does not require a “Semantic Web” ecosystem. Through APIs and the land rush that I postulate will develop around this ecosystem getting at relevant data will become rapidly easier.”
To go further with a concrete example, Apple deposit a patent in January 2010 presenting a part of its SIRI first imagined webservice architecture.
In the patent, 229 pages, (so long !) they explain a lot of things and they present the vision of the january 2010 ontology and webservice architecture. You can see an concrete example about restauration ontology here and imagine the articulation of APIs behind it.
It begins to be complicated, right? And this only for technically finding where to eat…we are still not talking about performance of APIs, API ToS etc…
when talking about all these issues, they explain me too that the webservice supply chain was so difficult to maintain for Apple from such providers and new ones everyday, that Apple has gathered everything into 5 big consistant services working together since this date, for a simpler and scalable cloud API backend.
They did not go further for me to understand exactly how it works today, I completely understand that is secret Apple know-how, that I will not know what is behind the "Walled Garden" but the fact is that they had to structure the API supply chain management because of so many webservices providers.
In a programmable web with more and more useful APIs everyday, where a lot of start-ups make their innovative product based on 3 or 4 APIs in average (often more), this API supply chain management begins to be real. You already know it with combined application maintenance ! or the changing everyday Twitter ecosystem, that even investors don’t want to trust anymore.
And knowing this, the API rating agency I am building goes in this way, to better analyse the API provider in a ecosystem developers can trust for business.
That the API economy would have some power to find sanctions to this kind of behaviour and the global trustness rating of a such API and company would be publicly downgraded from “liable API” to “well designed API but may highly perturbs your business”.
I’m making this notation system, you will soon see what does it looks like.
Just like in real life (with credit and business transactions), our entire society is based on trust.
We project ourselves into the future each time we have to make a choice, be it when buying a car, buying a house, planning a travel, starting a company and so on. In order to make these choices, we are, consciously or subconsciously, believing that the environment won’t change significantly. But what if there is a war? Or your bank goes bankrupt? If we start to believe too much that the future is not safe enough, we probably would not have all those kinds of projects, but rather look for some short-term provisions and storage.
We can imagine a future because we trust in others, we trust the society.
With developers, this is the same. We trust in a programming language and a source code because we trust other developers to update it or maintain it in the future. We trust the ecosystem. Either for open source or for APIs.
This trust in the real life society is based on relation between buyer and supplier, or loaner and credit owner. Due to a lot a default of payments, or even impromptu bankruptcy, we decided to rate each one to resume the global trustness in a nutshell and give a score or business liability.
We have this kind of notation in BtoC with banking credit score or insurance rating, and in BtoB between intercompanies credit insurance for example, even Government credit rating. This is the job of companies like Standard&poors , Fitch or Moody’s.
In the cloud, we trust other companies based on business and technical history, reliability and performance, and it seemed work well in the past.
But today, too much API suppliers :
changes terms and conditions without any prevention, (Twitter with its API or or google with google maps API for example)
break an API after a acquisition, facebook with face.com API or Urban link with simplegeo API)
are launching their API and changes bruptally API functionnalities then they update an new version.(we could name drop a lot of people here)
have financial issues and break their API because they stop the company, this is the case of start-up but the ecosystem has to be aware of that before.
The first impact is directly on the developer community which has to loose time for maintenance or in the worst case has to change completely its application because the API they used was a single point of failure in their business or application
But in all these case, the final impact in on the API economy industry and send a bad signal to developer community that they cannot trust any one else , and need to reinvent the wheel instead of using what other developers has done well trough their API
APIs are a wonderful help me help you system that make everybody advance together, making innovation on top of the ecosystem already built.
The developer community needs confidence and trustness into all webservices which compose the programmable web, because no one can spend its time to survey the thousands of APIs.
This is the reason why that the API economy is enough mature to have a API rating agency to give in few seconds the global trustness and quality on an API for all developer community.
The rating will be based on global trustness, quality of service, design for developers and open policy.
It needs to be crowdsourced, to be more neutral as possible, and more powerful too to really make lobbying on API best practices and change behaviour of companies with developers.
If you satisfy one of the top levels but you happen not to respect a condition from a lower level L, you automatically get downgraded to the level L-1, even if you satisfy the majority of needs in the upper levels. So every level is exclusive.
The following hierarchy was made with the mind of a developer who has business expectations. It goes from the wow-this-API-provides-access-to-data-that-rocks mindset to the my-business-relies-on-that-API-and-my-customers-love-it mindset.
Access: does the API exist?
The purpose of an API is to share data or functionality. If there is not even a minimum viable interface for developers to access the data, there is no API. So, at this level, it is often an alpha version without any application design, some even share a FTP.
Here the reaction may be: cool data, but they didn’t make their work properly :(
Consistency: is the API consistent?
Now developers want some architecture or protocols to go fast in API usage and want some reliable performance to think to use the API in production. It is often the beta phase.
The reaction would be here : yes nice API! i maybe use it in the future when it would be complete.
Design and documentation: is the API developer-friendly?
Now the API has some status history, performance management and API team begin to make it developer friendly with URL design, interactive documentation, code samples, exploration console, SDK for many languages, and prooves of investment in community with forum and support. This is the phase of new release with versionning and suffisant time to completely update to the new version with deprecated time.
The first reaction here would be : Wow this API rocks ! Is it free? what is the quota/rate limit? :)
Policy and business model: is the API model enables me to use it like i would?
Here is the business phase to give some prospective to developers about the API policy and trustness on API in business.
Here terms and conditions are clarified, quota or rate limit well defined and a business model is proposed on whom developers can invest time and money in creating a viable business and plan some future on it.
The policy and business model may evolve, but a company which would change terms and conditions without any warnings or which would make changes that highly perturbs existing businesses or even break them could not go up this level.
The first reaction here would be here : Great API. I have an idea of application !
Confidence: do I trust the company which provide it that will keep all things well for me in the future?
Now we are speaking about global trustness. Here a developper will check status history, roadmap, API team involvment, in blogging and evangelism, developers community ecosystem of existing and running applications, growth of the company which provide this API.
For a non profitable company or an emerging start-up is hard to be in this step because the forecast is hard to draw for making some projections. To be there, the API policy or pricing has to not have changed for a long time in a way that has perturbed the ecosystem.
The reaction here could be : “They understood developers, they made everything to ease me to use their API. I have an idea of a company ! I realised that no existing solution exists for ….”
API neutrality: do I invest my time and my money on this business?
Here is the “all you expect API”.
This step is today I think only reserved to API-driven company or companies which ARE an API and sell it only. At this step, policy of the API is very open and prooves that it would be in long time. The business model is a proven one and viable to ensure everybody that it will not change, often in a pay-as-you-go model.
Beware of non-profitable companies Free Open APIs of companies which are not profitable could not be in this category, because they are at the mercy of investors short-term strategy and have proven in the past that they were not trustable.
The 20th century was also about the arrogance of giant companies.
Banks and insurance companies presumed to do the worst short-term businesses in order to make yet more money, without any vision, doing high-risk business, with no consideration for final customers.
They were doing it because they managed to become the basis of the whole system. Those companies knew they were getting so big that, no matter the gravity of their mistakes, they would always get rescued by the government, thus by the people.
We were their customers, we were buying their products, they were redistributing some part of the profits to shareholders: everything was normal.
But when they fall, we have to pay the price for it: we happen to be the last stupid investors of an undead company. Why?
Because they were too big to fail.
It became too dangerous to let them down because all the system was based on them and we would not be able to bear the consequences of a default or a bankruptcy, we could rapidly dive into chaos.
This is, for instance, the story of AIG which has been rescued by the US government after the subprime crisis, or the story of the current Greece and Spain debt crisis which resulted in a rescue plan by the European Union.
The same applies to the API economy system. It is based on trust and long-term vision for developers, but often short-term vision for giant API suppliers.
When developers integrate APIs into their applications, they make a long-term loan from API suppliers. Most of the time everything is fine, business goes well and the ecosystem is growing.
However, some APIs have taken a large part of the market share in their field (Google Maps for Maps, Facebook for personal data or Twitter for tweets for example) and are now too big to fail in the API economy.
They all are the base of an ecosystem of mashups and applications. A lot of startups and developers have based their business on these kind of APIs and are now at the mercy of them.
For example, Twitter has now an estimated more than 1 million 3rd-party applications, Facebook more than 600 000 ones and Google Maps represents 40% of all the referenced mashups according to programmableweb.com and is now on more than 350 000 websites providing a map, and it is natively integrated on iOS and Android.
But can we do business without them?
When there is enough competition, it is indeed possible. The map ecosystem, for instance, has enough competition to provide other trustable API suppliers.
This is the case for Facebook and Twitter. Facebook has the monopoly on friend feeds in the largest parts of the world (exceptions are Russia, Brazil, India and China) and for now Twitter is the only service dedicated to tweets (seems obvious, I know!).
Concerning the latter, for 2 years now, it is even forbidden to provide complete tweets in a 3rd party Twitter client in order to preserve the official Twitter experience on twitter.com.
Developers have to know that if they base their business on these fabulous social media companies, they are walking on a rope where they can fall at each wind of API change.
And this wind of change, terms and condition change, or quota/rate limit change can come suddenly, without any warning. These providers had this behavior in the past; they will have it again.
Developers have to know that by using today this kind of free monopolistic open APIs coming from companies without a proven business model, they are betting on investors’ monetization strategy that can evolve and change very fast, without any warning.
I would strongly advise developers to preferably trust companies with a proven and viable business model. Even more if their business model is based on APIs. This will greatly enhance the probability that their policy stays the same in the future. Only idiots or short-term investors would change a winning strategy.
If a company with an API, goes out of business or breaks its API then developers, just as insurance customers we all were in the AIG case, will pay twice. First, with their development time and second, with the time they spent on maintenance or on replacing the API with an alternative.
The API economy is growing and developers all around the world seem to be happy with that.
APIs are an amazing way to open data and services to other developers. API suppliers give them access to some of your components for free or for a fee. This way, they can build amazing applications right from those components, focusing on the value they bring to the end-user.
Developers are the blood of this API economy and APIs are the muscles, giving the air and the energy to them, making these components work together in API- or composite APIs applications.
With a more and more coherent and structured cloud ecosystem, we are getting able to accomplish greater business services to help everybody live better in a cloud century. With Apple’s Siri (that used 40 APIs in 2010, and 55 APIs as of 2012) we now entered a new era of mashup applications. However, it is very important to keep this tendency.
According to Programmableweb.com, there are more than 8500 APIs with 50+ ones every week. Despite those good signs, some API suppliers have turned to the dark side of the Force, which testifies that the API ecosystem today is still fragile.
Those suppliers don’t respect developers enough. They often break their APIs or change their terms and conditions without any warning and just really don’t care about developers who actually make the 3rd party application ecosystem and provide feedback to improve the APIs.
They are just wasting developers’ time.
I would like developers to put positive pressure on API suppliers to make them respect their developers’ business and work.
Some of them are fair and play the game and I will talk soon about who they are in this blog. The others need to get more transparent and respectful with developers. Because developers make them who they are. By harming developers, they are harming themselves.