Goodbye, centralised data marketplaces. Hello, Ocean Protocol
Decentralisation of networks and entities is picking up at a great pace and I think it is time to say goodbye to centralised entities and embrace decentralisation.
Secure data containers
Before joining Dex, I worked for two years on building a secure container for sharing of data to enable data analytics and AI — a place where organisations could merge sensitive, precious data without losing control of it.
I didn’t have to convince organisations and governments on the power of data, but I hit a wall when it became clear they could lose control of their data once it left their premises.
My realisation was that, for the secure container to enable such data sharing, a neutral trusted data marketplace was needed. It is while looking at examples of data marketplaces (which could be the neutral places to enable data sharing), that I encountered the work of DEX.
DEX had proven itself as a leading Data as a Service company in Singapore, enabling data sharing. As a data marketplace, DEX tested many models and saw the same challenges to data sharing I experienced. This became even more apparent as we built on an international data sharing programme together.
Centralised data marketplace
In early 2017 I joined DEX as a CEO to further my vision of making data available to the data consumers in a scalable, fair and sustainable way.
At DEX we were in effect creating a centralised ecosystem. This posed not only huge scaling challenges but also raised questions about our readiness for new regulatory environments such as GDPR.
In these centralised data marketplaces, issues of scale were based on the following observations and experiences:
- Trust based on relationships and not on technology;
- Contracts are on paper and not easily technically verifiable;
- Audit based on trust and not on immutability;
- Provenance based on implied trust;
- Pricing based on the power of the company in the value chain;
- Security often being wrongly equated with privacy;
- Privacy based on the power of the company and their stated T&C;
- Numerous consent handling and enforcing issues;
- While regulated companies are over governed, other companies run away with data creating monopolies;
- Misaligned powers of buyer and sellers lead to ‘winner-takes-all’ scenarios.
Moreover, such misalignment of incentives made for unsustainable centralised marketplaces and silos.
When we held a major workshop to test our central data marketplace theories, we realised that we needed to pivot to trust frameworks which shifted regulation and compliance towards technology — trust built into the technology with a verifiable audit on data use.
We understood that we could not take away control from data providers and establish a winner-takes-all data marketplace.
We needed an open data marketplace where the data exchange protocol was open for all to adopt; and friction on pricing, consent, compliance etc. had been removed so that the ecosystem could focus on building the future.
My background is in distributed systems and computer networks and, given decentralisation of communication with TCP/IP, I kept looking at distributed (and decentralisation) protocols for transactional databases.
Blockchain took our sector by the storm and naturally I was drawn to it. I immediately thought that this technology had very interesting answers for the issues encountered by centralised databases and networks in general.
Having experience in building an AI-based product company and deep connections to the AI and Machine Learning world, I could see the need for building a new data marketplace that integrated the use of Blockchain technologies.
Back in mid 2016, I met Trent from BigchainDB and I kept in touch with their developments in blockchain and scalable databases. I found in Trent’s blogs a reflection of exactly what I was thinking as viable ideas for future developments in big data and AI. Undoubtedly, BigchainDB had the most promising solution and DEX was pivoting towards a decentralised marketplace with trust frameworks with government and industry data and oversight.
It is at this point that my team and I decided to build the perfect partnership — DEX and BigchainDB — to create an open source reference implementation of a decentralised marketplace with a blockchain based, data exchange protocol powering it — Ocean Protocol, a protocol that will power data marketplaces in a sustainable and scalable way.
In preparation for our next phase, we are bringing down our present DEX marketplace and are working closely with the Singapore public and private sectors to build sandboxes for different sectors to build the protocol and marketplace.
To ensure that the right DNA for the marketplace is set, DEX is working with the government who will provide the oversight to make Ocean Protocol the data exchange protocol for all.
We will open source all possible elements of the marketplace and protocol as we continue to build.