Does InterPlanetary File System (IPFS) hold the keys to solving full application hosting at the Edge?
One of our latest Section Labs initiatives has us exploring IPFS as a potential solution to solve some of the inherent challenges with distributed databases across hundreds of points of presence, from multi-cloud to regional data centers to edge.
To deepen our understanding of the problem space and how a practical implementation might work, we’re actively developing a proof of concept using OrbitDB, an open source serverless, distributed, peer-to-peer database.
The Challenge
Up until recently, most edge computing use cases have been stateless. However, as developers seek to push more advanced logic to the edge, the requirement for handling stateful data is inevitable and on our doorstep.
Over the past year, we’ve been exploring established solutions to solve the distributed database challenge, including conventional distributed databases and geo-distributed databases. Although encompassing various elements of a ‘distributed’ architecture, our findings to date reveal that current models still depend on some form of a centralized state management system. Furthermore, our experiments with a handful of existing market solutions have shown them to fall over when it comes to handling the scale of persistent data across 30+ endpoints (let alone the hundreds, or thousands that are expected as the edge computing market matures).
The Solution
After running into challenges with more conventional solution offerings, we opted to come at the problem from a different angle. To this end, we’ve been working up an initial proof of concept that involves deploying an IPFS swarm using our points of presence as peers and enabling distributed Node.js apps to access the IPFS swarm to access their OrbitDB data sets.
What is OrbitDB?
OrbitDB is a serverless, distributed, peer-to-peer database. OrbitDB uses IPFS as its data storage and IPFS Pubsub to automatically sync databases with peers. It’s an eventually consistent database that uses CRDTs for conflict-free database merges making OrbitDB an excellent choice for decentralized apps (dApps), blockchain applications and offline-first web applications.
The architecture of OrbitDB is fairly straightforward. As a peer-to-peer database, each peer has its own instance of a specific database. A database is replicated between the peers automatically, resulting in an eventually consistent data set.
Each supported database (it currently supports five types – log, feed, docs, keyvalue, counter) has its own API, each with its own methods to create, delete, retrieve and update data.
With its simplistic approach, OrbitDB also serves as a good foundational framework for others to build more sophisticated data models on top (e.g. AvionDB).
Our Implementation
For our MVP implementation, we’re exploring how we can deploy OrbitDB across Section’s Composable Edge Cloud. We’re still early in this exploration, but our approach involves running a private IPFS swarm across all of our edge endpoints (i.e. points of presence, PoPs), so that any app running on Section can access the peer network. Since we’re particularly interested in full application hosting on Section, we’ve deployed a distributed Node.js application with OrbitDB running alongside, across a series of edge endpoints.
The Node.js app connects to IPFS daemons running on the edge endpoint where the app is hosted. We use the OrbitDB framework to create the database structure and schema in IPFS storage. For our MVP, we’re testing a basic key/value store, but one of the challenges that we foresee is the current lack of support for relational databases (perhaps a successful basic implementation could lead us down a path to solve for this later… onward!)
By replicating the databases across all edge endpoints, some of the challenges that we’re running up against include:
- Network latency
- Time it takes to populate state (depends on size of data store)
- Memory-heavy
- Security
Many of these challenges have been documented and are being worked on. If interested, you can track them in this Github issue.
Where to from here?
Our product and solutions engineering teams continue to talk with customers around the feasibility of this approach, including implementation of test use cases to validate whether OrbitDB is a viable distributed database solution for the edge.
At the same time, we’re preparing to roll out some production workload to test replacement of our current key/value store used for Virtual Waiting Room with OrbitDB key/value store. (Some might say ‘eating our own dog food’.)
If you’re interested in exploring distributed database solutions, whether with OrbitDB or an alternative approach, we’d love to chat! Please reach out.
