Distributed computation prototype
DeluminæLab · Dec 2014 – Sep 2023 · a prototype built around DL-Instant
Inside the same DeluminæLab engagement I prototyped a compute grid around DL-Instant, so a rendering job could be split across machines rather than run on one. It reached a working cluster on Google Cloud Platform, with a network layer added to the engine for remote control.
What I did
This was a prototype inside the DL-Instant engagement, not a separate role. I designed the grid and chose its technology, then built a local cluster to validate the architecture. I evaluated Amazon, Google, Microsoft and OVH as providers before deploying on Google Cloud Platform with Python services built using Flask and SQLAlchemy. I added a network layer to DL-Instant for remote control, implemented TCP and UDP communication paths, and measured cluster performance.
Technical proof
Distributed systems
A compute grid prototyped across a local cluster and Google Cloud Platform, with a network layer added to DL-Instant for remote control.
Architecture
Architecture and technology choices for the grid, and an evaluation of Amazon, Google, Microsoft and OVH as providers.
Python · Flask · SQLAlchemy · Google Cloud Platform
Interested in similar work? Contact