BOINC, Gridcoin and scaling dynamically beyond local nodes

Providing a way to run workloads too large to run on a single RAIN computer has been part of the roadmap since the beginning. As work on Mark II draws to a close I’m starting to think about the work that will go into Mark III and one portion of that work is support for dynamic scalability. As much as I enjoy designing things from scratch, I do make an effort to make sure there isn’t an existing solution that fits the problem I’m trying to solve (discarding my work on ganging-together entire PINE64 A64 SBC’s when the Clusterboard became available is an example of this). So when I started looking into ways to create a dynamic, distributed network of RAIN machines I looked at existing grid computing systems to see if I could avoid starting from scratch. As it turns out, there is one clear choice in off-the-shelf, open-source grid computing systems and that system is BOINC. I’ve known about BOINC for a long time but I didn’t know it by that name. I knew it as SETI, which is the specific project from which the general-purpose BOINC system emerged. Today BOINC is used for a wide-range of scientific computing projects and I was very pleased to see that it’s an active, well-used project. I spent a little time learning about how BOINC works and it looks like it could be used as-is with RAIN the same way it can be used with almost any computer. The challenge isn’t in making RAIN support BOINC so much as it is in making the user’s application compatible with BOINC. Once that’s achieved, the application could be run on a single RAIN machine across the internal cluster, a distributed grid of RAIN machines and of course any other computer participating in the larger BOINC network. This is pretty cool, but there’s not much for me to improve on here so to me this is akin to other established parallel computing tooling like MPI; supported by RAIN but not really anything I need to contribute to. Something else I discovered while researching BOINC is Gridcoin. The idea behind Gridcoin is similar to any other cryptocurrency (such as Bitcoin), but instead of securing the network by executing pointless busywork, the network is secured by executing distributed applications in the BOINC network. At first this seemed a lot like the “global network” I describe in “Why Personal Supercomputers” but as I looked closer it doesn’t provide all of the features I’m looking for. What’s missing is the ability to use Gridcoin to secure future processing resources on the grid. Instead, Gridcoin is intended to be a general-purpose currency just other cryptocurrencies. What I’m looking for is a way to encourage RAIN operators to lend their unused compute cycles to others, and encourage them to do this by providing a way to tap-into other’s unused cycles when they are needed in the future. This allows users who are acquiring a RAIN machine to immediately run an application access to computing resources beyond their own machine, while incentivizing others who are interested in the technology but don’t have a specific application in mind to get value out of the system until they are ready to utilize it. In my mind this took the form of a “currency”, but it doesn’t have to be, and I don’t have any strong reason to create a currency vs. an accounting system that ensures users will have access to an amount of processing power equivalent to the power they have contributed to the grid. Based on this it looks like I can learn some things from BOINC and Gridcoin, but unfortunately neither is a perfect fit for my design. Regardless I’ve learned a lot from studying both, and I’m very excited about the fact that I can contribute currently unused capacity of the Mark II prototype to ongoing scientific work with little effort.