I was thinking about this in regards to all the “defederation” posts.

Let’s say you spin up a server and over night it gets super popular and grows enormous. Now your yearly expenses shoot up and you’re forced to either look for a new host or shut down.

Now what if instead, you could get a few other people to spin up more small instances and distribute parts of your biggest communities to them, however the users don’t notice because The communities are looking across instances instead of within their home instance?

That’s the idea at least. This would allow for many things but most importantly, it would make things a bit more manageable. Thoughts?

  • xavier666@lemm.ee
    4·
    1 year ago

    You are talking about horizontal scaling. It’s something which is non-trivial and would need the core of the app to be written in a very different way. Think of the challenges

    • Your sub-instance and your friend’s sub-instance which serves a common community can be geographically far apart
    • They need to be synced
    • Internal function calls which were once local now need to travel across the network
    • Depending upon the heterogeneity of the hardware, there can be a large jitter (bad for UX)

    The admin of my instance lemm.ee has tried to make horizontal scaling by decoupling some lemmy functions but pure horizontal scaling is really hard (glusterfs/spark).