all the quadratic communication and caching growth it requires.
I have trouble visualizing and understanding how the Internet works at scale, but can generally grasp how page-by-page or resource-by-resource requests work. I struggle to understand how one could efficiently parse the firehose of activity coming from every user on every instance that your own users follow, at least in user-focused services like Mastodon (or Twitter or Bluesky). With Lemmy, there will be many more people following the biggest communities with the most activity, so caching naturally scales. But with Twitter-like follows of individual accounts, there are going to be a lot of accounts on the long tail, with lots of different accounts being followed only by a few people. The most efficient method is to just ignore the small accounts, but obviously that ends up affecting a large number of accounts. But on the other hand, keeping up with the many small accounts will end up occupying all the resources on stuff very few people want to see.
A centralized service has to struggle with this as well, but might have better control over caching and other on-demand retrieval of content in lower demand, without inadvertently DDoSing someone else’s server.
Loops really isn’t ready for primetime. It’s too new and unpolished, and will need a bit more time.
I wonder if peertube can scale. YouTube has a whole sophisticated system for ingesting and transcoding videos into dozens of formats, with tradeoffs being made on computational complexity versus file size/bandwidth, which requires some projection on which videos will be downloaded the most times in the future (and by which types of clients, with support for which codecs, etc.). Doing this can require a lot of networking/computing/memory/storage resources, and I wonder if the software can scale.