This dwarfstar looks interesting, can you elaborate on your setup and what kind of inference speeds you are getting?

With the advent of AI bots trying to flood into Lemmy communities, I don’t really see this as a viable option on the long run.