• 0 Posts
  • 16 Comments
Joined 3 years ago
cake
Cake day: June 12th, 2023

help-circle
  • atzanteol@sh.itjust.workstoSelfhosted@lemmy.worldDo you host your own AI?
    link
    fedilink
    English
    arrow-up
    3
    arrow-down
    2
    ·
    21 hours ago

    I’ll check that out - speed isn’t my biggest issue so much as coding performance… The qwen 3.5 model I was using can write code, but it’s… Meh? Like sometimes it doesn’t even compile.

    I did try tweaking llama.cpp to do some cpu offloading and it does seem to allow for much larger contexts at a modest performance loss. I’ll check out larger models.








  • I was curious what distro you folks might recommend for this purpose.

    This is a bit like going to an automotive forum and asking “what’s the best car to buy”. You’re going to get a lot of “I’m running <blank>” and people telling you their preferences, which is NOT the answer to your question. The answer to your question is that literally any of them would be fine for your purposes. If you’re happy with Bazzite then stick with Bazzite. There’s no reason to switch.

    If I have to manage it entirely by command line, it will take 10 times longer for me to do anything I want to do, and I’d really prefer a GUI.

    Then use a GUI. The extra memory used is trivial and your system will be way over-powered for a reverse proxy to a home network anyway. In Linux land there’s really no such thing as a “server distro” and a “desktop distro” for the most part. I use Ubuntu, Debian and Fedora as servers. They can all have desktops on them too.

    You may find, however, that as you manage more than one system it becomes tiresome/tedious to have to use RDP for remote administration and may start learning the CLI over time. Especially since it’s often a lot easier to give somebody a list of commands to run on a forum than to say “open your network manager, which is different on Gnome from KDE, click the button that says…”.

    I need something that can sit there without updating until I tell it to

    Are you going to update frequently? You want to be sure you’re keeping security patches up-to-date. Auto-patching can be very good unless you have the discipline to keep up with it.

    I need a domain for that, and a lot of tutorials just skip on past this step in the domain configuration screens where you “enter your DNS servers” as though I know why I’d need other DNS servers,

    You’ve got a bit of reading on how DNS works. But basically there are “root DNS servers” that everybody knows by IP address that then know about other DNS servers by IP and forward traffic to them to resolve names. When you register a domain you are asking one of those DNS providers to resolve your hostname to your IP address. You can see this a bit by running dig +trace some.host.name and it will show the requests made. Your DNS servers would be the ones where you register your domain.

    BUT your IP address may change. So you generally need a way to update it if it does. There are providers like dyndns.org and others (search for dynamic domain service or something) that will give you a sub-domain for free/cheap and tools to auto-update it. Something like “mysite.dyndns.org”.


  • You know - I hear your frustration. We’ve spent a lot of our lives learning and perfecting a craft that can now be done by anybody willing to pay an LLM to do it for them. I get it. It’s a it disheartening to see your hard-earned skill commoditized like that. But it’s not the only skill we’ve learned. The technical knowledge and experience we built up over that time is invaluable as well and can’t be replaced with an LLM. Calculators didn’t stop mathematicians from existing either.

    But the world is different now. This genie is not going back in the bottle no matter how much we whine or say “but AI still makes mistakes!” Lashing out at those who use AI as either “not real programmers” or for “vibe coding” isn’t going to help either. Nobody said you had to debug this dudes code. You bitched of your own free will. You didn’t help, you were just mean.