4GB VRAM

Mmmmm… I would wait a few days, and try a GGUF quantization of Kimi Linear once its better supported: https://huggingface.co/moonshotai/Kimi-Linear-48B-A3B-Instruct

Otherwise you can mess with Qwen 3 VL now, in the native llama.cpp UI. But be aware that Qwen is pretty sycophantic like ChatGPT: https://huggingface.co/unsloth/Qwen3-VL-30B-A3B-Instruct-GGUF/blob/main/Qwen3-VL-30B-A3B-Instruct-UD-Q4_K_XL.gguf

If you’re interested, I can work out an optimal launch command. But to be blunt, with that setup, you’re kinda better off using free LLM APIs with a local chat UI.

Really raises the question of how profitable these scam ads have become.

It does, doesn’t it?

If Meta is literally making billions from ads, how much are the scammers buying them making?

People keep coming back to these obviously booby-trapped websites to claw at a thin veneer of simulated friendship because they’ve got nothing better to do with their lives.

Eh, part of it is FOMO. I can’t blame older folks if all their friends and kids and grandkids are on (say) Facebook, and that’s the most advanced means they have of communicating with them that they’re capable of using. What are they supposed to do?

Yeah, exactly… In other words, ‘general server buildout.’

That’s (as I understand it) a misconception.

Apple attaches their laptop RAM the same way all smartphones do. It’s a wide bus with LPDDR, which makes it an unusual configuration amongst laptops, but it’s technically conventional. And relatively cheap.

AMD’s Strix Halo chips are the same. Apple could use LPCAMM to make the memory upgradable if they wanted, they just… don’t.

When we talk ‘packaging’, we’re talking putting chips on advanced substrates with denser wires than one could possibly get on a motherboard (or a ‘mini’ motherboard which is kinda what Apple/smartphone RAM is packaged on), stuff silicon fabs have to do:

https://www.tsmc.com/english/dedicatedFoundry/services/advanced-packaging

And HBM falls into this bucket. The way its hooked up to the processor is physically different than PC RAM sticks, or Apple’s RAM. This is mostly not done on consumer stuff because its very expensive, and most of TSMC’s advanced packaging production capacity is reserved for server stuff.

The key is which model, and how.

For the really sparse MoEs, you might be better off trying ik_llama.cpp, especially if you are targeting a ‘small’ quant. But the dense Gemma models (as good as they are) are probably not the best choice for 8G RAM these days.

Ah. Yeah. And its on the fab to do that.

I always though it’d be cool for CPUs to switch to packaged RAM, too. Samsung apparently tried to do it with Wide I/O for mobile ARM stuff, but it never caught on.

Plenty of folks do AMD. A popular homelabsetup is 32GB AMD MI50 GPUs, which are quite cheap on eBay. Even Intel is fine these days!

But what’s your setup, precisely? CPU, RAM, and GPU.

…Is it that interchangeable?

TBH I know little of memory fabs and HBM ICs, but I know (say) TSMC can’t just switch from a power-optimized process to a high frequency one at the drop of a hat.

They can ALL be run on RAM, theoretically. I bought 128GB so I can run GLM 4.5 with the experts offloaded to CPU, with a custom trellis/K quant mix; but this is a ‘personal use’ tinkerer setup basically no one but hobbyists will touch.

Qwen Next is good at that because its very low active parameter.

…But they aren’t actually deployed that way. They’re basically always deployed on cloud GPU boxes that serve dozens/hundreds of people at once, in parallel.

AFAIK the only major model actually developed for CPU inference is one of the esoteric Gemma releases, aimed at mobile. And the bitnet experiments, which aren’t very big so far.

(In case it’s not obvious, this is my special interest, and I’m happy to ramble on about how to set up ‘niche gaming rig hybrid models’ for anyone interested).

I just got a 2x64GB 6000 kit before its price skyrocketed by like $130. I saw other kits going up, but had no clue I timed it so well.

…Also, why does “AI” need so much CPU RAM?

In actual server deployments, pretty much all inference work is done in VRAM (read: HBM/GDDR); they could get by with almost no system RAM. And honestly most businesses are too dumb to train anything that extensively. ASICs that would use, say, LPDDR are super rare, and stuff like Hybrid/IGP inference is the realm of a few random folks with homelabs… Like me.

I think ‘AI’ might be an overly broad term for general server buildout.

Reddit and much of the internet was like that a long time ago.

I like to think it came from the higher barrier to entry, kinda like Lemmy has now. One doesn’t just accidentally use Lemmy because it was recommended on their App Store feed. You have to seek it out.

PS

One issue I hadn’t thought of is putting traditional brakes (which generate a ton of heat) right next to the motors. Again, we’re just asking for mechanical issues here, and we’re ballooning unsprung mass to mitigate it, especially in heavier cars that take a lot to stop.

The entire floor pan could just be one thin battery, and everything above it could be passenger and storage space.

This seems like a minor thing, but the control electronics for the motors takes up a nontrivial amount of space. So do “traditional” subsystems like hydraulics, climate control, or an old fashioned car battery (which often exists in parallel to the EV drivetrain).

Theres also safety to consider. A traditional sedan “hood,” even a small one, is easier on standing pedestrians, so it hits their legs and they flop on top, instead of slamming them like a wall (as a bus-like front would).

Renault 5 RS Turbo

That’s because its a limited run show car. Its not meant to be practical.

That’s how EVs started! Sorta.

This is from a Porsche in 1900:

And some 2000s EVs tried it. But it’s impractical.

• It increases unsprung weight, e.g. weight not cushioned by suspension. Bad for ride/handling/steering feel.

• All that vibration is HARD on the motor. Read: unreliable.

• Motor is more exposed to temperature/dust. Again, reliability.

In reality, a decent suspension needs a lot of room under the body anyway. An axle to get the motor in the body is dirt cheap on the rear, and still pretty cheap on the front, and you could just mount this thing sideways to make it flat…

Yeah. You are preaching to the choir here.

…Still though, I just meant there’s no reason to use Gemma 3 27B (or 12? Whatever they used) unaugmented in AI Studio. The smallest flash seems to be more optimal for TPUs (hence it runs faster).

It’s possible a member of Blackburn’s staff or a supporter went looking for a libelous hallucination in Google’s models.

Good to see Ars with some common sense here.

FYI Gemma3 is Google’s open weights release, for local running and finetuning. It’s pretty neat (especially the QAT version), but also old and small; there’s no reason anyone would pick it over Gemini 2.5 in Google’s dev web app, except for esoteric dev testing. It’s not fast, it doesn’t know much, it’s not great with tooling (like web referencing), its literal purpose is squeezing onto desktop PCs or cheap GPUs.

…Hence this basically impacts no-one.

The worst risk is that Google may flinch and neuter future Gemma/Gemini over this, lest some other MAGA screams bloody murder over nothing.

You’re acting like the ‘internet’ (mostly Facebook, Google, Discord, a few apps, insular siloes) isn’t utterly dominating people’s lives. Or that its original purpose (portals for institutions like universities or militaries) hasn’t been smashed to smithereens. Go out on the street, and watch how regular folks access information and spend their time.

It’s not fine. It’s mostly an attention optimized monster designed to suck and lock people in, and siphon them away from actual portals.

Never has radio or TV or anything intruded in so many people’s psyche and taken over other institutions like this. If the US military knew any better, they’d see it for the national security threat it is, though I’m sure some sects already do.

Just jumped from Android to iOS, and I maintain Android phones for family, seen lots of both since the iPhone 4.

My advice…

Cheap out.

Either keep your phone or get an iPhone on firesale. There is no point to getting a Pro.

Android freaking sucks now, as you can either get a custom ROM and break finance apps, or bog your phone down (and let it go obsolete, quickly) with the stock operating system and rampant spyware via every app you install. You can’t even uninstall bloatware anymore, like you used to. And it will get worse once sideloading is gone.

And flagship Android phones are awful. They either cost a fortune unlocked and ‘mostly stock,’ or they run absolutely horrid, spammy, battery sucking UIs (looking at you, Samsung).

…Apple pro phones are no different. I just went on a whole vacation with iPhone 16 Plus and 17 Pros side by side, and its just not worth spending more on anymore. The UI has regressed some. But at least Apple reigns in/restrics apps you install and keeps the experience pretty clean.

If you want great pictures, grab a nice point and shoot with optical zoom and sensor stabilization. They’re incredible now. In fact, on a recent trip, I discovered my 2008 Canon takes better zoom photos than brand new iPhone 17 Pros.

EDIT:

If you must get an Android phone, get one with as close-to-stock a UI as possible.

I do not game on phones, but my best experiences have, ironically, been with ‘gaming’ phones like the Razer Phone 2 and Asus phones. They have gigantic batteries, lots of RAM, and lean, stock UIs that let you disable/uninstall apps, hence they’re fast as heck and last forever. I only gave up my Razer Phone 2 because the mic got clogged up with dust, and I miss it.

Maybe there’s a crossover point where it becomes fantasy?

I’m playing devil’s advocate here. But I do feel like hyper-reality and animation and robots could be easier to psychologically separate from real like than ‘real’ film or camgirls or whatever. Especially if the curtain is pulled back, and all their knobs are exposed.

At… low points in my life, I’ve used locally run LLMs as sounding boards in lieu of family or whatever, so this is where I’m coming from. Even mentally compromised, all the technical setup/troubleshooting and knobs makes it obvious I’m talking to a tool, not a grounded person. I feel a lot of AI would be healthier if presented that way, including the inevitable pornbots, instead of as the living oracle black boxes Tech Bros (and their apps) like to paint them as.

Also “minor” compression from high quality material isn’t so bad, especially if starting with a pre trained model. A light denoising step will mix it into nothing.