The Local Alternative
Transcript:
A comic in four panels:
Panel 1. Cepper, the Gothic Sorceress, sits at her workbench in the basement of the university with her iconic clothes and glasses on her head. She is surrounded by steampunk cogs, wires, circuits, and code snippets written on parchments. She's determined in front of her masterpiece, her own local AI Parrot looking like a big pigeon.
Cepper: "I did it! Running 100% locally now. My own machine, my own terms! hehe."
Panel 2. She excitedly asks it a question, but it takes an eternity to respond.
Cepper: "Avian Intelligence, what's the airspeed velocity of an unladen swallow?"
Local AI Parrot: "1... 1... m... e... t... e... r... s... loading 2%"
Panel 3. Cepper starts to realize the immense computational power required to run AI models remotely. She looks at her local AI Parrot and starts to wonder.
Cepper: "Ouch. That's painfully slow, even with the largest magical stone I had!"
Local AI Parrot: "p...e...r... s...e...c...o...n...d... loading 4%"
Panel 4. A shot late at night, she sleeps deeply on a big armchair, while the local AI Parrot still finish to output.
Local AI Parrot: "a...n...d... t...h...a...t...s... a...l...l... loading 100%"
sources: https://www.peppercarrot.com/en/miniFantasyTheater/045.html
197 - 
328
35 comments
ntoyohito@comicscamp.club
Oh man... I feel this SO SO much...
★rez@social.ultimaterez.com
🤣 Are you reading my mind? I JUST got a LocalAI container running and I'm trying to figure out how to make it significantly less slow.
★ozzelot@mstdn.social
And still unable to be decoupled from the fact that it was created by The Cabal Of The Evilest Wizards You Can Imagine, No, Eviler Than That
★csolisr@hub.azkware.net
About the only project that cares about divesting is this one: huggingface.co/jadael/comma-v0…
leberschnitzel@existiert.ch
she should try an MoE (magic of elements) model, they work pretty nice consumer stones
LappenjammerDieZweite@vivaldi.net
Is it an African or an European swallow?
LucKeyProductions@c.im
@LappenjammerDieZweite 11 m/s? Must be an American swallow that just stopped at a Burger King.
TopazRabbit@oldbytes.space
🦜 A...f...r...i...c...a...n... ...o...r... ...E...u...r...o...p...e...a...n...?
13 ★davidrevoy
@TopazRabbit 🤣 👍
3 ★condret@fedi.absturztau.be
the trick is to ask it to do something and then work on something else, while it is busy
Takiro@meow.social
Also its cooling is running like crazy while it's almost overheating :D
Minski@kinkycats.org
🥥 🥥
★susannelilith@rheinneckar.social
IE as a bird. 🐌
davidrevoy
@susannelilith 🤣 Poor IE, long time dead, and still receiving nukes. (but deserved 🤭 ).
3 ★Arioch
https://github.com/ggml-org/llama.cpp check this with the vulkan backend :blobwhistle:
2 ★s8n@posting.lolicon.rocks
that's not at all accurate
cryptax@mastodon.social
excellent !
gr4yweb@chaos.social
I feel that, running fast-whisper and speech-to-phrase for HomeAssistant Voice locally. Superb Comic! 👌
Zeugs@social.cologne
Tuboquant may fix that problem
alxndr@tech.lgbt
Mom, is being mean to AI bros again! Mooom!!!
davidrevoy
@alxndr 😆 💜
3 ★betelgeuse93@mastodon.uno
like my rpi5 with a 3B model 🤣🤣🤣
nom@mk.spook.social
@davidrevoy@framapiaf.org 4060TI => biggest AI bang for the buck. An'Apple seems to be mirroring its function+capability going forward. #GoodEnuf
VANTABlack2000@mastodon.gamedev.place
Great Comic as usual. It's fun to see the story progress every week. Can't wait to see next week's where Pepper still isn't done./sarcasm
andersgo@oslo.town
Self hosting is the way!
jimmac@mastodon.social
And that is exactly the big lie. The magic of brute force token jumping your way to a patch only works if you have gigawatts of Nvidia filled data center powered by methane jet engines. And nobody knows the actual cost of any of this. You try it locally and it's just making your m4 laptop, that normally churns Blender Cycles renders in seconds, crazy hot to touch and takes hours instead of seconds. It seems way less efficient than ... blockchain.
davidrevoy
@jimmac True, my local tests here were mostly with https://flathub.org/en/apps/com.jeffser.Alpaca (advantage of being easily removed and leave the computer clean after test). I tried few models available, but even on my workstation, I saw it was slow. Very educational to imagine how screaming some distant CPU and GPU must be on the servers of the AI compagnies, and how they make it totally invisible to the end users who are asking questions on their phone 'for the fun', not even questioning the (hidden) cost of it...
10 ★okr@sunny.garden
@jimmac that's the huge point Ed Zitron has been making for some time now: if you expose the true cost of this product to the users and bill them for every stupid prompt, oopsie and hallucination, nobody will want to pay for it.
okr@sunny.garden
@jimmac and it's so obvious and anyone with a PC can test it locally like you did and see for themself, and yet nobody cares, it's so frustrating
ElectroFetish@mastodon.social
Kiwix + Hotspot = profit and speed
On any device, even the weakest one
ariarhythmic@ohai.social
did she become so dependent on it so as to go through all that trouble? why would you want an alternative other than nothing?
★orsinium@fosstodon.org
Monty Python ❤️
★milagemayvary@mstdn.social
https://mstdn.social/@milagemayvary/116291424742312318
davidrevoy
@milagemayvary Thank you for the link! I'll watch it!
gimulnautti@mastodon.green
She needs to discover ternary magic 🤔
Post a reply
The comments are synchronised every 1h with the replies to this post on Mastodon:How to use this? (click here to unfold)
Open a new Mastodon account on the server of your choice. Then, Copy/Paste the adress above in your Mastodon 'Search' field. The post will appear and you'll be able to fully interact with it. You'll have full control of your posts: edit, remove, etc. After that, your message will appear here.
Just please note that it may take up to 1 hours for your changes to be reflected here.