ESP32S3 cluster running 1.58-bit (BitNet) Language model

(github.com)

66 points | by nkko 8 hours ago

6 comments

  • tdhz77 3 hours ago
    Soon ai in every lightbulb running Kubernetes
    • tombert 1 hour ago
      You know, I've always liked Futurama but I always kind of thought it was silly that literally everything has an AI and a personality.

      But, you know, I actually think that there might be a logic to it. Economies of scale might mean that almost-literally every computer you buy in the year 3000 has some kind of AI-assistance chip in there, and sure maybe it will have full AI with a personality spitting out one-liners.

      • abroadwin 34 minutes ago
        Kind of like how disposable vape pens often have a 24 MHz Cortex-M0+ with 3 kB SRAM and 24 kB flash, which would have seemed ludicrous a while back.
  • NDlurker 2 hours ago
    I'm curious how this would handle grammar checking on a basic word processor. Or maybe generate worlds for small text based games. I have no idea what the capabilities are of a cluster like this.
  • nonasking_ 1 hour ago
    Thanks for sharing. It's fascinating to see a 0.5B LLM being split across seven ESP32s like this.
  • cameron_b 5 hours ago
    It is a bit of a bummer to see that the degree of 'compression' makes it a fancy llm noise-maker. It is still charming.
  • matthewfcarlson 2 hours ago
    I’m actually working on a small project that’s exactly this! Less quant so it’s only 150M parameters but this is amazing.
  • sjakati98 4 hours ago
    Gemma 4 when?
    • nkozyra 1 hour ago
      We're gonna need a bigger ESP.