/
07.01.2025 at 11:35 pm
Cuttings

Mac Studio M1 Max Meets Qwen-32B

2.5 year-old machine runs the LLM well, save for context size limits.

In July 2022, I bought an Apple M1 Max Mac Studio - the base model.

(Brief specs: 10-core CPU, 24-core GPU, 1 core Neural Engine, 32GB RAM.)

Today, 2.5 years after, I've only just discovered how well the M1 Max handles LLMs, particularly the Qwen 32B Q4_K_M model!


A demonstration of performance - via keyword extraction and glossing:

Not bad! In comparison I've a separate 64GB M4 Max Macbook Pro, and a roaring RTX 4090 - but this is definitely a respectable speed.

The only problem: context size. If I hit up all the layers, my Studio can no longer fit a Qwen-32B with a 12K context size. (Surprisingly the model does load, but outputs garbage, and 'out of memory' errors. Which is not the Studio's fault, really - I chose the base model, after all.)

For now, I've set the context size to 8K - which is good enough for general legal/drafting/writing, glossaries, ideas, prompts, and medium-sized texts.


I use my Mac Studio mainly for processing work, and it's a fantastic Swiss knife to throw things at when my main machines are laden with heavier things (image generation, audio processing, etc).

So I'm happy to have discovered even more daily uses for my now 2.5 year old machine - now as an LLM host!

Filed under:
#
Words: 247 words approx.
Time to read: 0.99 mins (at 250 wpm)
Keywords:
, , , , , , , , ,

Other suggested posts

  1. 01.04.2023 at 11:27 am / Search Notes Fast: With Taskwarrior, jq & ripgrep
  2. 18.10.2022 at 09:36 am / The Airplane Test of Fluency
  3. 27.06.2021 at 12:49 pm / Eloquent Javascript / Chapter 2, Exercise 03
  4. 02.01.2018 at 12:00 am / Genuinely Technical Modesty
  5. 09.07.2016 at 12:00 am / The Lawgiving Coder
  6. 03.01.2016 at 12:00 am / Creators And Their Gems (Polished or Otherwise)
  7. 14.06.2015 at 12:00 am / Just the Two of Us (6-Bar Guitar/Sax Copy)
  8. 16.02.2014 at 12:00 am / Don't Care Too Much About Gear
  9. 10.12.2013 at 12:00 am / Yellow Rails
  10. 08.12.2013 at 12:00 am / Alien Blue Entrances
© Wan Zafran. See disclaimer.