Apple发布新版MLX,Mac Studio可分布式运行近万亿参数模型
Local and Distributed Large LLM Inference with MLX and RDMA on Mac Studio
Apple发布新版MLX,借助M5 Ultra的统一内存与雷雳5 RDMA,可在单台或四台Mac Studio上本地运行Muse Spark Open等前沿模型;四机张量并行下解码速度超36 tokens/秒,首字延迟较上代最高快4.6倍。
AApple Machine Learning Research阅读原文