Mira Murati's Thinking Machines Lab launched Inkling, a 975-billion-parameter open-source AI model, on July 15, 2024, marking a significant entry in the AI landscape.
Overview
- Inkling, an open-source AI model with 975 billion parameters, was officially released by the Thinking Machines Lab on July 15, 2024.
- This marks the first major launch from Mira Murati's lab since she left OpenAI in September 2024.
- Available on OpenRouter, Inkling is priced at $1 per million input tokens and $4.05 per million output tokens, although it competes with models that offer superior performance at comparable or lower costs.
After two years of development following her tenure at OpenAI, Mira Murati introduced Inkling to the public last week. This release is being hailed as the first substantial open-source model created from scratch by a Western lab.
Despite challenges faced by Western developers in the open-source arena, including competition from models like Alibaba's Qwen and Z.ai's GLM, Inkling emerges as a significant contender. Unlike many of its rivals, it has no regional restrictions and is available on Hugging Face under the Apache 2.0 license.
Inkling features a unique mixture-of-experts architecture, utilizing a total of 975 billion parameters, with 41 billion active during inference. The model is capable of processing text, images, and audio and supports a context window of up to one million tokens, having been pretrained on 45 trillion tokens.
Running Inkling locally, however, is not feasible. One of its notable capabilities is performing agentic tasks. According to the Model Context Protocol (MCP) Atlas, Inkling scored 74.1%, significantly surpassing Nvidia's Nemotron 3 Ultra, which scored nearly 30 points lower. Additionally, on SWE-Bench Verified, a benchmark for autonomous bug fixing, Inkling achieved a score of 77.6%, again outperforming Nemotron's score of 70.7%.
Inkling's pricing on OpenRouter is set at $1 per million input tokens and $4.05 per million output tokens, making it easy to integrate into existing setups without extra configuration.
Model Performance Assessment
While benchmarks provide useful insights, practical testing of Inkling revealed a mix of strengths and weaknesses. Initial evaluations began with coding, a key concern for many users. Inkling had difficulty with complex prompts; for instance, a lengthy prompt for creating a shooting game resulted in no output, while a simplified version produced a playable game but with limitations in design and functionality.
In a comparative analysis, a smaller model, Bonsai 27B, generated a more satisfying coding result from the same prompt, raising questions about Inkling's effectiveness despite its vast number of parameters.
In terms of associative creativity, Inkling showed promise by connecting concepts such as a twig and proletariat exploitation, but struggled with maintaining logical coherence. The model's output highlighted a metaphorical connection but lacked precision in its reasoning.
When tested on logical reasoning with a bridge-and-torch puzzle, Inkling initially miscalculated the solution, indicating a need for better engagement with the problem at hand. However, it performed well on simpler logical prompts, demonstrating its potential in this area.
Inkling also exhibited significant censorship, rejecting prompts that sought advice on sensitive topics, which raises questions about its practical applications in real-world scenarios. This approach to censorship is common among AI models, but Inkling's scale complicates typical community solutions.
In a creative writing exercise, Inkling produced a narrative involving time travel, showcasing its research capabilities and descriptive prowess, though it faltered in maintaining logical consistency within the story.
Conclusion
Inkling stands out as a leading open-source model from a Western lab, excelling in certain areas while falling short in others. Its performance in coding tests compared unfavorably to smaller models, prompting considerations about its practical use for developers focused on efficiency and output quality. While it may serve well in compliance-driven environments requiring customizable models, smaller developers might find better alternatives in cheaper, smaller models available on the market. Overall, Murati's lab has made a noteworthy contribution to the field, but the current version of Inkling is more specialized than a general-purpose solution.
