Recently, Apple introduced eight open source language models known as OpenELM (Open-source Efficient Language Models). These models are unique because they operate directly on the device, bypassing cloud servers. In this brief guide, we’ll demonstrate how to run and use them.
While I’m a fan of Ollama, which facilitates running most open-source LLMs locally with ease, OpenELM is not yet supported. However, a resolution for this issue is currently being addressed in a issue.
Apple’s Efficient Language Models Overview
Apple has released a new series of large language models known as OpenELM, available for download and implementation through the Hugging Face Hub. Half of these models are enhanced by Apple’s CoreNet library, a newly introduced resource aimed at optimizing the training of deep neural networks.
The remaining four, known as Instruct models, are engineered to serve as educational tools, capable of understanding and executing specific instructions. The entire collection of models is supported by a robust set of training and evaluation frameworks, all accessible on public datasets. This includes comprehensive training guides, various checkpoints, and a range of pre-training setups tailored to diverse needs.
Explore the full capabilities of the OpenELM family. Click here for more detailed information on each model.
- OpenELM-270M
- OpenELM-450M
- OpenELM-1.1B
- OpenELM-3B
- OpenELM-270M-Instruct
- OpenELM-450M-Instruct
- OpenELM-1.1B-Instruct
- OpenELM-3B-Instruct
Running OpenELM via HuggingFace
Install
To help you get started, a sample function is provided in all 4 models that you can grab with wget.
wget https://huggingface.co/apple/OpenELM-270M/resolve/main/generate_openelm.py\?download\=true -O generate_openelm.pyThe script generate_openelm.py produces output from OpenELM models through the Hugging Face Hub.
Ensure that your Python environment contains both Torch and Transformers.
pip3 install transformers torchTo test the OpenELM-270M model, simply run the following command:
python3 generate_openelm.py --model apple/OpenELM-270M
--hf_access_token [ACCESS_TOKEN]
--prompt 'What is a meme, and what is the history behind this word?'
--generate_kwargs repetition_penalty=1.2Refer to this link to obtain your hugging face access token.
You will need to request access for meta-llama/Llama-2-7b-hf since Llama-it’s used as Tokenization checkpoint, otherwise you will get error:
Cannot access gated repo for url https://huggingface.co/meta-llama/Llama-2-7b-hf/resolve/main/config.json.
Access to model meta-llama/Llama-2-7b-hf is restricted and you are not in the authorized list. Visit https://huggingface.co/meta-llama/Llama-2-7b-hf to ask for access.Results:
==========================================================================================================================
Prompt + Generated Output
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
What is a meme, and what is the history behind this word?
Meme is a term that refers to an image or video that has been shared widely on social media. It can be used in many different ways, but it's most commonly associated with memes. Meme is a type of image that is shared by users who are not necessarily aware of the origin of the image. The images are often posted by people who have no idea what they are posting.
The first use of the word meme was in 1984 when the term was coined by the comedian Bill Cosby. He created a meme about his wife, which he called "Bill Cosby's Wife." This meme became so popular that it was used as a catch-all term for all types of memes.
In 1995, the term meme was coined by the comedian Jon Stewart. Stewart created a meme about his wife, which he called "Jon Stewart's Wife." This meme became so popular that it was used as a catch-all term for all types of memes.
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Generation took 4.76 seconds.Thank you for reading until the end. Before you go:
If you’re looking to start a new React Native / Expo project, check out my expo starter kit. It will help you bootstrap your project quickly and efficiently :
https://expostarter.com
