How to start development with LLM?

Starting in any hot field may feel overwhelming. The abundance of programs, libraries, and courses promising to make you a professional in no time exacerbates the problem. And with all the hype, the LLM field is as hot as a neutron star.

Let me help my fellow developers navigate it and describe a few good starter tools: LM Studio, Ollama, Open WebUI, and Oobabooga.

LM Studio — Easy Start

I found LM Studio to be the most straightforward option for local development. Once installed, the application navigates you through the process well. The button to download the model is front and center (it is not so apparent with other tools). The UI provides a good default option (LLama3); you can start chatting immediately once the model is downloaded.

There is a handy model discovery tab to find all the latest and greatest models. The app is helpful at every stage, offering tips and valuable information in the UI.

However, the project falls short when it comes to development features. LM Studio has an integrated API server, but the lack of an LM Studio Docker image makes it very difficult to run an app in the cloud, which you’ll likely need to share your LLM application with the end user.

The LM Studio developer community is also smaller than the Ollama described next, so finding support or a library with a necessary feature might take more work.

Ollama — Developer Heaven

Ollama is an LLM command-line application and API server. It can be installed locally on Linux, Windows, Mac, or Docker. Once installed, run the following command in your terminal to start the chat session.

ollama run llama3.1

The API server is up and running immediately, so you can make cURL requests as described here. Here is an example:

curl http://localhost:11434/api/chat -d ‘{
 “model”: “llama3.1”,
 “messages”: [
   {
     “role”: “user”,
     “content”: “why is the sky blue?”
   }
 ]
}’

Ollama shines in its developer features and community. Here are links to the ones that you most likely will need:

• Docker images hide the complexities of configuring the system to run accelerated inference using Nvidia, AMD, and CPU hardware.
• Rest API, Python, and JS clients simplify application development.

Open WebUI — UI for Ollama

The Open WebUI project provides a handy GUI interface for debugging, tweaking, and exploring LLMs that work with the Ollama API server.

The Open WebUI setup is too complex for this article, so here is a detailed tutorial on running Ollama with Open WebUI in Docker.

The Open WebUI is less polished and feature-rich than LM Studio. However, it will cover you if you want to stay within the Ollama ecosystem.

At the time of writing, the Open WebUI didn’t have an integrated model library explorer, but there is a model library on the Ollama website.

Oobabooga — Swiss Army Knife of LLM

Oobabooga is the most difficult one to get started with. The instructions are spotty, and the interface is not intuitive. Here is a comprehensive tutorial on how to launch it locally.

However, Oobabooga is the most flexible of the three. The UI allows for the configuration of virtually all model and inference parameters.

Additionally, Oobabooga can download text-generation models directly from HuggingFace, while the tools above only work with models from their library by default. So, if you want to try a model unavailable in LM Studio or the Ollama model library, it might be easier to start with Ooobabooga.

It is generally hard to advise beginners on Oobabooga, but keeping it in your GenAI toolbox is vital.

Conclusion

As usual, there is no one-size-fits-all solution. Each application has its strengths and weaknesses. Luckily, you can use them together and combine the benefits:

• Start with LM Studio and play with LLM on your laptop or PC to familiarize yourself with basic concepts, API structure, and popular models.
• Use Ollama LLM server, API, and client libraries for application development.
• Use Oobabooga when you need to tweak LLM parameters and get the most out of your model.

If you enjoyed this article and want to support the team, please join our Discord and check out our GPU rental service — neuralrack.ai.

GPU Rental Pricing | 85% Less for Datacenter-Hosted GPUs

Thanks for reading!

How to start development with LLM? was originally published in Everyday AI on Medium, where people are continuing the conversation by highlighting and responding to this story.