Evan X. Merz

This post is my attempt to share a simple "Hello, World" application using NodeJS and Ollama. It's a simple web API that allows users to chat with an LLM.

How to build a simple LLM chat API using NodeJS and Ollama

I'm going to show you the simplest way to do this, but you DO need to have some understanding of web programming in order to make this happen. I'm assuming that you have nodejs and npm installed on your computer. I'm assuming that you know how to run console commands.

1. Install the Ollama app

If you run your LLM using the Ollama app, then it will automatically run when you boot into Windows. This is convenient because you don't need to remember to start a second process or run a virtual machine.

Here's the link to download and install the Ollama app for Windows.

NOTE: This app requires a restart, which was not indicated in the installation instructions when I installed it.

you will also need to open the Ollama app, go to Settings, and enable "Expose Ollama to the network". This option enables the local Ollama API that will be used by this example.

Finally, you will need to install the model that you intend to use. In the code below, you will see that I'm using llama3.2:1b, the 1 billion parameter open source model released by Meta. I like this model because it runs really fast on my local machine and it gives reasonably good output most of the time for most tasks.

2. Create a new nodejs application and install dependencies

Create a folder for your nodejs application, then run npm init and give the project any details you like. We will use app.js as our entry point, but since we aren't sharing this project, that isn't actually important.

npm init

The only dependencies for this example are express, ollama, and body-parser. Install them with the following command.

npm i express ollama body-parser

Then you need to set up a command to run the project. Modify the scripts section of package.json to tell npm to run the node server.

"scripts": { "dev": "node app.js" },

After writing our server script, we can then use the following command to run the API.

npm run dev

3. Write the code

Type this code into app.js in the root folder.

DON'T COPY PASTE THIS. If you copy paste then you won't learn anything. Type it out line by line.

/**
 * This project is a simple demonstration of how to build a simple
 * node/express API on top of the ollama API.
 */
const express = require('express');
const ollamaLibrary = require('ollama');
const bodyParser = require('body-parser');
const port = 8181;
const DEFAULT_MODEL = "llama3.2:1b";

// instantiate the express server
const app = express();

// tell express to use the body-parser package to extract json from a post
app.use(bodyParser.json());

const ollama = new ollamaLibrary.Ollama({
  url: "http://localhost:11434"
});

/**
 * Set up a single endpoint for chatting with an llm.
 */ 
app.post('/chat', async (req, res) => {
  let params = req.body;
  let prompt = params?.prompt;
  if(prompt != null && prompt.length > 0) {
    let model = params?.model ?? DEFAULT_MODEL;
    const response = await ollama.chat({
      model: model,
      messages: [{ role: 'user', content: prompt }],
      stream: false
    });
    console.log(response.message.content);
    res.status(200);
    res.json({ response: response.message.content });
  } else {
    res.status(400).send("Invalid prompt.");
  }
});

/**
 * Start the server, and listen for requests.
 */ 
app.listen(port, () => {
  console.log(`Server started and listening on port ${port}...`);
});

4. Test using Yaak or Postman

To test that the system is working, you can use an API testing tool like Yaak or Postman. Just send a post request to http://localhost:8181/chat with the following body json.

{
  "prompt": "Why is the sky blue?"
}

It should look something like this in Yaak.

Continuing to build a real LLM API

If you were going to extend this into being a real API, then you would want to add authentication and authorization, CORS support, and more. The hardest part about building an API like this is not necessarily the code, but the infrastructure to support it in production.