Run your own ChatGPT clone in the cloud

Our lab at Suffolk got a small amount of money from our dean to run our own hosted version of ChatGPT.

The benefit of running your own version is that you can get a simple web interface like ChatGPT, but with more control over what happens to the data that you submit to it, and you can skip a monthly subscription fee in favor of paying as you go. It also gives you the power to run a local model, although that’s rarely cost effective.

After evaluating two popular open source tools, LibreChat and OpenWebUI, we settled on OpenWebUI for the simpler setup experience. It looks like LibreChat is perhaps more powerful, but I ran into enough roadblocks trying to set it up on a cloud server with SSL protection that I switched to the simpler tool.

OpenWebUI:

• Allows you to have a chat interface, as well as something like OpenAI’s “playground” interface to test out prompts.
• Works with any OpenAI or Llama compatible API endpoints. For example, I currently am connecting it to GPT-4, o1, Google Gemini, and Deepseek. The ability to use one interface to switch back and forth between different models is promising.
• Lets you upload a custom knowledgebase for retrieval augmented generation and reference it in multiple chats.
• Provides a tool like OpenAI’s “custom GPT” feature to pre-build a specialized AI tool.
• Lets you integrate custom code for “agentic AI” that your chats can automatically select and invoke when enabled. OpenWebUI calls these “tools”. Tools can give you information, like the weather, current date, or run a calculation for you. They can also interact with the outside world to manipulate files in Google Drive, monitor your infrastructure, or send an email.
• Lets you search the web to get more information to ground your prompt.
• Includes lots of optional third-party libraries, with things like libraries of prompt ideas, custom GPT-like tools you can import, and other custom integrations you can use.

Most users of both tools seem to like to run the tools on their own machines rather than on a cloud server, and instructions are oriented to that use. So, getting a secure setup took a little extra work. For posterity, check out the custom Docker Compose file we ran below.

1. Get a Linux server in the cloud. We ran Ubuntu 22.04 on Amazon Lightsail. You could also use something like Digital Ocean.
2. Make sure your server has a static IP address.
3. Assign a hostname in your DNS provider to the new
4. Install docker.
5. Create a new docker-compose.yml with the contents below. I suggest in a sub directory, like openwebui.
6. Replace the text YOUR.HOSTNAMEHERE.COM with the hostname you added in your DNS provider.
7. Run docker compose up in the directory where you made the docker-compose.yml file.

This docker-compose.yml file does a few things that the default OpenWebUI instructions don’t handle:

1. Proxies the traffic behind an automatically renewing SSL certificate, keeping your password and private data encrypted.
2. Monitors the service in case it crashes and automatically restarts it if it does. (I was getting a weekly crash before adding this. Not ideal, but hey, it’s free and rapidly developing software).
3. Sets you up to be able to run OpenWebUI’s “pipelines”, a handy customization method. Think of pipelines as complex plugins that modify OpenWebUI’s capabilities.

I’m excited to pilot this in my seminar. I’ll report back with the student experience using this tool!