Prompt engineering and AI are booming, with almost every startup team integrating AI to make things easier for their users.
Today, we are covering 17 projects to maximize your productivity for developers building with AI.
You will find tools related to prompt engineering, code editors, agents and many more exciting things.
This list will surprise you.
1. Latitude LLM - prompt engineering platform to build and refine prompts with AI.
Latitude is the open source prompt engineering platform to build, evaluate, and refine your prompts with AI. You can create and iterate prompts in the platform by using their SDKs or the API.
The best part is that every time a prompt runs, it automatically logs the entire context, the output and other metadata relevant for evaluations and debugging.
This is how the dashboard looks.
✅ There is support for advanced features like parameters, snippets, logic and more.
✅ You get version control for prompts, collaborative prompt manager and even evaluations in batch or real-time.
A basic user flow can be:
-→ Create a new project.
-→ Write your first prompt using the editor.
-→ Test your prompt using the playground with different inputs and see the mode's responses.
-→ Before deploying, you can upload a dataset and run a batch evaluation to assess your prompt’s performance across various scenarios. Watch this video to see how evaluations can analyze the results of your prompts.
-→ You can deploy your prompt as an endpoint for easy integration with your applications.
-→ Use the Logs section to review your prompt’s performance over time.
-→ Refine your prompt and invite team members to your Latitude workspace to collaborate.
Watch this quick demo to learn more.
You can read the docs and the concepts involved like prompts, logs and evaluation that are involved.
You can use this quickstart guide by using the cloud version or self-hosting it.
They have 536 stars on GitHub and are growing very fast.
2. LiveKit Agents - build real-time multimodal AI apps.
LiveKit Agents is an end-to-end framework that enables developers to build intelligent, multimodal voice assistants (AI agents) capable of engaging users through voice, video and data channels.
Let me explain in simple words.
The Agents framework allows you to build AI-driven server programs that can see, hear and speak in real time. Your agent connects with end-user devices through a LiveKit session. During that session, your agent can process text, audio, images or video streaming from a user's device and have an AI model generate any combination of those same modalities as output and stream them back to the user.
✅ They support a lot of SDKs including Swift, Android, Flutter, Rust, Unity, Node, Go, PHP, React and more.
You can get started with pip.
pip install livekit-agents
They also have a lot of plugins that make it easy to process streaming input or generate output. For instance, there are plugins for converting text-to-speech or running inference with popular LLMs. One example of a plugin is:
pip install livekit-plugins-openai
They also provide open source React components and examples for building with LiveKit if you're using React.
You can read the docs and see the list of all plugins that are available. If you want to try, then you can do it at cloud.livekit.io.
If you're looking for some sample apps with code, check these:
⚡ A basic voice agent using a pipeline of STT, LLM, and TTS
⚡ Super fast voice agent using Cerebras hosted Llama 3.1
This is one of the most exciting projects out of 1000+ projects that I have ever seen in open source.
They have 3.2k stars on GitHub and are growing strong.
3. Julep - build stateful AI apps.
Julep is a platform for creating AI agents that remember past interactions and can perform complex tasks.
Imagine you want to build an AI agent that can do more than just answer simple questions. It needs to handle complex tasks, remember past interactions and maybe even use other tools or APIs. That's exactly where Julep comes in.
You can use it to create multi-step tasks incorporating decision-making, loops, parallel processing and a whole lot more.
✅ It will automatically retry failed steps, resend messages, and generally keep your tasks running smoothly.
✅ You can use Julep's document store to build a system for retrieving and using your own data.
mental model of julep
To get started, you can use npm or pip.
npm install @julep/sdk
or
pip install julep
There is also a quick example that I recommend reading where a sample Agent selects a topic, generates 100 related search queries, performs the searches simultaneously, summarizes the results and shares the summary on Discord. With proper code :)
Watch this quick demo and check more examples apps to understand more.
You can read the detailed docs which has a Python quickstart guide, Nodejs quickstart guide, tutorials and how-to guides.
You might say it's similar to Langchain but they both have slightly different concepts. For instance, LangChain is great for creating sequences of prompts and managing interactions with AI models. It has a large ecosystem with lots of pre-built integrations, which makes it easier to run things quickly.
Julep on the other hand is more about building persistent AI agents that can remember things. It shines when you need complex tasks that involve multiple steps, decision-making, and integration with various tools or APIs directly within the agent's process. Read more on the detailed comparison.
Julep has 1.3k stars on GitHub and growing strong.
4. Open WebUI - most loved AI Interface (Supports Ollama, OpenAI API...), runs offline.
Open WebUI is an awesome user-friendly self-hosted chat user interface designed to operate entirely offline.
This can help you build features at a rate you can never imagine.
You can use pip to quickly install it. Check the complete installation guide.
# install Open WebUI
pip install open-webui
# run Open WebUI
open-webui serve
Let's see some of the awesome features.
✅ You can customize the OpenAI API URL to link with LMStudio, GroqCloud, Mistral, OpenRouter and more.
✅ You can use it in your preferred language with our internationalization (i18n) support.
✅ There is an option of hands-free voice and video call features which gives a little more flexibility.
✅ Their official website has clear info on a bunch of models, prompts, tools and functions by the community.
✅ You can load documents directly into the chat or add files to your document library and access them using the # command before a query.
✅ You can perform web searches using providers like SearXNG, Google PSE, Brave Search, serpstack, serper, Serply, DuckDuckGo, TavilySearch and SearchApi to inject the results directly into your chat experience.
Also recommend watching this walkthrough to learn more.
You can read the docs which includes a getting started guide, FAQs (recommend reading) and tutorials.
It is built using Svelte, Python and TypeScript.
They have 41.6k stars on GitHub which says a lot about the popularity.
5. Quivr - RAG framework for building GenAI second brains.
Quivr, your second brain, utilizes the power of GenerativeAI to be your personal assistant. You can think of it as Obsidian but turbocharged with AI powers.
It is a platform that helps you create AI assistants, referred to as Brain. These assistants are designed with specialized cases like some can connect to specific data sources, allowing users to interact directly with the data.
While others serve as specialized tools for particular use cases, powered by Rag technology. These tools process specific inputs to generate practical outputs, such as summaries, translations and more.
Watch a quick demo of Quivr!
Some of the amazing features are:
✅ You can choose the type of Brain you want to use, based on the data source you wish to interact with.
✅ They also provide a powerful feature to share your brain with others. This can be done by sharing with individuals via their emails and assigning them specific rights.
✅ Quivr works offline, so you can access your data anytime, anywhere.
✅ You can access and continue your past conversations with your brains.
✅ But the best one that I loved is that you can install a Slack bot. Refer to this demo to see what you can do. Very cool!
Anyway, read about all the awesome stuff that you can do with Quivr.
You can read the installation guide and 60 seconds installation video. Refer to the docs for any other information.
They have also provided guides on how to deploy Quivr with Vercel, Porter, AWS and Digital Ocean.
It has 36.3k+ Stars on GitHub with 300+ releases.
6. Dify - innovation engine for GenAI apps.
Dify is an open-source platform for building AI applications.
Its intuitive interface combines AI workflow, RAG pipeline, agent capabilities, model management, observability features and more, letting you quickly go from prototype to production.
They combine Backend-as-a-Service and LLMOps to improve the development of generative AI solutions. You can use the cloud or self-host it (refer to docs).
You can even build and test powerful AI workflows on a visual canvas.
Let's see some of the awesome features:
✅ Dify provides 50+ built-in tools for AI agents, such as Google Search, DALL·E, Stable Diffusion and WolframAlpha.
✅ You can monitor and analyze application logs and performance over time.
✅ You can use the RAG pipeline to extract text from PDFs, PPTs and other common document formats.
✅ A lot of integration options are available from dozens of inference providers and self-hosted solutions, covering GPT, Mistral, Llama3, and any OpenAI API-compatible models. A full list of supported model providers can be found here.
✅ You can create AI Agents with just a few clicks, letting them independently use enterprise-defined tools and data to solve complex tasks.
You can read the docs.
Two of the impressive use cases that I loved:
⚡ Building a Notion AI Assistant
⚡ Create a MidJourney Prompt Bot with Dify
Dify has 47.7k stars on GitHub and has a lot of contributors.
7. Micro Agent - AI agent that writes (actually useful) code for you.
AI-assisted coding tools like GitHub Copilot and ChatGPT don't produce very reliable code and they often don't work correctly right out of the box, you find bugs, edge cases, or even references to non-existent APIs.
This can lead to a frustrating loop of trying the generated code, finding issues, going back to the AI for fixes and repeating.
The time spent debugging can negate the total time saved using AI tools in the first place.
Micro Agent uses AI to mitigate the problems of unreliable code generation.
Give it a prompt and it'll generate a test and then iterate on code until all test cases pass.
You can install it using this command.
npm install -g @builder.io/micro-agent
# Next, set your OpenAI API key when prompted or manually using this.
micro-agent config set OPENAI_KEY=<your token>
# Then you can run to start a new coding task
micro-agent
Micro Agent will prompt you to describe the function you want, generate tests, and start writing code in your preferred language to make the tests pass. Once all the tests are green, you'll have a fully functional, test-backed function ready to use.
Let's explore some of the most mind blowing use cases:
⚡ 30-second demo of Micro Agent generating tests and code for a TypeScript function that groups anagrams together from an array of strings.
⚡ Using Micro Agent to generate a simple HTML to AST parser (it was achieved on two iterations).
⚡ Unit test matching.
⚡ Visual matching (experimental).
⚡ Integration with Figma.
Micro Agent can also integrate with Visual Copilot to connect directly with Figma to ensure the highest fidelity possible design to code!
Visual Copilot connects directly to Figma to assist with pixel-perfect conversion, exact design token mapping, and precise usage of your components in the generated output.
Then, Micro Agent can take the output of Visual Copilot and make final adjustments to the code to ensure it passes TSC, lint, tests, and fully matches your design including final tweaks. Amazing right :)
You can read the docs and the official blog where the team discussed everything about the micro agent.
It's open source with 2.8k stars on GitHub.
8. Cline - autonomous coding agent right in your IDE.
The concept seems very similar to Cursor where Cline is an autonomous coding agent capable of creating/editing files, executing commands, and more with your permission every step of the way.
It's a VSCode extension and you can find it in the marketplace. It has 84k+ installs.
Cline works on Claude 3.5 Sonnet's agentic coding capabilities.
✅ Cline supports API providers like OpenRouter, Anthropic, OpenAI, Google Gemini, AWS Bedrock, Azure, and GCP Vertex. You can also configure any OpenAI compatible API or use a local model through Ollama.
✅ You can add context using four different commands.
@url: Paste in a URL for the extension to fetch and convert to markdown, useful when you want to give Cline the latest docs@problems: Add workspace errors and warnings ('Problems' panel) for Cline to fix.
✅ It uses a headless browser to inspect any website, like localhost, allowing it to capture screenshots and console logs. This gives him the autonomy to fix visual bugs and runtime issues without you needing to handhold and copy-pasting error logs yourself.
✅ You can even run commands in the terminal to do awesome stuff.
You can read the docs.
Cline has 7k stars on GitHub.
9. GPT Crawler - create your own custom GPT from a URL.
With GPT Crawler, you can crawl any site to generate knowledge files to make your own custom GPT from one or multiple URLs.
The objective is to make the docs site interactive, people can more simply find the answers they are looking for using a chat interface.
Watch this quick demo!
You will have to configure the crawler and then simply run it. After the crawl is complete, you will have a new output.json file, which includes the title, URL and extracted text from all the crawled pages.
You can now upload this directly to ChatGPT by creating a new GPT. Once uploaded, this GPT assistant will have all the information from those docs and be able to answer unlimited questions about them.
It's officially a assistant in ChatGPT.
You can read the docs on how to get started. You can find all the instructions on the official blog.
If you are wondering how Mitosis compiles those components, then watch this quick tutorial.
They have 18.6k stars on GitHub.
10. Composio - production ready toolset for AI Agents.
Composio is the only tool needed to build complex AI automation software. It allows AI models to access third-party tools and applications to automate their interactions with them.
For instance, you can connect GitHub with the GPT model via Composio and automate reviewing PRs, resolving issues, writing test cases and more.
You can automate complex real-world workflows by using 90+ tools and integration options such as GitHub, Jira, Slack and Gmail.
You can also automate actions like sending emails, simulating clicks, placing orders and much more just by adding the OpenAPI spec of your apps to Composio.
This is how you can use this.
# install it
pip install composio-core
# Add a GitHub integration
composio add github
Here is how you can use the GitHub integration to star a repository.
from openai import OpenAI
from composio_openai import ComposioToolSet, App
openai_client = OpenAI(api_key="******OPENAIKEY******")
# Initialise the Composio Tool Set
composio_toolset = ComposioToolSet(api_key="**\\*\\***COMPOSIO_API_KEY**\\*\\***")
## Step 4
# Get GitHub tools that are pre-configured
actions = composio_toolset.get_actions(actions=[Action.GITHUB_ACTIVITY_STAR_REPO_FOR_AUTHENTICATED_USER])
## Step 5
my_task = "Star a repo ComposioHQ/composio on GitHub"
# Create a chat completion request to decide on the action
response = openai_client.chat.completions.create(
model="gpt-4-turbo",
tools=actions, # Passing actions we fetched earlier.
messages=[
{"role": "system", "content": "You are a helpful assistant."},
{"role": "user", "content": my_task}
]
)
You can read the docs and examples.
Composio has 9k stars on GitHub.
11. Langflow - low-code app builder for RAG and multi-agent AI apps.
Langflow is designed to make AI development easier in real-world scenarios. It's Python-based and agnostic to any model, API or database.
It is kind of a dynamic graph where each node is an executable unit. You can watch the demo.
✅ You can use dynamic inputs by using curly brackets {}.
✅ You can use the full potential of LLMs by easily fine-tuning them from spreadsheets.
✅ You can go beyond the surface and code your own components.
✅ Higher-level components naturally come from AI building blocks. Store and share your creations.
The integration options are huge so you can build almost anything. They also provide composable building blocks which are like pre-built components that can be combined in numerous ways to create AI apps.
You can read the docs which contains a quickstart guide and playground where you can directly prototype make\ing adjustments and observing different outcomes with models.
Lanflow has 31.2k stars on GitHub.
12. OpenLLM - run LLMs as OpenAI compatible API endpoint in the cloud
OpenLLM lets developers run any open-source LLMs as OpenAI-compatible API endpoints with a single command.
⚡ Support llama3, qwen2, gemma and many quantized versions full list. ⚡ OpenAI-compatible API & includes ChatGPT like UI. ⚡ Accelerated LLM decoding with state-of-the-art inference backends. ⚡ Ready for enterprise-grade cloud deployment (Kubernetes, Docker, and BentoCloud).
Get started with the following command.
pip install openllm # or pip3 install openllm
openllm hello
OpenLLM provides a chat user interface (UI) at the /chat endpoint for an LLM server. You can visit the chat UI at http://localhost:3000/chat and start different conversations with the model.
As I said before, OpenLLM supports LLM cloud deployment via BentoML, the unified model serving framework and BentoCloud, an AI inference platform for enterprise AI teams.
If you don't know, BentoCloud provides a fully managed infrastructure optimized for LLM inference with autoscaling, model orchestration, observability, which is just a fancy way of saying it allows you to run any AI model in the cloud.
Once the deployment is complete, you can run model inference on the BentoCloud console.
You can read about the supported models and how to start the LLM server.
Explore docs as you can also chat with a model in the CLI using openllm run and specifying model version - openllm run llama3:8b.
If you love exploring walkthroughs, watch this demo by Matthew!
They have 9.9k stars on GitHub, almost hitting the mark of 10k :)
13. GPT Engineer - AI builds what you ask.
GPT-engineer lets you specify software in natural language, sit back, and watch as an AI writes and executes the code, and you can ask the AI to implement improvements.
It's safe to say it's an engineer who doesn't need a degree 😅
It's a commercial project for the automatic generation of web apps. It features a UI for non-technical users connected to a git-controlled codebase.
I know this feels confusing, so watch the below demo to understand how you can use GPT Engineer.
You can get started by installing the stable release using this command.
python -m pip install gpt-engineer
By default, gpt-engineer expects text input via a prompt file. It can also accept image inputs for vision-capable models. This can be useful for adding UX or architecture diagrams as additional context for GPT engineer. Read about all the awesome features.
If you want a complete walkthrough, watch this awesome demo by David!
I recommend checking out the roadmap to understand the overall vision.
They have 52.2k stars on GitHub and are on the v0.3 release.
14. Void - open source alternative to cursor.
As you know, a lot of projects like PearAI and Zed came out after Cursor launched, but most of these are just forks of VSCode and don’t really improve the ecosystem that much.
If you're looking to switch, I’d recommend Cursor (not open source), Continue (similar to Cursor) and Void which is an open source alternative to Cursor.
I'm covering Void because it's backed by YCombinator so it makes them credible enough. They have not released it but you can easily get the early access to try it out.
It has very similar AI syntax features as Cursor.
✅ You can do intelligent searches With AI.
✅ You can view and edit underlying prompts.
✅ There is contextual awareness, third party integrations, Fine-Tuned generation and you can even host Ollama locally to never run out of API credits again.
As I said previously, Void is a fork of VS Code so you can transfer over all your themes, keybinds and settings with just one click.
I've covered Continue before so skipped it this time, but I would definitely prefer Cursor until Void is completely released.
Void has 7.5k stars on GitHub and is still in its early release.
15. Unsloth - Finetune Llama 3, Mistral, Phi & Gemma LLMs 2-5x faster with 80% less memory.
Unsloth makes finetuning large language models like Llama-3, Mistral, Phi-3, and Gemma 2x faster, use 70% less memory, and with no degradation in accuracy!
✅ What is finetuning?
If we want a language model to learn a new skill, a new language, some new programming language, or simply want the language model to learn how to follow and answer instructions like how ChatGPT functions, we do finetuning!
Finetuning is the process of updating the actual brains of the language model through some process called back-propagation. But, finetuning can get very slow and very resource intensive.
Unsloth can be installed locally or through another GPU service like Google Colab. Most use Unsloth through the interface Google Colab which provides a free GPU to train with.
Some of the things that stand out:
✅ Open source trains 5x faster, and the pro version claims to be 30x faster.
✅ No approximation methods are used resulting in a 0% loss in accuracy.
✅ No change of hardware, Works on Linux and Windows via WSL.
You can read the installation instructions and performance benchmarking tables on the website.
You can read the docs and all the uploaded models on Hugging Face directly.
They have also provided a detailed guide on How to Finetune Llama-3 and Export to Ollama.
Unsloth has 16.8k+ stars on GitHub.
16. Khoj - Your AI second brain.
Khoj is the open source, AI copilot for search. Easily get answers without having to sift through online results or your own notes.
Khoj can understand your Word, PDF, org-mode, markdown, plaintext files, GitHub projects and even Notion pages.
It's available as a Desktop app, Emacs package, Obsidian plugin, Web app and Whatsapp AI. Obsidian with Khoj might be the most powerful combo!
You can get started with Khoj locally in a few minutes with the following commands.
$ pip install khoj-assistant
$ khoj
Watch it in action!
Some of the exciting features:
✅ You can share your notes and documents to extend your digital brain.
✅ Your AI agents have access to the internet, allowing you to incorporate real-time information.
✅ You'll get a fast, accurate semantic search on top of your docs.
✅ Your agents can create deeply personal images and understand your speech.
For instance, saying: "Create a picture of my dream house, based on my interests". It will draw this!
Read all the features including shareable chat, online chat, file summarization, and complete details in various categories.
You can read the docs and you can try Khoj Cloud to try it quickly.
Watch the complete walkthrough on YouTube!
It has 12.8k stars on GitHub and is backed by YCombinator.
17. Prompt Tools - tools for prompt testing.
This project has a collection of open source self-hostable tools for experimenting with, testing, and evaluating LLMs, vector databases and prompts. The core idea is to enable developers to evaluate using familiar interfaces like code, notebooks and a local playground.
In just a few lines of code, you can test your prompts and parameters across different models (whether you are using OpenAI, Anthropic or LLaMA models). You can even evaluate the retrieval accuracy of vector databases.
Get started with pip.
pip install prompttools
They provide notebook examples which you can run.
You can read the docs amd check their playground.
Prompt tools have 2.7k stars on GitHub.
With the right tools, building AI features is much easier and your team can deliver quality work faster.
Do you think these projects will help teams build AI features at least faster than before?
Have a great day! Till next time.
You can connect me on GitHub, Twitter and LinkedIn if you loved the content :)
"Write more, inspire more!"
