Dec 6th, 2024
Building the Future: A Comprehensive Overview of the AI Tech Stack
AI is no longer some abstract concept with “possible” applications to the world of software: the applications of AI in the software space are here, concrete, and have already been adopted by around 72% of companies, according to a recent McKinsey Global Survey.
The benefits of a solid AI tech stack can often mean the difference between a company that successfully integrates AI solutions into its software, and a company that fails to keep up. Frequently, an AI tech stack is the secret that allows companies to stay competitive in the rapidly changing world of AI-driven business.
What Is an AI Tech Stack?
An AI tech stack is the stack for an application with AI capabilities. Typically, in this context, when we say “AI” we mean “generative AI” – meaning applications capable of generating AI images, text, and other content.
The difference between an AI stack and a regular tech stack is that the AI stack includes libraries, models, drivers, LLM frameworks, and other components that specifically drive the AI portion of the application.
What Are the Components of An AI Tech Stack?
There are many layers to a robust AI technology stack, each crucial to the development, deployment, and maintenance of an AI-capable application.
Compute resources and network infrastructure
All applications, whether AI-capable or not, rely on compute resources at their most basic level to function. AI is incredibly power-hungry, so robust and powerful infrastructure is a must. You might obtain this via shared space on an AWS instance, or perhaps you have your own on-premise hardware capable of powering an artificially intelligent system.
High-speed networking infrastructure is also crucial for efficient data transfer and processing.
Data resources
Raw data is the fuel of any AI-based system. Without data, it is impossible to train and evaluate models. You need data, and lots of it, which means you also need solutions for storing and managing that data. Services like AWS Glue and Azure Data Factory facilitate efficient data management.
An AI tech stack may also need to include vector databases for storing and searching through vector embeddings. A vector embedding is a mathematical representation of a piece of data that is easier for a computer to understand and allows a computer to process, analyze and learn from data more quickly.
Modeling and Training
You’ll need to choose the right models and algorithms to train your AI on the data you provide it. Some types of learning algorithms work better with certain types of data (unsupervised algorithms, for example, can be used to interpret unlabeled data, while supervised learning algorithms require data to be labeled.)
Deep learning frameworks like PyTorch and Tensorflow provide the tools necessary to build and train neural networks, while organizations like OpenAI provide access to foundational models through APIs and endpoints.
Deploying and Monitoring
Deploying an AI-capable application requires a solution capable of both hosting the model and handling network requests. Ideally, a good deployment solution includes monitoring of performance, security, and regressions, and allows for easy scalability. Services like Docker, Kubernetes, and Google Cloud are excellent options.
Key Considerations for Choosing a Technology Stack for Future AI Planning
- Data: The cornerstone of any AI-based system is data. Gathering, storing, and analyzing your data is going to be the main thing influencing your tech stack decisions, especially when it comes to future-proofing your stack. There are many things to consider when thinking about the data you’ll use to power your AI: types of data (structured, unstructured, semi-structured, temporal, spatial), and the methods used to collect, process, and analyze it.
- Models: These days, powerful LLMs can be just an API call away. Many model providers make AI models easily accessible. Some are closed-source and others are open-source. Closed-source models provide access to their functionality via publicly available endpoints, but their internal workings, as well as how their models were trained, are kept private.
Open-source models often make their internal structures and mechanisms, as well as their functionality, publicly available. There are also models that make only their weights and parameters publicly available (open-weight) or that require agreement to the creator’s terms of service (open-model.)
4 Layers of AI and Why are they Essential in Software Development Planning For AI
Generative AI can be broadly broken down into four layers: the foundational layer, model layer, infrastructure layer, and application layer. Here’s an overview of the four layers and why they are essential in software development planning for AI.
- Foundational Layer (Silicone Chips): Gen AI’s foundational layer is comprised of the physical hardware used to power the compute resources. These include silicone chips made by companies like AMD and NVIDIA, as well as SSDs, network infrastructure, and other compute resources.
- Model Layer: Language learning models provided by companies like OpenAI, responsible for analyzing and learning from the provided data.
- Infrastructure Layer: Companies like AWS, Azure, Google Cloud, etc. provide the additional infrastructure for AI, such as database resources, servers that host models and also run the processes that handle network requests and provisioning virtual machines.
- Application Layer: The layer where the trained model is deployed, either to users via some kind of interface, or to other applications via an API. These might be built with JavaScript or Python, using a service like Adobe Firefly, or be as extensive as the Chat interface OpenAI uses for its ChatGPT models.
Building your AI Tech Stack
Putting together your own AI technology stack requires a similar approach to building any tech stack: a step-by-step plan outlining your needs and goals, budget, current tech stack, integrations, and long-term maintenance plan is crucial. In general, you’ll need tools for each layer of your stack:
- Application interface languages and frameworks: Python, JavaScript, Adobe
- Model providers: OpenAI, Anthropics
- LLM frameworks: LlamaIndex, LangChain, Haystack
- Vector and operational databases: MongoDB, SQL
- Monitoring: PromptLayer, Arize, Galileo
- Deployment: Google VertexAI, HuggingFace, Azure
Fortunately, AI isn’t the scary, inaccessibly difficult topic it used to be. These days, any member of your organization can grasp the basics of an AI tech stack, and savvy developers with a background in Python, Kubernetes, Docker, and AWS can pick up new skills that will enable them to work with various parts of this new modern tech stack.
