AWS Explained: What It Is, What It Does Well, and What You Should Watch Out For

If you spend any time around technology, startups, software, or even business news, you will eventually hear people talk about AWS. Some people mention it as if it is simply a place to host websites. Others talk about it as the backbone of the modern internet. The truth is somewhere in between, but much closer to the second idea.

AWS, which stands for Amazon Web Services, is one of the largest and most important cloud computing platforms in the world. It gives businesses access to computing power, storage, databases, networking, security tools, analytics, artificial intelligence services, and much more. Instead of buying and maintaining physical servers in their own buildings, companies can rent what they need from AWS and scale up or down depending on demand.

That sounds simple on the surface, but AWS is much more than rented infrastructure. It has become a full technology ecosystem that supports startups, global enterprises, governments, financial institutions, media companies, ecommerce platforms, and AI applications. It is one of the central forces shaping how digital systems are built today.

Why AWS matters

The easiest way to understand AWS is to think of it as a giant toolbox for building and running digital services. A company that wants to launch an app, store customer data, analyze business information, deploy machine learning models, or run secure internal systems can do all of that on AWS.

This matters because technology no longer belongs only to technology companies. Banks are technology companies in practice. Retailers are technology companies in practice. Logistics companies, hospitals, entertainment firms, and manufacturers all depend on software and data. AWS sits underneath a large share of that activity.

Its importance is not only technical. AWS is also a major business and economic force. It influences corporate investment decisions, startup costs, enterprise software strategy, AI infrastructure spending, energy demand, and even regulatory debates about digital sovereignty and market concentration.

The main things AWS does

AWS offers a very wide set of services, but they can be understood in a few major categories.

Compute

This is one of the core foundations of AWS. Compute services allow companies to run applications, websites, internal systems, and background jobs.

Amazon EC2 gives businesses virtual servers with a high level of control. AWS Lambda allows developers to run code without managing servers directly. AWS also supports containers, which are a common way to package and deploy modern software.

For companies, this means they can choose the level of control or convenience they want. Some teams want full infrastructure control. Others want to focus on writing code and let AWS handle more of the operational work.

Storage

Every digital system needs a place to keep data. AWS provides multiple ways to store files, backups, application assets, and large datasets.

Amazon S3 is one of the best known AWS services. It is widely used for storing objects such as images, videos, documents, logs, backups, and data lakes. Other storage services support different performance and architectural needs.

Storage may sound boring, but it is one of the most essential building blocks in the cloud. Reliable storage is one of the reasons AWS became so deeply embedded in modern software infrastructure.

Databases

AWS offers several types of databases, including relational databases, managed data warehouses, and NoSQL systems. This allows companies to choose tools that match their application needs.

For example, some applications need structured transactional data. Others need fast scaling with flexible schemas. Others need analytical processing across very large datasets. AWS has options for all of these use cases.

The major appeal here is convenience. Instead of installing, tuning, patching, backing up, and operating databases manually, companies can use managed AWS services that reduce operational effort.

Networking and content delivery

AWS provides networking tools that help organizations connect applications, isolate environments, route traffic, and improve performance. It also supports content delivery so digital content can be served more efficiently to users around the world.

This becomes especially important for businesses with international users or high traffic applications. Low latency and reliability are not luxuries when a company depends on digital products for revenue.

Security and identity

Security is one of AWS’s most important strengths, but it is also one of the most misunderstood areas.

AWS provides identity management, encryption tools, threat detection, compliance support, and many other security services. It invests heavily in infrastructure level security and offers a broad set of tools for protecting workloads.

However, using AWS does not mean security becomes automatic. AWS follows a shared responsibility model. That means AWS secures the underlying infrastructure, while customers remain responsible for how they configure and use the services. If a company misconfigures permissions or exposes sensitive data, that is still the company’s problem.

Data analytics

Many businesses do not just want to store data. They want to understand it. AWS offers services for data pipelines, data lakes, warehousing, analytics, monitoring, and reporting.

This is one reason AWS has become so important beyond engineering teams. It supports business intelligence, forecasting, operations analysis, customer behavior studies, and many other forms of decision making.

Artificial intelligence and machine learning

This is one of the most important current areas for AWS. Cloud computing and AI are increasingly tied together, because training models and running AI applications require vast computing resources, storage, orchestration, and specialized infrastructure.

AWS now offers a broad set of AI and machine learning tools, including services that support model building, model deployment, and generative AI applications. This includes platforms designed to help businesses build AI systems without having to assemble every piece from scratch.

For many organizations, AWS is not just where their software runs. It is where their future AI products will be built.

Hybrid and edge infrastructure

Not every company can or should move everything into the public cloud at once. Some workloads need to stay closer to local operations for latency, compliance, or legacy integration reasons.

AWS has expanded into hybrid and edge solutions so businesses can use AWS tools and patterns outside traditional cloud regions. This is important because real world enterprise technology is rarely clean and simple. Most large organizations operate across old systems, new systems, public cloud, private infrastructure, and multiple geographies at the same time.

What makes AWS so attractive

AWS has several qualities that explain why it became such a dominant force.

Breadth of services

AWS offers a huge range of tools under one umbrella. That means companies can often build large parts of their technology stack on a single platform.

This can simplify procurement, integration, and operations. It can also help teams move faster because they do not need to stitch together as many unrelated vendors.

Global scale

AWS operates a massive global infrastructure footprint. That allows companies to serve users in different regions, build resilient architectures, and improve business continuity.

Global scale is not just a technical advantage. It is also a strategic advantage for businesses that want to expand internationally or serve customers across multiple markets.

Flexibility

AWS works for very different kinds of users. A startup can use it to launch a product with limited upfront spending. A large bank can use it for highly controlled enterprise systems. A media platform can use it to distribute content at scale. A research team can use it for large computational workloads.

This range of use cases is one of AWS’s greatest strengths.

Speed of innovation

AWS keeps releasing new services, updates, and capabilities. That rapid pace helps it stay relevant as technology changes.

In recent years, AI has been the clearest example. AWS has invested aggressively so that it remains a serious platform for enterprises building and deploying AI systems.

Strong ecosystem

AWS is not just a vendor with its own products. It also supports a huge ecosystem of software partners, consultants, security vendors, system integrators, and developers. This makes it easier for organizations to find expertise, third party tools, and implementation support.

The downsides and risks of AWS

For all its strengths, AWS is not perfect. In fact, many of its biggest weaknesses come from the same qualities that make it powerful.

Complexity

AWS offers so many services that it can become overwhelming. A new user may see flexibility. An experienced architect may see hidden complexity.

Building well on AWS requires good decisions about identity, networking, storage, cost management, access control, observability, disaster recovery, and architecture. Companies that move too quickly without strong design discipline can create environments that are difficult to manage.

In other words, AWS is powerful, but it does not automatically create simplicity.

Cost can become difficult to control

One of the big promises of cloud computing is financial flexibility. Instead of buying hardware upfront, companies pay for usage.

That can be a real advantage. But over time, many organizations discover that cloud bills become hard to predict. Costs can rise due to data transfer, idle resources, logging, storage growth, duplicate environments, inefficient workloads, and poor governance.

AWS can absolutely save money in some situations, especially when flexibility and speed matter. But it can also become expensive if usage is not managed carefully.

Vendor dependence

The deeper a company goes into AWS specific tools and services, the harder it may become to move elsewhere. This is often called vendor lock in.

Some degree of dependence is normal in any platform choice, but in cloud computing it can become strategically important. If a company builds core systems around AWS native services, migration can become time consuming, expensive, and operationally risky.

This issue has become more visible in regulatory discussions, especially in regions where governments are paying closer attention to cloud competition and switching barriers.

Outage and concentration risk

AWS is highly reliable compared with what most organizations could build on their own, but it is not immune to outages or disruptions. No large scale platform is.

This creates an important lesson for customers. If a business becomes too dependent on one cloud provider without building resilient architecture, then even a limited disruption can have wide impact. The real risk is not simply whether AWS has outages. The real risk is whether customers design systems that can tolerate failure gracefully.

Security responsibility does not disappear

Many people assume that moving to the cloud means security becomes someone else’s problem. That is not how it works.

AWS provides powerful security capabilities, but customers still need strong governance, access control, monitoring, and operational discipline. Misconfigured storage, weak permission models, and poor credential management can still lead to major incidents.

Cloud platforms can improve security posture, but only when they are used carefully.

Sustainability concerns

AWS has made public commitments around sustainability, renewable energy, and long term environmental goals. Those efforts matter and should not be dismissed.

At the same time, the broader AI and data center boom is raising real concerns about electricity consumption, water usage, and local infrastructure stress. As cloud and AI demand keep growing, these environmental and political questions are becoming more important.

AWS in business, competition, and the economy

To understand AWS today, it is not enough to look only at technology. You also have to look at market structure and macroeconomics.

AWS remains one of the most important cloud companies in the world and a major profit engine for Amazon. Even as competition has become stronger, AWS still holds a leading position in cloud infrastructure.

That leadership matters because the cloud market is not just about hosting websites. It now underpins enterprise software, digital transformation, AI deployment, ecommerce operations, media delivery, and much of the infrastructure behind online services.

At the same time, competition has become more intense. Microsoft Azure and Google Cloud continue to grow aggressively. Specialized providers focused on AI infrastructure are also becoming more relevant. This means AWS is still the market leader, but it operates in a much more contested environment than it did in earlier years.

The AI boom has made this even more interesting. Cloud providers are investing enormous amounts of capital into data centers, chips, networking, and infrastructure to support the next wave of AI demand. AWS is one of the central players in this race.

This has economic consequences beyond the technology sector. Large scale cloud and AI investment affects semiconductor demand, construction, land use, electricity markets, and investor expectations. It also affects inflation pressures in certain inputs and can shape regional economic development.

In other words, AWS is no longer just part of the software economy. It is now part of the industrial and energy story as well.

Regulatory and political issues

Another reason AWS is worth watching is regulation.

Governments and regulators are increasingly asking difficult questions about cloud competition, data sovereignty, switching costs, and the concentration of digital infrastructure in the hands of a few large providers.

For customers in Europe and other regions, issues such as where data is stored, who controls operations, and what legal jurisdiction applies are becoming much more important. This is one reason sovereign cloud and regional control models have received more attention.

These questions are not temporary. They reflect a deeper shift in how governments view cloud infrastructure. Cloud is no longer seen only as a commercial service. It is increasingly treated as strategic digital infrastructure.

Who should use AWS

AWS is a strong choice for organizations that need scale, flexibility, a wide service catalog, and global reach. It is particularly attractive for companies building complex digital products, enterprise platforms, analytics systems, and AI driven applications.

It is also a strong fit for businesses that value managed services and want to move faster without owning every piece of infrastructure themselves.

Who should be cautious

AWS may not be the ideal choice for every situation.

Very small teams may find it more complex than necessary. Organizations with predictable and simple workloads may discover that a narrower or more specialized infrastructure approach is easier to manage. Companies that are highly sensitive to vendor dependence may want to limit how deeply they rely on provider specific services.

The right question is not whether AWS is good or bad. The right question is whether it matches the technical, financial, and organizational realities of the business using it.

Final thoughts

AWS is one of the most powerful platforms in modern technology. It has changed how companies build software, launch products, store data, and scale operations. Its strengths are clear: enormous breadth, global infrastructure, security maturity, deep enterprise relevance, and a central role in the future of AI.

But its weaknesses are real as well. It can be complex, expensive, difficult to govern, and hard to leave once deeply adopted. It also sits at the center of larger debates about competition, energy use, regulation, and the economics of AI infrastructure.

That is what makes AWS so important. It is not just a cloud platform. It is a foundation of the digital economy, a major business engine, a strategic technology asset, and increasingly a topic of political and economic significance.

Anyone who wants to understand the future of technology, business, and infrastructure should pay close attention to AWS.

OpenAI Vector Stores API

OpenAI Vector Stores API Explained

1. What the Vector Stores API is

A vector store is a hosted knowledge base that OpenAI manages for you. You upload files, and OpenAI automatically parses the content, splits it into chunks, creates embeddings, and stores everything in a system that supports both semantic search and keyword search. The stored content can then be searched directly, or used by higher level tools like the file_search tool in the Responses API and the Assistants API.

This makes vector stores a core building block for retrieval augmented generation. Instead of forcing a model to rely only on its built in knowledge, you give it access to your own documents and let it retrieve the most relevant passages at the moment it needs them.

2. The most important capabilities

2.1 Store and manage searchable knowledge bases

With the Vector Stores API, you can create and manage vector_store objects that track ingestion status, file counts, usage, and expiration policy. A vector store becomes usable when its status is completed.

2.2 Automatic ingestion pipeline for files

When you attach a file to a vector store, ingestion is automatic: parsing, chunking, embedding, and indexing happen without you implementing that pipeline yourself. File ingestion is asynchronous, so you typically poll until it is finished, or use official SDK helpers that upload and poll for you.

2.3 Semantic search directly on the store

You can query a vector store using client.vector_stores.search(...) with a natural language query and get back matching chunks and related metadata, including similarity scores and file of origin.

2.4 Power the file_search tool for Responses and Assistants

Vector stores are the backing index for the hosted file_search tool. In the Responses API, you pass vector_store_ids to the file_search tool so the model can retrieve relevant passages before answering.

In the Assistants API, you attach vector stores to assistants and threads via tool_resources, and the file search tool can query them during a run.

2.5 Metadata, filtering, batching, chunking strategy, and expiration

You can attach attributes to vector store files and use filtering in file search calls, for example filtering by a category attribute.

You can attach many files at once using file batches, with a maximum batch size of 2000 files, and optionally override metadata or chunking strategy per file.

You can also set an expiration policy using expires_after so the store is deleted after a period of inactivity, and you are no longer charged after expiration.

3. How to use it in practice, detailed workflow

3.1 Choose the interaction style

You typically choose one of these approaches:

1. Direct retrieval workflow
   You create a vector store, upload files, then call client.vector_stores.search(...) yourself and decide what to do with the results.

2. Hosted tool workflow
   You create a vector store, upload files, then let the model call the hosted file_search tool from within the Responses API or Assistants API.

3.2 Create a vector store

Creating a store is straightforward:

from openai import OpenAI

client = OpenAI()

vector_store = client.vector_stores.create(name="Support FAQ")
print(vector_store.id, vector_store.status)

A vector store tracks ingestion progress through fields like status and file_counts.

3.3 Upload files and wait for ingestion to complete

The easiest path in Python is to use the SDK helper that uploads and polls:

from openai import OpenAI

client = OpenAI()

vector_store = client.vector_stores.create(name="Support FAQ")

client.vector_stores.files.upload_and_poll(
    vector_store_id=vector_store.id,
    file=open("customer_policies.txt", "rb"),
)

vs = client.vector_stores.retrieve(vector_store.id)
print(vs.status, vs.file_counts)

OpenAI notes ingestion is asynchronous and recommends polling helpers, or monitoring file_counts until processing completes.

Important limits and defaults to keep in mind:

1. Maximum file size is 512 MB

2. Each file should contain no more than 5,000,000 tokens

3. Default chunking is 800 token chunks with 400 token overlap

If you have many files, use the file batch helper shown in the Assistants docs:

from openai import OpenAI

client = OpenAI()

vector_store = client.vector_stores.create(name="Financial Statements")

file_paths = ["edgar/goog-10k.pdf", "edgar/brka-10k.txt"]
file_streams = [open(p, "rb") for p in file_paths]

file_batch = client.vector_stores.file_batches.upload_and_poll(
    vector_store_id=vector_store.id,
    files=file_streams,
)

print(file_batch.status)
print(file_batch.file_counts)

This pattern is designed for multi file ingestion with polling.

3.4 Run semantic search directly

Once ingestion is complete, you can search:

results = client.vector_stores.search(
    vector_store_id=vector_store.id,
    query="What is the return policy?",
)

for r in results.data:
    print("score:", r.score)
    print("file_id:", r.file_id)
    print("text:", r.text[:200])
    print()

OpenAI describes this as semantic search that can find matches even when few or no keywords overlap.

4. Combining vector stores with other OpenAI APIs, detailed patterns

4.1 Vector stores plus Responses API

This is the most direct way to get RAG behavior with minimal orchestration. You pass the vector store into the file_search tool, and the model can choose to call it.

from openai import OpenAI
client = OpenAI()

response = client.responses.create(
    model="gpt-4.1",
    input="Answer using the internal docs. What is deep research by OpenAI?",
    tools=[{
        "type": "file_search",
        "vector_store_ids": [vector_store.id],
    }],
    include=["file_search_call.results"],
)

print(response.output_text)

This usage, including the include=["file_search_call.results"] option to inspect retrieved results, is shown in the File Search guide.

If you store attributes on files, you can filter retrieval:

response = client.responses.create(
    model="gpt-4.1",
    input="Summarize announcements only.",
    tools=[{
        "type": "file_search",
        "vector_store_ids": [vector_store.id],
        "filters": {
            "type": "in",
            "key": "category",
            "value": ["announcement"],
        },
    }],
)
print(response.output_text)

The File Search guide shows a filters object with an in filter on a metadata key.

4.2 Vector stores plus Assistants API

This is useful when you want a persistent assistant that can keep using the same knowledge base across many user conversations.

Key ideas from the docs:

1. You attach at most one vector store to an assistant and at most one vector store to a thread.

2. If the thread also has a vector store created through message attachments, the file search tool can query both the assistant store and the thread store during a run.

A typical attachment step looks like this:

assistant = client.beta.assistants.update(
    assistant_id=assistant.id,
    tool_resources={"file_search": {"vector_store_ids": [vector_store.id]}},
)

That exact tool_resources pattern is shown in the Assistants File Search documentation.

The Assistants docs also document default retrieval configuration used by the file search tool, including chunk size, overlap, embedding model, maximum chunks added to context, and ranking behavior.

4.3 Vector stores plus Retrieval API style workflows

Sometimes you do not want the model to decide when to retrieve. You want a deterministic pipeline:

1. Run client.vector_stores.search(...)

2. Take the top results

3. Feed them into a model call as context

The Retrieval guide describes retrieval as useful on its own and especially powerful when combined with models to synthesize responses.

4.4 Vector stores plus tool calling for structured actions

A common production pattern is:

1. Use file search for unstructured knowledge such as policies and manuals

2. Use function tools for structured actions such as database lookup, order status, scheduling, or calculations

3. Let the model combine retrieved passages with fresh structured data

Vector stores handle your document knowledge base, while function tools handle operations and up to date system data. The File Search guide emphasizes that file search is a hosted tool that the model can call automatically.

5. A complete Python example

This example shows:

1. Creating a vector store

2. Uploading files with polling

3. Adding per file attributes

4. Using Responses API with filtered file search

from openai import OpenAI

client = OpenAI()

# 1. Create a vector store
vector_store = client.vector_stores.create(name="Company Knowledge Base")

# 2. Upload multiple files and poll until processed
file_paths = ["docs/policies.txt", "docs/announcements.txt"]
streams = [open(p, "rb") for p in file_paths]

client.vector_stores.file_batches.upload_and_poll(
    vector_store_id=vector_store.id,
    files=streams,
)

# 3. Attach attributes to a specific file already uploaded to the OpenAI Files API
# If you already have a file_id, you can add it with attributes like this:
# client.vector_stores.files.create(vector_store_id=vector_store.id, file_id="file_123", attributes={...})

# 4. Ask a question, restricting retrieval to a category
resp = client.responses.create(
    model="gpt-4.1",
    input="Summarize only the announcements in two paragraphs.",
    tools=[{
        "type": "file_search",
        "vector_store_ids": [vector_store.id],
        "filters": {
            "type": "in",
            "key": "category",
            "value": ["announcement"],
        },
    }],
)

print(resp.output_text)

The ingestion helpers, per file attributes, and filters used above are all documented in OpenAI’s Retrieval and File Search guides. 

OpenAI Files API

How It Works, What It Is For, and How to Use It with Other APIs

The OpenAI Files API is the platform’s shared file storage layer. You upload a file once, receive a file_id, and then reuse that file across other OpenAI endpoints such as Responses file inputs, Batch jobs, vector stores for file search, and fine tuning. The Files API itself is simple, but it becomes powerful because many other APIs accept file_id as a reference.

---

1. What You Can Do with the Files API

The Files API provides a small set of core operations.

1. Upload a file
   Endpoint: POST /v1/files
   Inputs include the file bytes, a required purpose, and an optional expiration policy. Individual files can be up to 512 MB, and a project can store up to 2.5 TB.

2. List files
   Endpoint: GET /v1/files
   Supports pagination and filtering by purpose.

3. Retrieve file metadata
   Endpoint: GET /v1/files/{file_id}
   Returns info such as filename, size, purpose, created time, and optional expiration time.

4. Download file content
   Endpoint: GET /v1/files/{file_id}/content
   Returns the original file contents.

5. Delete a file
   Endpoint: DELETE /v1/files/{file_id}
   Deletes the file and removes it from all vector stores that reference it.

File purposes matter

When you upload, you must set a purpose. Typical values include assistants, batch, fine-tune, vision, user_data, and evals.

Purposes control what downstream APIs will accept the file, and sometimes the allowed formats. For example, Batch input must be .jsonl and is limited to 200 MB. Fine tuning also requires .jsonl.

Expiration policy

You can optionally set expires_after with an anchor of created_at and a number of seconds. By default, files uploaded with purpose=batch expire after 30 days, while other purposes persist until manually deleted.

---

2. The Most Important Function of the Files API

The single most important function is: turning a raw document into a reusable identifier.

Once you have a file_id, you can:

1. Feed the file directly into the Responses API as an input_file item.

2. Put the file into a vector store so the model can search it via the file_search tool.

3. Use the file as the input for a Batch job, where each line of a .jsonl file represents a request.

4. Use the file as training data for fine tuning workflows that require .jsonl.

That is why the Files API is usually the first step in any workflow that involves documents at scale.

---

3. Using the Files API Step by Step (Detailed)

3.1 Install and authenticate (Python)

The official OpenAI Python library supports Python 3.9 or newer.

# pip install openai

import os
from openai import OpenAI

# Recommended: export OPENAI_API_KEY in your environment
client = OpenAI()

3.2 Upload a file

Key inputs:

1. file: a binary stream, or a tuple for filename plus content

2. purpose: required

3. expires_after: optional expiration policy

from openai import OpenAI

client = OpenAI()

uploaded = client.files.create(
    file=open("example.pdf", "rb"),
    purpose="user_data",
    expires_after={
        "anchor": "created_at",
        "seconds": 7 * 24 * 60 * 60,  # 7 days
    },
)

print(uploaded.id)
print(uploaded.filename)
print(uploaded.purpose)

Limits to keep in mind:

1. One file can be up to 512 MB

2. A project can store up to 2.5 TB total

3.3 List files

You can paginate with after, set limit, sort by created_at, and filter by purpose.

files_page = client.files.list(
    purpose="user_data",
    limit=20,
    order="desc",
)

for f in files_page.data:
    print(f.id, f.filename, f.bytes, f.created_at, f.purpose)

3.4 Retrieve metadata for one file

info = client.files.retrieve("YOUR_FILE_ID")
print(info.id, info.filename, info.bytes, info.created_at, info.purpose, info.expires_at)

This maps to GET /v1/files/{file_id}.

3.5 Download file content

You can use the SDK method client.files.content(file_id) which returns a binary response object.
If you want a fully explicit approach, you can directly call the REST endpoint:

import os
import requests

api_key = os.environ["OPENAI_API_KEY"]
file_id = "YOUR_FILE_ID"

resp = requests.get(
    f"https://api.openai.com/v1/files/{file_id}/content",
    headers={"Authorization": f"Bearer {api_key}"},
    stream=True,
)
resp.raise_for_status()

with open("downloaded_file.bin", "wb") as out:
    for chunk in resp.iter_content(chunk_size=1024 * 1024):
        if chunk:
            out.write(chunk)

print("Saved downloaded_file.bin")

The endpoint is GET /v1/files/{file_id}/content.

3.6 Delete a file

Deleting removes the file and also removes it from any vector stores that include it.

deleted = client.files.delete("YOUR_FILE_ID")
print(deleted.deleted)

---

4. Combining the Files API with Other OpenAI APIs (Detailed)

4.1 Use files as direct input to the Responses API

The Responses API can accept a file in three common ways:

1. By file_id returned from the Files API

2. By Base64 encoded data

3. By an external URL

It also processes different file types differently. PDFs can include both extracted text and page images on vision capable models, while many other document types are text extracted only. Spreadsheet files have a special augmentation flow.

Example: upload, then ask a question about the file

from openai import OpenAI

client = OpenAI()

file_obj = client.files.create(
    file=open("document.pdf", "rb"),
    purpose="user_data"
)

resp = client.responses.create(
    model="gpt-5",
    input=[
        {
            "role": "user",
            "content": [
                {"type": "input_file", "file_id": file_obj.id},
                {"type": "input_text", "text": "Summarize the main points and list key risks."},
            ],
        }
    ],
)

print(resp.output_text)

This pattern is best when you have a small number of files and the question is scoped enough that the model can consume the file content directly.

4.2 Use files for retrieval with vector stores and the file_search tool

If your files are large, many, or you want retrieval augmented generation, use file search. File search lets the model retrieve relevant passages from a knowledge base using semantic and keyword search.

Workflow overview:

1. Upload file with the Files API

2. Create a vector store

3. Add the file to the vector store

4. Query using the Responses API with the file_search tool and your vector store id

Python example (end to end):

import time
import requests
from io import BytesIO
from openai import OpenAI

client = OpenAI()

def upload_from_url(url: str) -> str:
    r = requests.get(url)
    r.raise_for_status()
    file_content = BytesIO(r.content)
    filename = url.split("/")[-1]

    uploaded = client.files.create(
        file=(filename, file_content),
        purpose="assistants",
    )
    return uploaded.id

file_id = upload_from_url("https://cdn.openai.com/API/docs/deep_research_blog.pdf")

vector_store = client.vector_stores.create(name="knowledge_base")

client.vector_stores.files.create(
    vector_store_id=vector_store.id,
    file_id=file_id,
)

# Simple readiness check by listing files and inspecting status fields in the response
# Some apps use SDK helpers that poll until processing completes
for _ in range(30):
    vs_files = client.vector_stores.files.list(vector_store_id=vector_store.id)
    # Break when processing is done in your environment
    # You can print vs_files and inspect fields like status in the returned objects
    time.sleep(2)

response = client.responses.create(
    model="gpt-4.1",
    input="What is deep research by OpenAI?",
    tools=[{
        "type": "file_search",
        "vector_store_ids": [vector_store.id],
        "max_num_results": 3,
    }],
    include=["file_search_call.results"],
)

print(response.output_text)

This uses the documented vector store creation, adding files, and invoking file search from the Responses API.

Important lifecycle note:

1. Removing a file from a vector store is not the same as deleting the file itself. Removing only detaches it from that vector store.

2. Deleting the file removes it from all vector stores.

Also note: the Assistants API is deprecated and scheduled to shut down on August 26, 2026, with Responses positioned as the preferred API going forward.

4.3 Use the Files API with the Batch API

Batch processing starts with a .jsonl file where each line is one request. The Batch guide lists supported endpoints including /v1/responses.

Step A: Create a .jsonl batch input file

Here is a simple example that targets /v1/responses. The body fields match the normal request body you would send to that endpoint.

import json

requests_list = [
    {
        "custom_id": "job_001",
        "method": "POST",
        "url": "/v1/responses",
        "body": {
            "model": "gpt-5",
            "input": "Classify the sentiment: I love this product."
        },
    },
    {
        "custom_id": "job_002",
        "method": "POST",
        "url": "/v1/responses",
        "body": {
            "model": "gpt-5",
            "input": "Classify the sentiment: This was a disappointing purchase."
        },
    },
]

with open("batchinput.jsonl", "w", encoding="utf-8") as f:
    for item in requests_list:
        f.write(json.dumps(item, ensure_ascii=False) + "\n")

Step B: Upload the .jsonl file via Files API with purpose="batch"

from openai import OpenAI

client = OpenAI()

batch_input_file = client.files.create(
    file=open("batchinput.jsonl", "rb"),
    purpose="batch",
)

print(batch_input_file.id)

Step C: Create the batch job using the file id

batch = client.batches.create(
    input_file_id=batch_input_file.id,
    endpoint="/v1/responses",
    completion_window="24h",
    metadata={"description": "sentiment classification"},
)

print(batch.id, batch.status)

Batch input files have size constraints, and the Files API upload docs call out that Batch input supports .jsonl up to 200 MB.

4.4 Use the Files API for fine tuning data

Fine tuning training files are uploaded via Files API and must be .jsonl.

A minimal first step is just the file upload:

from openai import OpenAI

client = OpenAI()

training_file = client.files.create(
    file=open("training.jsonl", "rb"),
    purpose="fine-tune",
)

print(training_file.id)

You then pass that file_id into the fine tuning job creation API for your chosen fine tuning workflow.

---

5. When to Use Files API vs Uploads API

If your file fits within the standard Files API limit, upload directly with POST /v1/files. Files can be up to 512 MB.

If you need to upload larger data, OpenAI also provides an Uploads API that supports multipart uploading up to 8 GB total and produces a normal File object when completed.