The most important question in artificial intelligence is no longer whether a closed commercial API gives a slightly better answer on a benchmark. The real question is who controls the infrastructure, the data, the cost curve, the audit trail and the right to adapt the system. Open models, especially open-weight models, have changed the economics of AI. Universities, public agencies, municipalities, research centres and small companies can now run capable language models on infrastructure they own, rather than depending entirely on remote proprietary platforms.

There is, however, an important distinction. A fully open AI model should expose not only weights, but also the code, training process, data documentation, evaluation methods and enough artefacts for serious inspection and reuse. An open-weight model lets users download and run the weights, but may still keep training data and key pipeline details private. For public interest deployments, fully open models such as OLMo and Apertus should be preferred whenever possible. For operational deployments, strong open-weight families such as Qwen, DeepSeek, Llama, Gemma, Mistral, GLM, Kimi and gpt-oss can also be useful, provided their licence, safety profile and governance limits are documented.

A Linux server with 4× AMD Radeon RX 7900 XTX provides 96 GB of aggregate GPU memory. This does not turn the system into a single seamless 96 GB accelerator. The cards communicate through PCIe, and performance depends on tensor parallelism, quantization, context length, KV cache size, batching strategy and the inference engine. Still, this is already enough for a serious local AI service.

Small models in the 7B to 14B range are the easiest operational choice. Qwen, Gemma, Llama, OLMo and smaller Mistral-class models can support chat, document summarisation, classification, retrieval assistance, educational tools, citizen-service assistants and internal helpdesks. With vLLM or SGLang, continuous batching and short prompts, such a server can support many lightweight concurrent sessions. It is not a frontier lab, but it is more than enough for a university department, a municipality, a research group or a controlled internal public-service deployment.

The most attractive production tier is the 24B to 35B range. Models such as Devstral, OLMo 32B, Qwen3.5-35B-A3B and DeepSeek-R1-Distill-Qwen-32B offer a strong balance between capability and cost. With 4-bit quantization, they fit comfortably on 4× RX 7900 XTX while leaving room for KV cache. This is the practical sweet spot for many public-sector and research workloads: legal and administrative retrieval, code assistance, technical support, Greek-language fine-tunes, structured drafting, and RAG over official corpora. In realistic interactive use, the same server can serve roughly tens of active sessions, depending on the length of input and output.

The 70B tier is possible, but less forgiving. Apertus 70B, Llama 70B, some Qwen 72B variants and larger multimodal models require careful quantization, context discipline and lower concurrency. They should be treated as heavier expert models for selected users or more demanding tasks, not as the default model for mass service. Models above 100B, including gpt-oss-120B, may fit under specific quantized formats, but leave limited memory for long context and concurrent users. Very large frontier MoE systems, such as full DeepSeek-V3/R1, GLM-5.2 or Kimi K2.6, are not a realistic everyday target for a 4× RX 7900 XTX machine. They belong either on larger clusters or should be accessed through smaller distilled variants.

An 8× RX 7900 XTX system doubles the aggregate memory to 192 GB. This makes 30B to 35B models much more comfortable, improves the experience of 70B deployments and makes some 120B quantized models more plausible for limited-concurrency reasoning workloads. It still does not replace enterprise GPU clusters with high-bandwidth interconnects and data-centre accelerators. The lesson is simple: 4× cards are enough for a strong local service, 8× cards are enough for a serious institutional node, but full frontier MoE inference remains a data-centre problem.

The hardware matters, but the stack matters more. A practical AMD-based local AI server can run Ubuntu Server or Rocky Linux, ROCm for AMD acceleration, PyTorch on ROCm, vLLM or SGLang for production inference, llama.cpp for GGUF and lighter deployments, LiteLLM as an OpenAI-compatible gateway, Open WebUI as the user interface, Keycloak for identity, OpenSearch or Qdrant for retrieval, PostgreSQL for metadata and Grafana or Superset for monitoring.

This stack reduces vendor lock-in. Applications call a stable API while models can be changed behind the gateway. Data can remain local. Logs can be retained for audit. RAG can force answers to be grounded in official documents. Costs can be measured per request. Model Cards, Dataset Datasheets and service dashboards can make the system inspectable. A ministry, university or municipality can therefore treat AI not as a subscription product, but as a governed digital commons.

Local open models are not just a cheaper way to imitate commercial chatbots. They are a way to build public AI capacity. A municipality can provide assistance without sending sensitive citizen queries to a third-party cloud. A university can give students and researchers access to AI tools under clear academic rules. A ministry can run RAG over laws, circulars, procurement data and administrative procedures without surrendering institutional knowledge to opaque platforms.

The right architecture is hybrid and layered: many small and medium models for routine tasks, one stronger model for complex analysis, strict RAG for legal and administrative answers, human responsibility for final decisions, and full technical documentation for every critical workflow. In this model, a low-cost AMD server is not a replacement for national supercomputers. It is the missing middle layer between laptops and hyperscale data centres. It turns AI from a rented service into a public, auditable and reusable infrastructure.