# How to Make Your AI Model ONNX-Compatible for Microsoft Foundry Local

I recently presented on [**Foundry Local**](https://www.foundrylocal.ai/) at the [Azure User Group Sweden meetup](https://www.meetup.com/azureusergroupsundsvallsverige/events/316100227), showing how to run AI models entirely on-device: no cloud, no API keys, no per-token costs. The demo landed well, but the best moment came during Q&A, when someone asked the question every developer eventually hits:

> "This is great for the models you ship. But how do I make *my* model ONNX-compatible so I can run it with Foundry Local?"

Great question! This blog post is the detailed answer.

🎥 [Watch the recording](https://www.youtube.com/watch?v=I3SJPdB4Te0)

* * *

## TL;DR

If you only remember five things:

1.  Foundry Local runs **ONNX Runtime (ORT)** under the hood, so anything ORT can load, Foundry Local can serve.
    
2.  For most models, one command with the **ORT GenAI Model Builder** is all you need.
    
3.  Reach for **Olive** when you need GPTQ/AWQ quantization, NPU targeting, or fine-tuning in the same pipeline.
    
4.  Start at **INT4** with `block_size=32`. It is the sweet spot for on-device.
    
5.  Conversion is not the finish line. You still need an `inference_model.json` to register the model with Foundry Local.
    

**Who this is for:** developers who have a fine-tuned or custom model and want it running locally, and anyone evaluating whether their architecture is supported before investing a weekend in it.

* * *

## Understanding the Ecosystem

Conversion goes badly when you do not know which component owns which problem. So before any commands, here is the map and the vocabulary.

### 🔤 Jargon Buster: Key Terms

| Term | Definition |
| --- | --- |
| **ONNX** | Open Neural Network Exchange, an open format for representing machine learning models. Think of it as a universal translator between ML frameworks. |
| **ONNX Runtime (ORT)** | Microsoft's high-performance inference engine that runs ONNX models across different hardware (CPU, GPU, NPU). |
| **ONNX Runtime GenAI** | A layer on top of ORT that adds what language models need: KV caching, sampling, beam search, and a token-generation loop. |
| **Foundry Local** | Microsoft's end-to-end solution for shipping on-device AI in your applications, with a lightweight runtime and an OpenAI-compatible endpoint. |
| **Quantization** | Reducing model precision (for example, 32-bit to 4-bit) to shrink size and speed up inference, with minimal quality loss. |
| **Execution Provider (EP)** | The hardware backend ORT uses: CPU, CUDA (NVIDIA GPU), DirectML (Windows GPU), QNN/OpenVINO/VitisAI (NPU). |
| **Inference** | Running a trained model to get predictions, as opposed to training the model. |

### How Foundry Local Works Under the Hood

![](https://cdn.hashnode.com/uploads/covers/651bff05e4455a8ac9ec7688/04c93a7a-f382-49a9-a07a-87accd8f681b.png align="center")

**Key insight:** Foundry Local uses **ONNX Runtime** under the hood. That single fact simplifies everything that follows. You are not really targeting Foundry Local, you are targeting ORT. If ORT can load your model, Foundry Local can serve it.

* * *

## Two Paths to ONNX Conversion

With that established, the next decision is which tool does the converting. There are two, and the choice is mostly about how much control you need.

![](https://cdn.hashnode.com/uploads/covers/651bff05e4455a8ac9ec7688/7a89be26-e919-489c-ac2e-0b886305f482.png align="center")

| Tool | Best For | Complexity |
| --- | --- | --- |
| **ORT GenAI Model Builder** | Quick conversion of supported architectures | ⭐ Simple |
| **Microsoft Olive** | Advanced optimization, fine-tuning, custom pipelines | ⭐⭐⭐ Advanced |

My advice: always try the Model Builder first. It takes one command to find out whether your architecture is supported, and if it works you are done. Olive is the right answer for the cases where it does not.

We will cover both.

* * *

## Prerequisites

### System Requirements

*   **Python 3.10+** (3.11 recommended)
    
*   **16GB+ RAM**, and comfortably more than the unquantized model size, because conversion loads full-precision weights before quantizing
    
*   **GPU** (optional, speeds up conversion)
    
*   **50GB+ free disk space** (you will hold the source weights and the output at the same time)
    

### Install Required Packages

```python
# Create and activate virtual environment
python -m venv .venv
.\.venv\Scripts\Activate.ps1

# Install core packages
pip install onnxruntime-genai
pip install transformers torch
pip install huggingface_hub

# For Olive (optional, for advanced optimization)
pip install olive-ai
```

### 🔤 Jargon Buster: Virtual Environment

A **virtual environment** is an isolated Python installation. It keeps your project's dependencies separate from other projects, preventing version conflicts. Always use one for ML projects.

* * *

## Method 1: ONNX Runtime GenAI Model Builder

The Model Builder is Microsoft's official tool for converting models into the ONNX GenAI format. It is not a general-purpose converter, and that is exactly why it works so well: it knows the shape of each supported architecture and generates a graph that ORT GenAI is already tuned to run fast.

### Supported Model Architectures

That design has one consequence worth checking before you spend an hour downloading weights. Your architecture must be on the list.

| Architecture | Example Models | Status |
| --- | --- | --- |
| **Llama** | Llama 3, Llama 3.1, CodeLlama | ✅ Supported |
| **Phi** | Phi-3, Phi-3.5, Phi-4 | ✅ Supported |
| **Qwen** | Qwen2, Qwen2.5 | ✅ Supported |
| **Mistral** | Mistral 7B, Mixtral | ✅ Supported |
| **DeepSeek** | DeepSeek-R1, DeepSeek distills | ✅ Supported |
| **Gemma** | Gemma, Gemma 2 | ✅ Supported |
| **Granite / OLMo / ChatGLM** | Various | ✅ Supported |
| **Whisper** | whisper-tiny to whisper-large | ✅ Supported |
| **Custom or research architectures** | Your own model class | ❌ Needs Olive or a hand-written exporter |

> 💡 **Tip:** This list moves fast. Check the [onnxruntime-genai repository](https://github.com/microsoft/onnxruntime-genai) for the current set before concluding your model is unsupported. What matters is the `model_type` field in your `config.json`, not the name on the Hugging Face card. A model fine-tuned from Llama is still a Llama.

### Step 1: Identify Your Source Model

The Model Builder accepts inputs from several places:

![](https://cdn.hashnode.com/uploads/covers/651bff05e4455a8ac9ec7688/4e775af9-1507-4b9b-8a68-847c9b9133d5.png align="center")

### Step 2: Basic Conversion from Hugging Face

Start with a known-good model so you can confirm your environment works before pointing the tool at your own weights. Phi-3-mini is small and fast to convert:

```powershell
python -m onnxruntime_genai.models.builder `
    -m microsoft/Phi-3-mini-4k-instruct `
    -o ./phi3-onnx `
    -p int4 `
    -e cpu
```

**Parameter breakdown:**

| Parameter | Value | Meaning |
| --- | --- | --- |
| `-m` | `microsoft/Phi-3-mini-4k-instruct` | Hugging Face model ID |
| `-o` | `./phi3-onnx` | Output directory |
| `-p` | `int4` | Precision (quantization level) |
| `-e` | `cpu` | Target execution provider |

### 🔤 Jargon Buster: Precision Levels

| Precision | Bits per Weight | Model Size | Quality | Speed |
| --- | --- | --- | --- | --- |
| `fp32` | 32 bits | Largest | Best | Slowest |
| `fp16` | 16 bits | Large | Excellent | Fast |
| `int8` | 8 bits | Medium | Very Good | Faster |
| `int4` | 4 bits | Smallest | Good | Fastest |

**Rule of thumb:** start with `int4` for on-device deployment. It gives the best size-to-quality ratio for most use cases, and if the output disappoints you can step up to `int8` without changing anything else in your pipeline.

### Step 3: Conversion with Advanced Options

The basic command is deliberately conservative. When you are ready to trade conversion time for output quality, or you are targeting specific hardware, the extra options are where the real tuning happens:

```powershell
python -m onnxruntime_genai.models.builder `
    -m microsoft/Phi-3-mini-4k-instruct `
    -o ./phi3-onnx-optimized `
    -p int4 `
    -e cpu `
    -c ./cache `
    --extra_options `
        accuracy_level=4 `
        block_size=32 `
        algo_config=k_quant
```

**Extra options explained:**

| Option | Values | Purpose |
| --- | --- | --- |
| `accuracy_level` | 1-4 | Higher = better quality, slower conversion |
| `block_size` | 32, 64, 128 | Quantization granularity (smaller = better quality) |
| `algo_config` | `default`, `rtn`, `k_quant` | Quantization algorithm |
| `-c` | path | Cache directory for downloaded files |

### 🔤 Jargon Buster: Quantization Algorithms

*   **RTN (Round-To-Nearest):** simple and fast. Rounds each weight to the nearest quantized value. Good default.
    
*   **K-Quant:** more sophisticated, borrowed from the llama.cpp ecosystem. Allocates precision based on how much each weight matters. Better quality, slower conversion.
    
*   **GPTQ:** calibration-based. Runs sample data through the model to decide how to quantize. Best quality, but you need a calibration dataset and Olive to run it.
    

### Step 4: Converting Local or Fine-tuned Models

Once the sample conversion succeeds, point the same tool at your own model. Swap `-m` (a Hugging Face ID) for `-i` (a local directory):

```powershell
python -m onnxruntime_genai.models.builder `
    -i ./my-finetuned-phi3 `
    -o ./finetuned-onnx `
    -p int4 `
    -e cpu `
    -c ./cache
```

The `-i` flag points to your local model directory, which must contain `config.json` and the weight files. Everything else stays the same.

### Step 5: Converting Models with LoRA Adapters

If you fine-tuned with LoRA, you have two artifacts instead of one. The Model Builder can merge them during conversion:

![](https://cdn.hashnode.com/uploads/covers/651bff05e4455a8ac9ec7688/cc8bd975-a998-4df4-bb9a-1dc779ef8823.png align="center")

### 🔤 Jargon Buster: LoRA

**LoRA (Low-Rank Adaptation)** is a technique for fine-tuning large models efficiently. Instead of updating all weights, LoRA trains small "adapter" matrices that modify the model's behavior. This makes fine-tuning faster and produces smaller files.

```powershell
python -m onnxruntime_genai.models.builder `
    -i ./base-model `
    -o ./merged-onnx `
    -p fp16 `
    -e cuda `
    --extra_options adapter_path=./my-lora-adapter
```

### Step 6: Converting GGUF Models

If your model already lives in the llama.cpp world, you can bring it across without going back to the original PyTorch checkpoint:

```powershell
python -m onnxruntime_genai.models.builder `
    -m model_name `
    -i ./model-f16.gguf `
    -o ./gguf-to-onnx `
    -p int4 `
    -e cpu `
    -c ./cache
```

> ⚠️ **Important:** only `fp16` and `fp32` GGUF files are supported. Pre-quantized GGUF files such as `Q4_K_M` cannot be converted directly, because the quantization has already been baked in using a scheme ONNX does not share.

### Understanding the Output

Whichever path you took, the output folder should look like this:

```plaintext
phi3-onnx/
├── model.onnx              # The converted model (main file)
├── model.onnx.data         # Model weights (if large)
├── genai_config.json       # Runtime configuration
├── tokenizer.json          # Tokenizer vocabulary
├── tokenizer_config.json   # Tokenizer settings
├── special_tokens_map.json # Special token definitions
└── added_tokens.json       # Additional tokens
```

**The** `genai_config.json` **file** is the one to understand. It tells ONNX Runtime GenAI how to drive your model, and it is plain JSON you can edit by hand when something needs adjusting:

```json
{
  "model": {
    "bos_token_id": 1,
    "eos_token_id": 2,
    "pad_token_id": 0,
    "type": "phi3",
    "vocab_size": 32064,
    "context_length": 4096
  },
  "search": {
    "do_sample": false,
    "max_length": 4096,
    "min_length": 0,
    "num_beams": 1,
    "temperature": 1.0,
    "top_p": 1.0,
    "top_k": 50
  }
}
```

* * *

## Method 2: Microsoft Olive (Advanced Optimization)

If the Model Builder covered your case, you can skip to testing. If it did not, this is where you go next.

**Olive** (ONNX LIVE) is Microsoft's model optimization toolkit. Where the Model Builder is one opinionated command, Olive is a pipeline you compose. Use it when you need:

*   Advanced quantization techniques (GPTQ, AWQ)
    
*   Fine-tuning and optimization in a single pipeline
    
*   NPU-specific optimization for Qualcomm, Intel, or AMD silicon
    
*   Custom optimization passes
    

### Why Olive?

![](https://cdn.hashnode.com/uploads/covers/651bff05e4455a8ac9ec7688/1f619975-7a09-4a48-ba30-5a733b5d1cf0.png align="center")

Learn more: [Why Olive?](https://microsoft.github.io/Olive/why-olive.html)

### Step 1: Install Olive

```powershell
pip install olive-ai
pip install transformers onnxruntime-genai
```

### Step 2: Quick Optimization with the Olive CLI

Despite the flexibility, the common case is still a single command. `auto-opt` picks a sensible pipeline for your target hardware:

```powershell
olive auto-opt `
    --model_name_or_path Qwen/Qwen2.5-0.5B-Instruct `
    --output_path ./qwen-olive `
    --device cpu `
    --provider CPUExecutionProvider `
    --use_model_builder `
    --precision int4
```

**What Olive does for you here:**

1.  Downloads the model from Hugging Face
    
2.  Captures and converts the graph to ONNX
    
3.  Applies transformer-specific graph optimizations
    
4.  Quantizes to INT4
    
5.  Emits a Foundry Local-compatible output folder
    

> 💡 **Tip:** swap `--device npu --provider QNNExecutionProvider` (Qualcomm), `OpenVINOExecutionProvider` (Intel), or `--device gpu --provider DmlExecutionProvider` (Windows GPU) to retarget the same command. This retargeting is the main reason to prefer Olive over the Model Builder for edge deployments.

### Step 3: Advanced Olive Configuration

When the CLI defaults are not enough, describe the pipeline explicitly in a config file:

```yaml
# olive-config.yaml
input_model:
  type: HfModel
  model_path: microsoft/Phi-3-mini-4k-instruct

systems:
  local_system:
    type: LocalSystem
    accelerators:
      - device: cpu
        execution_providers:
          - CPUExecutionProvider

passes:
  conversion:
    type: OnnxConversion
    target_opset: 17
  
  optimize:
    type: OrtTransformersOptimization
    model_type: phi3
    opt_level: 2
    
  quantize:
    type: OnnxMatMul4Quantizer
    block_size: 32
    is_symmetric: true

output_dir: ./phi3-olive-output
```

Run with:

```powershell
olive run --config olive-config.yaml
```

### When to Use Olive vs Model Builder

To summarise the choice:

| Scenario | Recommended Tool |
| --- | --- |
| Quick conversion of a supported HF model | Model Builder |
| Simple INT4 quantization | Model Builder |
| GPTQ or AWQ quantization | Olive |
| Fine-tuning and conversion in one pipeline | Olive |
| NPU targeting (QNN, OpenVINO, VitisAI) | Olive |
| Custom optimization passes | Olive |

* * *

## Choosing the Right Quantization Strategy

Whichever tool you use, you still have to pick a precision. This is the decision that most affects how your model feels to users, so it is worth a minute of thought rather than copying whatever the last tutorial used.

### Decision Flowchart

![](https://cdn.hashnode.com/uploads/covers/651bff05e4455a8ac9ec7688/9c0a8ed4-4b5f-497e-a0cf-7753f2e756aa.png align="center")

### Quantization Comparison

| Configuration | Model Size | Quality | Inference Speed | Best For |
| --- | --- | --- | --- | --- |
| fp16 | ~7GB (7B model) | ★★★★★ | ★★★☆☆ | Quality-critical apps |
| int8 | ~3.5GB | ★★★★☆ | ★★★★☆ | Balanced |
| int4 (block=128) | ~2GB | ★★★☆☆ | ★★★★★ | Edge devices |
| int4 (block=32) | ~2.2GB | ★★★★☆ | ★★★★☆ | Best balance |

### Execution Provider Selection

```powershell
# For NVIDIA GPU
-e cuda

# For Windows GPU (AMD, Intel, NVIDIA)
-e dml

# For CPU (universal)
-e cpu

# For web deployment
-e webgpu
```

* * *

## Testing Your Converted Model

A conversion that completes without errors is not the same as a conversion that works. Test in two stages: first prove the ONNX model generates sensible text on its own, then wire it into Foundry Local. Doing it in that order tells you immediately whether a problem is in the model or in the plumbing.

### Quick Test with Python

```python
import onnxruntime_genai as og

# Load your converted model
model = og.Model('./phi3-onnx')
tokenizer = og.Tokenizer(model)

# Configure generation
params = og.GeneratorParams(model)
params.set_search_options(max_length=200)

# Create prompt
prompt = "<|user|>\nWhat is the capital of Sweden?<|end|>\n<|assistant|>\n"
input_tokens = tokenizer.encode(prompt)

# Generate response
generator = og.Generator(model, params)
generator.append_tokens(input_tokens)

print("Response: ", end="", flush=True)
while not generator.is_done():
    generator.generate_next_token()
    new_token = generator.get_next_tokens()[0]
    print(tokenizer.decode([new_token]), end="", flush=True)

print()  # New line at end
```

If that prints a coherent answer, the conversion itself is sound. Now to get it into Foundry Local.

### Register the Model with Foundry Local

This is the step most guides skip, and it is where people get stuck. Foundry Local will not pick up a bare ONNX folder. It needs a manifest that tells it the model's name and how to format prompts.

Create an `inference_model.json` file **inside your model folder**, alongside `model.onnx`:

```json
{
  "Name": "my-phi3-custom",
  "PromptTemplate": {
    "assistant": "{Content}",
    "prompt": "<|user|>\n{Content}<|end|>\n<|assistant|>\n"
  }
}
```

The `prompt` template must match the chat format your model was trained on. Get this wrong and the model will still respond, just badly, which makes it a uniquely annoying bug to diagnose. Copy the format from the original model card's chat template.

Then point Foundry Local's cache at the parent folder and run it:

```powershell
# Point the cache at the directory CONTAINING your model folder
foundry cache cd C:\models

# Confirm Foundry Local can see it
foundry cache ls

# Run it
foundry model run my-phi3-custom
```

### Call It from Your Application

Foundry Local exposes an OpenAI-compatible endpoint, so any OpenAI client works. The SDK's job is simply to start the service and tell you which port it landed on:

```python
import openai
from foundry_local import FoundryLocalManager

# Starts the Foundry Local service and loads your model
manager = FoundryLocalManager("my-phi3-custom")

client = openai.OpenAI(
    base_url=manager.endpoint,
    api_key=manager.api_key  # not used locally, but required by the client
)

response = client.chat.completions.create(
    model=manager.get_model_info("my-phi3-custom").id,
    messages=[{"role": "user", "content": "Explain quantum computing simply."}]
)

print(response.choices[0].message.content)
```

Because the endpoint is OpenAI-compatible, porting an existing cloud-backed app is usually a two-line change: swap the base URL and the model name. That is the whole pitch for Foundry Local.

* * *

## Troubleshooting Common Issues

Most conversion problems fall into a handful of buckets, and nearly all of them announce themselves either at conversion time or at the very first token. Here is how to triage.

### Error Decision Tree

![](https://cdn.hashnode.com/uploads/covers/651bff05e4455a8ac9ec7688/c8bcf038-5f58-4f8e-ac8b-17be19301355.png align="center")

### Common Fixes

| Problem | Cause | Solution |
| --- | --- | --- |
| `KeyError: 'model_type'` | Unsupported architecture | Use Olive or wait for support |
| `CUDA out of memory` | Model too large for GPU | Use `-e cpu` or reduce layers |
| `Tokenizer not found` | Missing files | Re-run conversion, check output |
| `Slow first inference` | KV cache warming | Normal behavior, subsequent calls faster |
| `Poor output quality` | Over-quantization | Use `int8` or `fp16` instead |
| `Model not loading` | Corrupted conversion | Delete output, re-convert |

### Hugging Face Authentication

Some models require authentication:

```powershell
# Login to Hugging Face
huggingface-cli login

# Or pass token directly
python -m onnxruntime_genai.models.builder `
    -m meta-llama/Llama-3-8B-Instruct `
    -o ./llama3-onnx `
    -p int4 `
    -e cpu `
    --extra_options hf_token=your_token_here
```

* * *

## Complete Conversion Workflow

Putting it all together, here is what actually happens between your command and your first generated token:

![](https://cdn.hashnode.com/uploads/covers/651bff05e4455a8ac9ec7688/f8dcb9ef-ec64-40cc-ac52-376ed76f4937.png align="center")

* * *

## Best Practices Checklist

Everything above, compressed into the list I actually run through before shipping a converted model:

*   \[ \] **Verify architecture support** before downloading anything large
    
*   \[ \] **Convert a known-good model first** to validate your environment
    
*   \[ \] **Start with INT4** for fast iteration, then step up only if quality demands it
    
*   \[ \] **Test output quality** with prompts representative of your real workload
    
*   \[ \] **Compare against the original** model to catch quantization regressions
    
*   \[ \] **Verify the prompt template** in `inference_model.json` matches the model's chat format
    
*   \[ \] **Document your conversion parameters** so the build is reproducible
    
*   \[ \] **Keep the original model** for debugging and A/B comparison
    
*   \[ \] **Test on target hardware**, since CPU results do not predict GPU or NPU results
    
*   \[ \] **Set an appropriate** `max_length` to keep KV cache memory bounded
    

* * *

## Resources

### This Talk

*   [Recording — Azure User Group Sweden](https://www.youtube.com/watch?v=I3SJPdB4Te0)
    

### Official Documentation

*   [Foundry Local — official site](https://www.foundrylocal.ai)
    
*   [Foundry Local Documentation](https://learn.microsoft.com/en-us/azure/foundry-local/)
    
*   [Foundry Local GitHub](https://github.com/microsoft/foundry-local)
    
*   [ONNX Runtime GenAI](https://github.com/microsoft/onnxruntime-genai)
    
*   [Microsoft Olive](https://microsoft.github.io/Olive/) | [Why Olive?](https://microsoft.github.io/Olive/why-olive.html)
    

### Hands-on Learning

*   [Foundry Local Lab](https://github.com/Microsoft-foundry/foundry-local-lab) - Step-by-step exercises
    
*   [Olive Recipes](https://github.com/microsoft/olive-recipes) - Example configurations
    

### Community

*   [Microsoft Foundry Discord](https://aka.ms/foundry-local-discord)
    
*   [ONNX Runtime GitHub Discussions](https://github.com/microsoft/onnxruntime/discussions)
    

* * *

## Conclusion

When that question came up at the meetup, my honest reaction was that the answer sounds harder than it is. Converting a model for Foundry Local is not a research project. It is a short checklist:

1.  **Check architecture support.** Most popular families (Llama, Phi, Qwen, Mistral, Gemma) already work.
    
2.  **Try the Model Builder first.** One command usually produces a working model.
    
3.  **Escalate to Olive when you need to.** Fine-tuning, GPTQ, and NPU targeting live there.
    
4.  **Start at INT4.** Best size-to-quality ratio for on-device, and easy to walk back.
    
5.  **Do not forget** `inference_model.json`**.** Conversion gets you a model; the manifest gets you a served model.
    

The payoff is worth the afternoon. Once converted, your model runs entirely on the user's machine: no per-token bill, no network round trip, and no data leaving the device. For a lot of use cases, that last point is not an optimization. It is the whole reason the project exists.

So pick a model, run the conversion, and see how far the first command gets you. It is usually further than you expect.

* * *

*Questions, or hit an error this post did not cover? Drop a comment below, or find me at the next speaking event.*

* * *

## Appendix: Quick Reference Commands

### Convert Hugging Face Model (Basic)

```powershell
python -m onnxruntime_genai.models.builder -m MODEL_ID -o OUTPUT_DIR -p int4 -e cpu
```

### Convert with Quality Optimization

```powershell
python -m onnxruntime_genai.models.builder -m MODEL_ID -o OUTPUT_DIR -p int4 -e cpu --extra_options accuracy_level=4 block_size=32 algo_config=k_quant
```

### Convert Local Model

```powershell
python -m onnxruntime_genai.models.builder -i LOCAL_PATH -o OUTPUT_DIR -p int4 -e cpu
```

### Convert with LoRA

```powershell
python -m onnxruntime_genai.models.builder -i BASE_MODEL -o OUTPUT_DIR -p fp16 -e cuda --extra_options adapter_path=LORA_PATH
```

### Olive Quick Optimize

```powershell
olive auto-opt --model_name_or_path MODEL_ID --output_path OUTPUT_DIR --device cpu --provider CPUExecutionProvider --use_model_builder --precision int4
```

### Test Model

```python
import onnxruntime_genai as og
model = og.Model('./output')
tokenizer = og.Tokenizer(model)
# ... generate text
```

### Register and Run in Foundry Local

```powershell
foundry cache cd PARENT_FOLDER
foundry cache ls
foundry model run MODEL_NAME
```

* * *

## About the Author

**Siddhesh Prabhugaonkar** is a **Generative AI & Agentic AI Enablement and Adoption Specialist** with two decades as an Architect, Consultant, and Trainer across IT, Cloud, and Generative AI. He is a **Microsoft Certified Trainer**, a **Pluralsight Instructor**, and helps enterprises move from GenAI curiosity to production adoption at scale.

His consulting and training practice spans **GenAI, Azure, Microsoft Foundry, Foundry Local, ONNX Runtime, Anthropic Claude, GitHub Copilot, Amazon Q, Google Gemini, OpenAI Codex, Cursor, Windsurf**, and modern full‑stack engineering (.NET, MEAN, MERN). Notable engagements include GenAI enablement for **ADP**, IoT platform consulting for **IIT Bombay's E‑Yantra** program, and early work on Microsoft's Repository platform (which later became **Entity Framework**).

> *Empowering organizations and individuals to adopt, build, and scale with Generative AI, Cloud, and Modern Software Engineering.*

**Connect & explore:**

*   💼 LinkedIn — [linkedin.com/in/siddheshprabhugaonkar](https://www.linkedin.com/in/siddheshprabhugaonkar)
    
*   📝 Blog — [azureauthority.in](https://azureauthority.in/)
    
*   📬 Newsletter — [cloud-authority.com](https://cloud-authority.com/)
    
*   🎥 YouTube — [youtube.com/c/SiddheshPrabhugaonkar](https://www.youtube.com/c/SiddheshPrabhugaonkar)
    
*   🤝 Book a 1:1 on Topmate — [topmate.io/siddheshp](https://topmate.io/siddheshp)
    
*   🎓 Research Papers (Google Scholar) — [scholar.google.com](https://scholar.google.com/citations?user=TuqOYtwAAAAJ&hl=en)