The local AI workflow I would start with is an incident assistant for an integration team. An operator selects a failed flow, and the assistant prepares a short note containing the observed error, relevant runbook passages and a suggested next check. That is a small enough job to test properly.

The operator can open the sources and decide what to do.

The attraction of running it locally is straightforward: the selected operational context can stay on company infrastructure, and a repeated summary has no external model token charge. There is still a machine to operate and software to maintain. I would make the first version useful on one class of incident before trying to give it access to the whole estate.

Define the result before choosing the model

Take a warehouse reservation timeout. The useful output would identify the affected integration, when the timeout occurred, whether a booking or reservation reference was received, and the runbook section for checking an uncertain outcome. It should distinguish recorded facts from suggested explanations. An invented statement that the warehouse is down could send the operator in the wrong direction.

I would ask for a compact structured result: summary, observedFacts, suggestedChecks and sourceIds. The application validates that shape and verifies that cited source IDs were actually supplied to the model. A valid structure alone says nothing about whether the explanation is correct, so the interface should show the original error and cited passages beside the draft.

A concrete first implementation

One possible stack is an internal ASP.NET Core application, a PostgreSQL database holding approved documentation and incident metadata, and Ollama serving a local Qwen3 8B model. This is a proposed starting point, not a claim that I have benchmarked it for every company. The model needs to fit in memory with the chosen context size, and the team needs to test its language and task quality.

The application handles identity and retrieves context before calling the model. Ollama should be reachable only from the application or a controlled administration path. The browser does not need direct access to the model server. I would begin with one concurrent generation, a bounded queue, a request timeout and a clear busy response when capacity is exhausted.

Each request should include an integration ID and a bounded time window. The server retrieves a small set of relevant errors and the approved runbook for that integration. It then sends only that material and the task instructions to the model. That keeps the first implementation understandable and gives us a way to inspect exactly what produced an answer.

Start retrieval with identifiers and text search

For an error code such as WH-RESERVATION-TIMEOUT, an exact match against a maintained runbook is a good first retrieval method. PostgreSQL has full-text search for broader document queries. I would combine explicit integration and error-code filters with text search before adding a vector index.

Each stored passage needs a document ID, version, source URL or path, last-reviewed date and access information. When a runbook changes, the index should update or remove old passages. Otherwise the assistant can give technically plausible advice for a retired version of the integration.

Embeddings may help when people describe the same issue in different words. I would add them if a test set shows that useful passages are being missed. That introduces another model, an indexing job and a decision about chunk size. Those are manageable tasks, but I want a measured retrieval problem to justify them.

Enforce access before building the prompt

Incident text and documents can contain misleading instructions, either deliberately or by accident. I would treat them as data and keep the first assistant read-only. It gets no shell, database write access or replay endpoint. A suggested command can be displayed as a draft for review. Executing it would be a separate feature with explicit authorization and an audit trail.

Logs also need filtering before indexing. Credentials, access tokens and full customer payloads usually add risk without improving the incident explanation. An allowlist of useful fields is easier to reason about than hoping a later redaction step catches everything.

Measure what the operator can use

I would collect thirty representative, sanitized incidents and write the expected facts and useful source passages for each. Include missing logs, contradictory timestamps, a retired runbook and an error with no known procedure. Someone familiar with the integrations should judge whether the suggested next check is supported by the evidence.

For each run, record:

• Retrieval quality and unsupported claims.
• Edits needed and response time.
• Whether the operator could finish the note sooner.

Compare against opening the runbook directly. If retrieval finds the wrong procedure, changing the generation model may leave the underlying problem untouched.

Record model identity, prompt version and document versions with the test results. Run the same examples when any of them changes. Keep the previous model and configuration available so an update can be rolled back if familiar incidents start producing worse advice.

Budget for the complete request

Local cost includes hardware, electricity, storage, patching and the time spent maintaining the documents and application. Measure electricity at the wall over a representative workload. For capacity, include retrieval, prompt processing, generation and time waiting in the queue. A generation-only tokens-per-second result overstates what an end user receives across the complete workflow.

Ollama’s generation response exposes timing and token-count fields that can help with these measurements. I would also record total request duration at the application boundary. Compare cost per accepted incident note, including retries, against any external alternative being considered.

An external model can remain an explicitly selected option for approved data. Automatic fallback deserves particular care: a local outage must not silently change where incident information is processed. The workflow should have a visible failure mode when a request cannot be completed under its data-handling rules.

The first delivery

I would deliver one working incident type, a small approved document set, a quality report, a restore procedure and a list of cases the assistant cannot handle yet. That gives the team something they can use and improve. Additional integrations can follow once their runbooks, access rules and examples are ready.

The company then owns a useful piece of application logic: selecting context, controlling access, recording evidence and reviewing a draft. A model replacement still needs testing, but those parts can remain in place. That is the practical reason I would put effort into a local workflow.