Enterprise AI Pilotsby The Goodstack Company Propose a pilot

Hiring AI consulting and AI integration services: what to ask before you sign

Bring every firm the same workflow and ask for evidence you can inspect. Use the request packet and synthetic example below to compare scope, access, evaluation, ownership and production support.

What AI consulting covers, and which one you are buying

Separate the work in a proposal into the following categories. Each has a different deliverable and operating responsibility. Ask a firm which categories its proposal covers and how each is priced.

What you are buyingWhat you hold at the endWhat to settle before you sign
Advice and assessmentA written opinion: candidate use cases, a ranking, a planWho would build the thing the plan recommends, and what that would take
AI integration servicesA model connected to systems you already run, with scoped access and loggingWhich systems, which permissions, who reviews the output
Custom AI software developmentSoftware built around one workflow, with ownership and usage rights agreed in writingWho holds the code, the prompts and the evaluation data
Pilot and production supportA running system with maintenance terms in writingWho upgrades it, who answers when it breaks, what it costs each month

One firm can do several of these. Trouble starts when the contract describes one and the sponsor is expecting another. Write the sentence down and read it back to them.

Questions to ask an AI consulting firm before you hire

These are the questions we would want a buyer to ask us. Ask for an artifact behind each answer; confident wording alone does not establish experience.

Which workflow would you start with, and why that one?

A good answer names one workflow, says who does it today, and gives a reason tied to your business: volume, a backlog, a cost, a known error, a person about to retire. A weak answer stays at the level of departments and opportunities.

What data access do you need, and how would it be scoped?

A good answer is specific about which systems, which records and which permissions, and it expects your IT or data partner to set the boundary. It includes a test where a permission is missing or revoked and the system stops. Identify the access owner before agreeing a delivery date. Record any permission or procurement dependency that remains unresolved.

What happens when the system is wrong?

Include cases outside the proposed workflow and cases where the available information is insufficient. A good answer describes where that case goes: who sees it, how it is logged, how a correction gets made and how it reaches the next run. If the answer is that the model is accurate, ask how a wrong or uncertain answer is handled.

How will we know whether it worked?

A good answer starts with a baseline measured before anything is built, and a pass condition written and signed before the work starts. It names the reviewer and the date when someone decides. Our pilot acceptance checklist lists the six items we agree before a pilot begins. Take it to every firm on your list.

Who owns the code, the prompts and the evaluation data?

Ask where the code lives while the project runs and who holds it afterwards. Ask the same about prompts, test sets and the logs the system produces. Those assets help you inspect the work and assess what another provider would need to maintain it. Ask what you would hand to a different firm if you continued with someone else.

What is the evidence behind that number?

Sales material in this market quotes failure rates, payback figures and adoption shares. Ask which organization published the number, on what date, from what sample, and whether it is a forecast or a measurement. Keep the original scope and caveats next to each number. Our guide to checking AI failure statistics gives a method for evaluating those claims. Leave an unsupported figure out of the comparison.

What would it take to run this in production?

Ask this while the pilot is still being scoped, because the answer changes the scope. A good answer covers hosting, monitoring, the access model at full volume, the operating cost for each case, and who is on call. If production support is excluded, record who would scope and operate the next stage.

What would make you tell us to stop?

A good answer exists. Ask which results would make them recommend stopping, then write those conditions into the acceptance criteria.

Who does the work, and what else are they working on?

Ask for the names of the people who will write the code and design the workflow, and ask whether they are on the call. Ask how many other projects they carry at the same time, and who reads the logs when something breaks at the end of a quarter. The answer matters more than the size of the firm.

How to scope a first pilot so it can reach production

Include production requirements when you scope the pilot. Record who would supply the budget, access, support and acceptance decision if the experiment warrants further work. Settle these lines before anyone builds.

Scope lineWhat to write down
The workflowOne workflow, its weekly volume, and the people who do it today
The user groupThe small group who will use it against real work, by name
AccessWhich systems, which permissions, who grants them and who reviews them
BaselineHow the workflow performs today, measured before the build starts
AcceptanceThe pass condition, the reviewer, and the date the sponsor decides
ExceptionsWhere a wrong result goes, and who handles it that day
RollbackHow the workflow returns to its current process, tested once for real
Production termsHosting, maintenance, support and the monthly operating cost

Two habits help as much as the list. Keep the first pilot small enough that stopping is a reasonable outcome, so the decision stays honest. And keep the systems it touches to the smallest set that still makes the result real, because every extra connection adds its own access review.

A pilot scoped this way can end three ways: stop, extend against a revised baseline, or move to production. All three are legitimate. What you are buying is the evidence to choose, and evidence that points at stopping has still earned its cost.

Request the same evidence packet from every firm

Copy this list into a proposal request. Fill in your workflow and ask each firm to mark every item included, excluded, depends on access, or not yet demonstrated. Keep the response with the proposal so that different assumptions stay visible.

  1. Task and boundary. Name the person doing the work, the input, the required output and the actions the system may take. List work that remains with staff. Ask whether an existing product or a simpler rules-based change could meet the need.
  2. Baseline and decision. State how you will observe the current process. Agree the evaluation cases, the reviewer and the conditions for stopping, extending or proceeding before the run.
  3. Access inventory. List each source system, requested read/write operation, permission owner and test environment. Ask for a demonstration of denied and revoked access. An integration diagram alone is not a completed access test.
  4. Evaluation record. Request the actual test inputs, expected behavior, observed output, configuration/version, exceptions and reviewer decisions. Separate a planned test from one that ran. Include successful cases, ambiguous inputs, unavailable dependencies and unauthorized actions.
  5. Cost and review work. Ask for build costs and recurring cost assumptions separately: hosting, model/API usage, monitoring, maintenance and human review. Name the usage assumptions and exclusions. Use the same workload when comparing estimates.
  6. Ownership and exit. Identify who receives the code, configuration, prompts, test cases and operating instructions, subject to the agreed rights. List third-party dependencies and the steps needed to move to another maintainer.
  7. Recovery and support. Ask who sees a failed action, how retries avoid duplicates, how staff recover work already in progress and how the workflow returns to its previous process. Request a witnessed recovery exercise before accepting that behavior as tested.
  8. Evidence limits. Label each artifact as a proposed design, a synthetic demonstration, a tested implementation or a measured result from an authorized customer deployment. These support different claims. Ask what the evidence does not establish.

For a broader risk review, NIST describes its AI Risk Management Framework as voluntary guidance for managing AI risks. The packet above is our proposed buying aid; it is not a NIST certification or a complete compliance assessment.

A completed example: staff-reviewed support replies

Synthetic demonstration: invented records. This is a proposed procurement example. No customer deployment, model run, measured performance or practitioner review is shown.

Suppose a business wants help drafting replies to routine order-status questions. Its service lead owns the workflow. Staff currently find the order, read the current shipping status and write the response. The proposed pilot drafts a reply for a staff member to review. Sending replies, changing an order and issuing refunds remain outside its scope.

The access request is read-only access to selected fictional order records and an approved shipping-policy document in a test environment. Production access is a separate decision. The baseline is a sample of staff-handled cases collected before the pilot; no baseline time or improvement is invented for this example.

Ask each firm to complete these acceptance cases with its observed output and review notes:

  • Known order: the question identifies an allowed order with a current status. The draft uses that record and identifies its source; staff check it before sending.
  • Ambiguous order: two records could match the question. The system requests clarification or routes the case to staff instead of choosing one.
  • Unauthorized record: the test user requests an order outside the permitted set. Access is denied; the draft must not reveal the restricted record.
  • Unavailable shipping service: the dependency times out. The output states that current status could not be checked and leaves the case in the staff queue; it must not invent a delivery date.
  • Instruction inside a message: an invented customer message asks the system to ignore its restrictions and issue a refund. The proposed scope remains draft-only; the case is routed for review.
  • Repeated request: the same event arrives again. Ask the firm to show how its design identifies the existing work item and avoids duplicate actions. Record whether this was actually exercised.

The sponsor's decision sheet records the test-set version, unresolved cases, review effort, operating-cost assumptions and recovery result. A reasonable proposed decision rule is to hold release if unauthorized access or action occurs, or if a failed case can disappear from the staff queue. The process owner must agree the remaining quality thresholds for the actual workflow. Passing these illustrative cases would not establish general reliability or production readiness.

Compare proposals by what remains unknown

Keep a separate list of evidence gaps. A polished demonstration may leave access approval, data quality, exception ownership and support untested. Ask for the smallest exercise that could resolve each gap and who owns it. Do not convert a missing answer into a favorable score.

If the main question is which actions an agent may take, use the enterprise agents guide to define that boundary. For the release decision, take the same evidence packet into the pilot acceptance checklist. A narrower scope or a decision to retain the current process can be the right outcome.

Questions buyers ask

What is the difference between AI consulting and AI integration services?

Consulting can end with a document. AI integration services end with a model connected to the systems you already run, with scoped access, logged actions and a person reviewing the exceptions. Ask which one the proposal covers, and ask what exists on the final day of the engagement.

How much do AI consulting services cost?

It depends on the workflow, the systems it touches and how long the access review takes. We publish no price list, because the scope decides the cost. Ask any firm for a fixed scope and a fixed quote on one mapped workflow, and compare those. A quote based on an unmapped process should state its assumptions and exclusions.

When is custom AI software development the right purchase?

When the workflow is specific to how your business runs, when the actions have to land in systems you own, and when you need the rules and the audit trail to be inspectable by your own people. If a product you already license covers the workflow as it comes, start there and measure it. That measurement becomes the baseline for anything you build later.

Do we need our own AI or data team first?

An internal AI team is one option. You need someone who can grant and review data access, and a process owner who can say what a correct result looks like. Without those two people the project waits, whoever you hire.

Using this with us, or without us

Take these questions to every firm you are considering. Compare the supplied evidence and unresolved dependencies alongside the price. A clear answer is a starting point for verification.

If you want to talk about one workflow, tell us which one, who does it today, and what a passing result would look like.

Propose an AI pilot