Ask Questions Your Data Can Actually Answer

Define the decision, the records, and the denominator before asking for an explanation.

“Why did sales fall?” sounds like a clear question. A table of orders may be able to show whether a particular order total fell. It may contain nothing about website visitors, canceled purchases outside the system, advertising changes, prices, stock availability, or the reasons customers made their decisions.

AI can produce a story anyway. That is why the first analytical skill is deciding which question the available data can answer.

A strong analysis begins with a contract: what will be calculated, from which records, for which period and population, and what the result can legitimately mean.

Begin with the decision the answer could change

Ask what you would do differently after seeing the result. Would you investigate a checkout problem, revise staffing, check stock availability, or simply correct a report?

If no possible answer would affect an action or understanding, the question may need refinement. Conversely, a consequential decision may require more evidence than the convenient table provides.

For a fictional shop, the immediate decision is whether to investigate an apparent decline in average order size. That is narrower than explaining all business performance. The first analysis can establish whether the defined average changed; a later investigation can examine possible causes.

Google’s problem-framing material separates understanding the desired outcome from choosing a model and its evaluation measures. The same discipline is useful before ordinary analysis: identify the actual goal before deciding that machine learning is needed. See Google’s introduction to problem framing.

Read the schema as a description of the evidence

A schema tells you which fields exist and how records relate. A data dictionary explains what those fields mean. Both are needed.

Our fictional extract has one row per order and includes order_id, order_date, customer_id, status, and order_value. Values are in one fictional currency. Dates are local calendar dates under the shop’s stated reporting convention. Status is the recorded status at the extract’s fixed review date.

The packet contains all six order records for the two teaching months. It has no refund ledger, item costs, web sessions, advertising spend, or customer satisfaction measure.

“Order value” is the source field’s definition. It is not automatically collected cash, net revenue, profit, or lifetime customer value. Those quantities need their own inputs and accounting definitions.

Identify the unit of analysis

The unit of analysis is the thing represented by one analytical observation. It might be an order, customer, visit, item, machine, or day.

Here, June has two completed orders but only one customer with completed orders. July has three completed orders from three customers. Dividing the same total by orders and by customers answers different questions.

A table with one row per order item would need aggregation before it could be treated as one row per order. A table of website sessions would not automatically count distinct people. Write the unit in the analysis contract so the model cannot switch it halfway through the explanation.

When joining tables, preserve the intended unit. Article 097 will show how multiple related rows can silently multiply a total.

Define the metric and denominator precisely

Our initial metric is average value per completed order:

Sum of order values for completed orders ÷ number of completed orders.

Canceled orders are excluded by the stated rule. Missing values, if present, would need explicit treatment; they should not silently become zero.

The calculation is:

The average falls by 25 currency units, or approximately 33.3% relative to June. The completed-order total is unchanged. Completed-order count rises by 50%.

These statements can all be true. A report saying simply “sales fell 33%” would blur the distinction between total value and average value.

Do not calculate a percentage change without identifying its baseline. A fall from 75 to 50 is one-third of 75. A later rise from 50 back to 75 would be one-half of 50. The direction changes the denominator.

Check coverage before interpreting a comparison

The teaching packet states that it contains all orders for its defined periods. A real export may omit older records, selected channels, failed checkouts, or orders whose status changed after the export.

Record the extraction time and inclusion rule. A report based on current status may revise past-month counts as orders are canceled or completed. A report based on status at month end answers a different question and requires historical status information.

Month length matters for some comparisons. June and July do not have the same number of days. Comparing total monthly workload may be appropriate for one decision; comparing workload per open day may be appropriate for another. The per-order average above has an order denominator, but its interpretation can still depend on the mix of trading days and customers.

If the table covers one branch, do not describe the result as the whole company. If it covers completed orders, do not infer the behavior of people who never placed an order.

Separate four kinds of questions

Description asks what is recorded: “What was the average completed-order value in each month?”

Explanation asks what factors may account for a pattern: “Did a change in product mix contribute?” That needs item-level information or other evidence.

Prediction asks about an unknown outcome: “What order value should we expect next month?” That requires a forecasting design and evaluation appropriate to the horizon.

A causal question asks what would change under an intervention: “Would a new offer increase order value?” That requires a design capable of addressing alternative explanations, such as an appropriate experiment.

The same dataset can support a descriptive answer while being insufficient for the other three. This is a useful boundary, not a failure to analyze.

Ask AI to label each requested claim by type. If it proposes an explanation, require the supporting variables or a statement of what additional evidence would be needed.

Rewrite questions into answerable versions

The second rewrite is deliberately modest. A customer who does not purchase in one month is not necessarily lost. The expected buying cycle and observation window matter.

The third request cannot be repaired simply by dividing orders by the number of rows in the order table. Conversion needs a defined eligible opportunity or audience denominator that exists independently of successful orders.

Write an analysis contract

A compact contract for the teaching example reads:

“Using the complete six-row teaching extract, compare June and July completed-order count, completed-order value, and average value per completed order. Treat the recorded status at the stated extract date as authoritative for this exercise. Exclude canceled orders. Use order IDs as the counting unit and the supplied currency as the value unit. Report formulas and a hand-checkable table. Do not describe the totals as net revenue or infer why the average changed.”

Add a completion test: the monthly totals must reconcile to the included source rows, and every excluded row must have a stated reason.

This contract gives AI a specific task and gives the reviewer something to verify. It also makes future changes explicit. Adding refunds or item costs would revise the analysis rather than quietly change the meaning of an existing metric.

Choose the next measurement deliberately

After the first calculation, the shop might investigate product mix, discounts, or customer mix. Pick the evidence that would most directly resolve the decision.

If the concern is profit, adding item costs and appropriate expense definitions may matter more than adding a sentiment score. If the concern is checkout conversion, obtain reliable session or exposure records with a suitable joining and counting strategy.

More data is not automatically better. A new field is useful when its definition, timing, and coverage help answer the question. AI can help draft the measurement plan, but someone must verify that the source system actually records what the plan assumes.

A reusable prompt

Given this schema and data dictionary, identify which parts of my question are answerable. Define the decision, unit of analysis, metric formula, denominator, inclusion rules, comparison periods, and data coverage. Separate description, explanation, prediction, and causal claims. Calculate only supported measures, list missing evidence, and write an analysis contract with a reconciliation check. Do not rename a source field as profit, revenue, conversion, or churn without the required definition and inputs.

For students: challenge the question before calculating

Take a teacher-provided table and write three questions: one it can answer directly, one it can partly answer, and one it cannot answer. Explain the missing evidence for the latter two.

Students in business can focus on orders, customers, and profit definitions. Science students can distinguish specimens, repeated measurements, and experiments. Social-science students can examine which population a survey actually covers.

Ask AI to critique your questions after you have written them. Keep an example where the model’s proposed metric used the wrong denominator and explain your correction. That demonstrates analytical understanding more clearly than a polished paragraph of conclusions.

Practice: correct the monthly report

A fictional report says: “July sales fell by one-third, customers are less loyal, and marketing stopped working.” Rewrite it using only the six-row packet.

Show the included order IDs and calculations. List the additional data needed to investigate loyalty and marketing. Explain why the canceled June order is excluded from the defined completed-order metric but should remain in the source extract.

Completion check: You report completed-order totals of 150 in both months, averages of 75 and 50, and counts of two and three. You distinguish orders from customers, preserve the source-field meaning, and make no unsupported causal or loyalty claim.

Stretch: Design one new measurement that would address the most important unresolved decision. Specify the event, timestamp, unit, source owner, and expected coverage rather than simply asking for “more data.”