A confidence number in a profile report needs an evidence definition

Start by naming the number's unit

A numeric model score is a different object from a sourced fact and from a measured success rate. A sourced fact points to a public statement you can inspect; a measured success rate comes from validation on known cases; a score is an output whose meaning depends on what it was trained and tested to compare. When a report shows a number without saying which of these it is, record it as an undefined unit. Do not silently convert it into a probability about a person, because that conversion is an inference the report has not earned. [NIST AI 600-1: generative AI confabulation and source verification]

• Write the number exactly as displayed, with its label and any accompanying words.
• Note whether the report states a unit, a baseline, a range or a validation set.
• If any of those are missing, mark the number undefined rather than precise.

Ask three questions before interpreting

Ask what the number measures, what it was compared against, and whether any validation is supplied. A score might describe similarity to a stored example, agreement between sources, or an internal model preference; those are not probabilities about a person. Validation is the part that would let you read the number as calibrated. General AI risk guidance notes that performance should not be extrapolated from narrow, non-systematic or anecdotal assessments, and that generated output can appear confident while being inconsistent with its inputs. Apply that caution to any score presented without a stated basis. [NIST AI 600-1: generative AI confabulation and source verification]

• What does the number measure?
• What was it compared against?
• Is any validation supplied, and on what kind of cases?

Keep an undefined score apart from observed facts

Build two separate records. The first holds what you observed in the permitted public source: a username, a public post, an image, a stated affiliation. The second holds the report's numeric output and everything the report infers from it. Keep them visually apart so a score never lends borrowed authority to a statement that lacks its own source. Where the report is more definite than the source, write the missing qualifier next to the claim and reduce the claim to what the source actually supports.

Do not let a decimal imply validation

Precision in presentation is not evidence of accuracy. A number written to several decimal places can look calibrated while measuring nothing about the real world. Treat the format as cosmetic until the report defines the score's basis. Review and verify sources in any generated output before relying on it, because generated text can present unsupported content in a confident tone. If the report offers no definition, your worksheet should carry the word undefined beside the number rather than a probability estimate. [NIST AI 600-1: generative AI confabulation and source verification]

Record the exact limit and stop there

If the source is absent, private, mismatched or ambiguous, write that precise limit and stop the unsupported inference. Do not substitute private access, a similarity guess or a higher score for missing evidence. A useful limit names what was missing: the post was deleted, the image matched only loosely, the profile could belong to more than one person. Naming the gap lets you decide what to do next without turning uncertainty into a conclusion about someone.

• State the specific gap, not a general disclaimer.
• Do not treat a privacy setting, silence or matching photo as proof of identity or intent.
• If a lead matters, plan a lawful check through the public source itself, not a bypass.

A reader worksheet for one report

Make three columns: report statement, source basis, and status. In the source basis column, paste or paraphrase the public statement the report relies on. In the status column, choose sourced fact, inferred claim, or undefined score. Any row where the number appears alone stays in the undefined column. This is an editorial aid you write yourself; it is not a control offered by any platform. It keeps a hypothetical example honest: if a report showed 0.87 beside a public username, the worksheet would record a sourced username and an undefined numeric label, not an 87 percent chance that anything about the person is true.

A couple more questions